buildtest.buildsystem.base

BuilderBase class is an abstract class that defines common functions for any types of builders. Each type schema (script, compiler) is implemented as separate Builder.

ScriptBuilder class implements ‘type: script’ CompilerBuilder class implements ‘type: compiler’

Module Contents

Classes

BuilderBase

The BuilderBase is an abstract class that implements common functions for

class buildtest.buildsystem.base.BuilderBase(name, recipe, buildspec, executor, buildexecutor, testdir)

Bases: abc.ABC

The BuilderBase is an abstract class that implements common functions for any kind of builder.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

_build_setup(self)

This method is the setup operation to get ready to build test which includes getting unique build id, setting up metadata object to store test details such as where test will be located and directory of test. This section cannot be reached without a valid, loaded recipe.

_emit_command(self)

This method will return a shell command used to invoke the script that is used for tests that use local executors

_generate_unique_id(self)

Generate a unique build id using uuid.uuid4().

_get_burst_buffer(self, burstbuffer)

Get Burst Buffer directives (#BB) lines specified by BB property

Parameters

burstbuffer (dict, required) – Burst Buffer configuration specified by BB property

Returns

list of burst buffer directives

Return type

list

_get_data_warp(self, datawarp)

Get Cray Data Warp directives (#DW) lines specified by DW property.

Parameters

datawarp (dict, required) – Data Warp configuration specified by DW property

Returns

list of data warp directives

Return type

list

_get_environment(self, env)

Retrieve a list of environment variables defined in buildspec and return them as list with the shell equivalent command

Returns

list of environment variable lines to add to test script.

Return type

list

_get_scheduler_directives(self, bsub, sbatch, cobalt, pbs, batch)

Get Scheduler Directives for LSF, Slurm or Cobalt if we are processing test with one of the executor types. This method will return a list of string containing scheduler directives generally found at top of script. If test is local executor we return an empty list

Parameters

• bsub – bsub property from buildspec

• sbatch – sbatch property from buildspec

• cobalt – cobalt property from buildspec

• pbs – pbs property from buildspec

• batch – batch property from buildspec

_get_variables(self, variables)

Retrieve a list of variables defined in buildspec and return them as list with the shell equivalent command.

Returns

list of variables variable lines to add to test script.

Return type

list

_set_execute_perm(self, fname)

Apply chmod 755 to input file name. The path must be an absolute path to script

_set_metadata_values(self)

This method sets self.metadata that contains metadata for each builder object.

_write_build_script(self)

This method will write the build script used for running the test

_write_test(self)

This method is responsible for invoking generate_script that formulates content of testscript which is implemented in each subclass. Next we write content to file and apply 755 permission on script so it has executable permission.

build(self)

This method is responsible for invoking setup, creating test directory and writing test. This method is called from an instance object of this class that does builder.build().

complete(self)

This method is invoked to indicate that builder job is complete after polling job.

copy_stage_files(self)

Copy output and error file into test root directory since stage directory will be removed.

endtime(self)

This method is called upon termination of job, we get current time using datetime.datetime.now() and calculate runtime of job

abstract generate_script(self)

Build the testscript content implemented in each subclass

get_test_extension(self)

Return the test extension, which depends on the shell used. Based on the value of shell key we return the shell extension.

shell: bash –> sh (default)

Returns

returns test extension based on shell type

Return type

str

incomplete(self)

This method indicates that builder job is not complete after polling job either job was cancelled by scheduler or job failed to run.

run(self)

Run the test and record the starttime and start timer. We also return the instance object of type BuildTestCommand which is used by Executors for processing output and error

run_command(self)

Command used to run the build script. buildtest will change into the stage directory (self.stage_dir) before running the test.

runtime(self)

start(self)

Keep internal time for start of test. We start timer by calling Timer class

starttime(self)

This method will record the starttime when job starts execution by using datetime.datetime.now()

stop(self)

Stop timer of test and calculate duration.