buildtest.builders.base

BuilderBase class is an abstract class that defines common functions for any types of builders. Each type schema (script, compiler, spack) is implemented as separate Builder which extends BuilderBase class.

Module Contents

Classes

BuilderBase

The BuilderBase is an abstract class that implements common functions used for building and running test.

class buildtest.builders.base.BuilderBase(name, recipe, buildspec, executor, buildexecutor, testdir, numprocs=None, numnodes=None, compiler=None)

Bases: abc.ABC

The BuilderBase is an abstract class that implements common functions used for building and running test.

buildtest will create a builder object which resembles a test. The builder will contains metadata that is unique to each builder that is captured in the report file upon completion of test.

The BuilderBase provides common functions for any builder. The builder is an instance of BuilderBase. The initializer method will setup the builder attributes based on input test by name parameter.

Parameters:

• name (str) – Name of test in buildspec recipe

• recipe (str) – The loaded test section from the buildspec file

• buildspec (str) – Full path to buildspec file

• buildexecutor (buildtest.executors.setup.BuildExecutor) – An instance of BuildExecutor class used for accessing executors

• testdir (str) – Test directory where tests are written. Must be full path on filesystem.

property dependency

shell_detection()

Detect shell and shebang used for test script

_set_metadata_values()

This method sets self.metadata that contains metadata for each builder object.

get_test_extension()

Return the test extension, which depends on the shell type. By default we return sh file extension for all shells except for csh which will return “csh” extension.

Returns

str: Returns test extension name for generated test.

is_local_executor()

Return True if current builder executor type is LocalExecutor otherwise returns False.

Returns:

returns True if builder is using executor type LocalExecutor otherwise returns False

Return type:

bool

is_slurm_executor()

Return True if current builder executor type is LocalExecutor otherwise returns False.

Returns:

returns True if builder is using executor type LocalExecutor otherwise returns False

Return type:

bool

is_batch_job()

Return True/False if builder.job attribute is of type Job instance if not returns False. This method indicates if builder has a job submitted to queue

start()

Keep internal timer for test using class buildtest.utils.timer.Timer. This method will start the timer for builder which is invoked upon running test.

stop()

Stop internal timer for builder.

retry(retry)

build(modules=None, modulepurge=None, unload_modules=None)

This method is responsible for invoking setup, creating test directory and writing test. This method is called from an instance object of this class that does builder.build().

Parameters:

• modules (str, optional) – Specify a list of modules to load in the build script

• modulepurge (bool, optional) – A boolean to control whether ‘module purge’ is run before running test

• unload_modules (str, optional) – Specify a list of modules to unload in the build script

run(cmd, timeout=None)

Run the test and record the starttime and start timer. We also return the instance object of type BuildTestCommand which is used by Executors for processing output and error

Returns:

If success, the return type is an object of type buildtest.utils.command.BuildTestCommand

If their is a failure (non-zero) returncode we retry test and if it doesn’t pass we raise exception of buildtest.exceptions.RuntimeFailure

record_starttime()

This method will record the starttime when job starts execution by using datetime.datetime.now()

record_endtime()

This method is called upon termination of job, we get current time using datetime.datetime.now() and calculate runtime of job

runtime()

Calculate runtime of job by calculating delta between endtime and starttime. The unit of measure is seconds.

get_runtime()

Return runtime of test

state()

failed()

Mark test as failure by updating the self._state. A fail test will not be reported in test report

complete()

Mark test as complete by updating the self._state. A complete test assumes test ran to completion

running()

is_pending()

is_complete()

If builder completes execution of test this method will return True otherwise returns False. A builder could fail due to job cancellation, failure to submit job or raise exception during the run phase. In those case, this method will return False.

is_failed()

Return True if builder fails to run test.

is_running()

Return True if builder fails to run test.

copy_stage_files()

Copy output and error file into test root directory.

_build_setup()

This method is the setup operation to get ready to build test which includes the following:

1. Creating Test directory and stage directory

2. Resolve full path to generated test script and build script

3. Copy all files from buildspec directory to stage directory

_write_build_script(modules=None, modulepurge=None, unload_modules=None)

This method will write the content of build script that is run for when invoking the builder run method. Upon creating file we set permission of builder script to 755 so test can be run.

_write_test()

This method is responsible for invoking generate_script that formulates content of testscript which is implemented in each subclass. Next we write content to file and apply 755 permission on script so it has executable permission.

_emit_command()

This method will return a shell command used to invoke the script that is used for tests that use local executors

Returns:

a list to show generated command used to run test.

Test can be run without any argument with path to script: /path/to/script.sh Test can be run with shell name followed by path to script: bash /path/to/script.sh Test can be run with shell name, shell options and path to script: bash -x /path/to/script.sh

Return type:

list

_default_test_variables()

Return a list of lines inserted in build script that define buildtest specific variables that can be referenced when writing tests. The buildtest variables all start with BUILDTEST_*

sched_init()

This method will resolve scheduler fields: ‘sbatch’, ‘pbs’, ‘bsub’, ‘cobalt’

get_job_directives()

This method returns a list of lines containing the scheduler directives

_get_burst_buffer(burstbuffer)

Get Burst Buffer directives (#BB) lines specified by BB property

Parameters:

burstbuffer (str) – Burst Buffer configuration specified by BB property in buildspec

Returns:

List of string values containing containing #BB directives written in test

Return type:

list

_get_data_warp(datawarp)

Get Cray Data Warp directives (#DW) lines specified by DW property.

Parameters:

datawarp (str) – Data Warp configuration specified by DW property in buildspec

Returns:

List of string values containing containing #DW directives written in test

Return type:

list

_set_execute_perm(fname)

Set permission to 755 for a given file. The filepath must be an absolute path to file

_get_environment(env)

Retrieve a list of environment variables defined in buildspec and return them as list with the shell equivalent command

Parameters:

env (dict) – list of environment variables defined by env property in buildspec

_get_variables(variables)

Retrieve a list of variables defined in buildspec and return them as list with the shell equivalent command.

Parameters:

variables (dict) – list of variable defined by vars property in buildspec

add_metrics()

This method will update the metrics field stored in self.metadata['metrics']. The metrics property can be defined in the buildspdec to assign value to a metrics name based on regular expression, environment or variable assignment.

output()

Return output content

error()

Return error content

abstract generate_script()

Build the testscript content implemented in each subclass

post_run_steps()

This method is called after test is complete. This method will copy files from stage directory such as output, error and test script. We will check state of test and mark job is complete.

is_valid_metric(name)

check_test_state()

This method is responsible for detecting state of test (PASS/FAIL) based on returncode or regular expression.

_process_compiler_config()

This method is responsible for setting cc, fc, cxx class variables based on compiler selection. The order of precedence is config, default, then buildtest setting. Compiler settings in ‘config’ takes highest precedence, this overrides any configuration in ‘default’. Finally we resort to compiler configuration in buildtest setting if none defined. This method is responsible for setting cc, fc, cxx, cflags, cxxflags, fflags, ldflags, and cppflags.

__str__()

Return str(self).

__repr__()

Return repr(self).