buildtest.executors.setup

This module is responsible for setup of executors defined in buildtest configuration. The BuildExecutor class initializes the executors and chooses the executor class (LocalExecutor, LSFExecutor, SlurmExecutor, CobaltExecutor) to call depending on executor name.

Module Contents

Classes

BuildExecutor

A BuildExecutor is responsible for initialing executors from buildtest configuration

Attributes

logger

buildtest.executors.setup.logger

class buildtest.executors.setup.BuildExecutor(site_config, account=None, max_pend_time=None)

A BuildExecutor is responsible for initialing executors from buildtest configuration file which provides a list of executors. This class keeps track of all executors and provides the following methods:

• setup: This method will write executor’s before_script.sh that is sourced in each test upon calling executor.

• run: Responsible for invoking executor’s run method based on builder object which is of type BuilderBase.

• poll: This is responsible for invoking poll method for corresponding executor from the builder object by checking job state

Initialize executors, meaning that we provide the buildtest configuration that are validated, and can instantiate each executor to be available.

Parameters

• site_config (buildtest.config.SiteConfiguration) – instance of SiteConfiguration class that has the buildtest configuration

• account (str, optional) – pass account name to charge batch jobs.

• max_pend_time (int, optional) – maximum pend time in second until job is cancelled.

__str__(self)

Return str(self).

__repr__(self)

Return repr(self).

list_executors(self)

is_local(self, executor_type)

is_slurm(self, executor_type)

is_lsf(self, executor_type)

is_pbs(self, executor_type)

is_cobalt(self, executor_type)

get(self, name)

Given the name of an executor return the executor object which is of subclass of BaseExecutor

_choose_executor(self, builder)

Choose executor is called at the onset of a run and poll stage. Given a builder object we retrieve the executor property builder.executor of the builder and check if there is an executor object and of type BaseExecutor.

Parameters

builder (buildtest.buildsystem.base.BuilderBase) – An instance object of BuilderBase type

setup(self)

This method creates directory var/executors/<executor-name> for every executor defined in buildtest configuration and write scripts before_script.sh if the field before_script is specified in executor section. This method is called after executors are initialized in the class __init__ method.

load_builders(self, builders)

Adds builder objects into self.builders class variable. This method will only add objects that are instance of BuilderBase class.

Parameters

builder (buildtest.buildsystem.base.BuilderBase) – An instance object of BuilderBase type

run(self)

This method is responsible for running the build script for each builder async and gather the results. We setup a pool of worker settings by invoking multiprocessing.pool.Pool and use multiprocessing.pool.Pool.apply_sync() method for running test async which returns an object of type multiprocessing.pool.AsyncResult which holds the result. Next we wait for results to arrive using multiprocessing.pool.AsyncResult.get() method in a infinite loop until all test results are retrieved. The return type is the same builder object which is added to list of valid builders that is returned at end of method.

get_builders(self)

Return a list of valid builders that were run