buildtest.config

Module Contents

Classes

SiteConfiguration

This class is an interface to buildtest configuration

Attributes

logger

buildtest.config.logger

class buildtest.config.SiteConfiguration(settings_file=None, verbose=None)

This class is an interface to buildtest configuration

The initializer will declare class variables in its initial state and resolve path to configuration file. Once file is resolved we will load the configuration using load().

Parameters:

settings_file (str, optional) – path to buildtest configuration file

property file

load()

Loads configuration file

resolve()

This method will resolve path to configuration file. The order of precedence is as follows:

1. command line argument via buildtest --config <path>

2. User Configuration: $HOME/.buildtest/config.yml

3. Default Configuration: $BUILDTEST_ROOT/buildtest/settings/config.yml

name()

Return name of matched system from configuration file

detect_system()

This method detects which system configuration to use by checking target hostname with list of hostname entries defined in hostnames property. If there is a match we set self._name to map to system name and load the target configuration by setting self.target_config to the desired system configuration.

If no system is found we raise an exception.

Raises:

ConfigurationError – If there is no matching system

validate(moduletool=None)

This method validates the site configuration with schema.

Parameters:

moduletool (bool, optional) – Check whether module system (Lmod, environment-modules) match what is specified in configuration file. Valid options are Lmod, environment-modules

_executor_check()

Validate executors

get_all_executors()

Return list of all executors

is_executor_disabled(executor)

_validate_container_executors()

_validate_local_executors()

Check local executor by verifying the ‘shell’ types are valid

_validate_lsf_executors()

This method validates all LSF executors. We check if queue is available and in Open:Active state.

_validate_slurm_executors()

This method will validate slurm executors, we check if partition, qos, and cluster fields are valid values by retrieving details from slurm configuration. These checks are performed on fields partition, qos or cluster if specified in executor section.

_validate_cobalt_executors()

Validate cobalt queue property by running `qstat -Ql <queue>. If its a non-zero exit code then queue doesn’t exist otherwise it is a valid queue.

_validate_pbs_executors()

Validate pbs queue property by running by checking if queue is found and queue is ‘enabled’ and ‘started’ which are two properties found in pbs queue configuration that can be retrieved using qstat -Q -f -F json. The output is in the following format

$ qstat -Q -f -F json
 {
     "timestamp":1615924938,
     "pbs_version":"19.0.0",
     "pbs_server":"pbs",
     "Queue":{
         "workq":{
             "queue_type":"Execution",
             "total_jobs":0,
             "state_count":"Transit:0 Queued:0 Held:0 Waiting:0 Running:0 Exiting:0 Begun:0 ",
             "resources_assigned":{
                 "mem":"0kb",
                 "ncpus":0,
                 "nodect":0
             },
             "hasnodes":"True",
             "enabled":"True",
             "started":"True"
         }
     }
 }

_validate_torque_executors()

get_profile(profile_name)

Return configuration for a given profile name

Parameters:

profile_name (str) – name of profile to retrieve

Returns:

dictionary containing a profile configuration

Return type:

dict