buildtest.system

This module detects System changes defined in class BuildTestSystem.

Module Contents

Classes

BuildTestSystem	BuildTestSystem is a class that detects system configuration
Cobalt	The Cobalt class checks for Cobalt binaries and gets a list of Cobalt queues
LSF	The LSF class checks for LSF binaries and returns a list of LSF queues
PBS	The PBS class checks for Cobalt binaries and gets a list of Cobalt queues
Scheduler	This is a base Scheduler class used for implementing common methods for
Slurm	The Slurm class implements common functions to query Slurm cluster

Attributes

system

class buildtest.system.BuildTestSystem

BuildTestSystem is a class that detects system configuration

system

check(self)

Based on the module “distro” get system details like linux distro, processor, hostname, etc…

check_scheduler(self)

Check existence of batch scheduler and if so determine which scheduler it is. Currently we support Slurm, LSF, and Cobalt we invoke each class and see if its valid state. The checks determine if scheduler binaries exist in $PATH.

detect_module_tool(self)

Check if module tool exists, we check for Lmod or environment-modules by checking if environment variable LMOD_VERSION, MODULE_VERSION or MODULES_CMD exist. We check this with input specification in buildtest configuration. If user specifies lmod as the module tool but detected environment-modules, buildtest should pick this up and report this as part of configuration check

get(self)

class buildtest.system.Cobalt

Bases: Scheduler

The Cobalt class checks for Cobalt binaries and gets a list of Cobalt queues

binaries = ['qsub', 'qstat', 'qdel', 'nodelist', 'showres', 'partlist']

_get_queues(self)

Get all Cobalt queues by running qstat -Ql and parsing output

class buildtest.system.LSF

Bases: Scheduler

The LSF class checks for LSF binaries and returns a list of LSF queues

binaries = ['bsub', 'bqueues', 'bkill', 'bjobs']

_get_queues(self)

Return json dictionary of available LSF Queues and their queue states. The command we run is the following: bqueues -o 'queue_name status' -json which returns a JSON record of all queue details. $ bqueues -o ‘queue_name status’ -json

{

“COMMAND”:”bqueues”, “QUEUES”:2, “RECORDS”:[

{

“QUEUE_NAME”:”batch”, “STATUS”:”Open:Active”

}, {

“QUEUE_NAME”:”test”, “STATUS”:”Open:Active”

}

]

}

class buildtest.system.PBS

Bases: Scheduler

The PBS class checks for Cobalt binaries and gets a list of Cobalt queues

binaries = ['qsub', 'qstat', 'qdel', 'qstart', 'qhold', 'qmgr']

_get_queues(self)

Get queue configuration using qstat -Q -f -F json and retrieve a list of queues.

class buildtest.system.Scheduler

This is a base Scheduler class used for implementing common methods for detecting Scheduler details. The subclass implement specific queries that are scheduler specific. The Slurm, LSF, PBS and Cobalt class inherit from Base Class Scheduler.

logger

check(self)

Check if binaries exist binary exist in $PATH

class buildtest.system.Slurm

Bases: Scheduler

The Slurm class implements common functions to query Slurm cluster including partitions, qos, cluster. We check existence of slurm binaries in $PATH and return if slurm cluster is in valid state.

binaries = ['sbatch', 'sacct', 'sacctmgr', 'sinfo', 'scancel']

_get_clusters(self)

Get list of slurm clusters by running sacctmgr list cluster -P -n format=Cluster. The output is a list of slurm clusters something as follows:

$ sacctmgr list cluster -P -n format=Cluster
cori
escori

_get_partitions(self)

Get list of all partitions slurm partitions using sinfo -a -h -O partitionname. The output is a list of queue names

$ sinfo -a -h -O partitionname system system_shared debug_hsw debug_knl jupyter

_get_qos(self)

Retrieve a list of all slurm qos by running sacctmgr list qos -P -n  format=Name. The output is a list of qos. Shown below is an example output

$ sacctmgr list qos -P -n format=Name normal premium low serialize scavenger

buildtest.system.system