Command Line Interface to buildtest configuration (buildtest config)
Once you have implemented your buildtest configuration, you can query the configuration
details using buildtest config command. Shown below is the command usage.
buildtest config --help
$ buildtest config --help
usage: buildtest [options] [COMMANDS] config [-h] ...
options:
-h, --help show this help message and exit
subcommands:
Query information from buildtest configuration file
edit (e) Open configuration file in editor
path (p) Show path to configuration file
systems List all available systems
validate (val)
Validate buildtest settings file with schema.
view (v) View configuration file
profiles (prof)
Query profile from buildtest configuration
executors (ex)
Query executors from buildtest configuration
compilers (co)
Search compilers
Note
buildtest cg is an alias for buildtest config command.
Validate buildtest configuration (buildtest config validate)
First thing you should do once you implement your configuration file is to make sure your configuration is valid with the schema.
This can be achieved by running buildtest config validate. When you invoke this
command, buildtest will load the configuration and attempt to validate the file with
schema settings.schema.json. If validation is successful you will get the following message:
$ buildtest config validate
/tmp/tmpg3p4j9in/config.yml is valid
Note
If you defined a user setting (~/.buildtest/config.yml) buildtest will validate this file instead of default one.
If there is an error during validation, the output from jsonschema.exceptions.ValidationError
will be displayed in terminal. For example the error below indicates that
moduletool property was expecting one of the values
[environment-modules, lmod, N/A] but it received a value of none:
Invalid buildtest configuration
$ buildtest config validate Traceback (most recent call last): File "/Users/siddiq90/Documents/buildtest/bin/buildtest", line 17, in <module> buildtest.main.main() File "/Users/siddiq90/Documents/buildtest/buildtest/main.py", line 39, in main buildtest_configuration = check_settings(settings_file, retrieve_settings=True) File "/Users/siddiq90/Documents/buildtest/buildtest/config.py", line 41, in check_settings validate(instance=user_schema, schema=config_schema) File "/Users/siddiq90/.local/share/virtualenvs/buildtest-1gHVG2Pd/lib/python3.7/site-packages/jsonschema/validators.py", line 934, in validate raise error jsonschema.exceptions.ValidationError: 'none' is not one of ['environment-modules', 'lmod', 'N/A'] Failed validating 'enum' in schema['properties']['moduletool']: {'description': 'Specify modules tool used for interacting with ' '``module`` command. ', 'enum': ['environment-modules', 'lmod', 'N/A'], 'type': 'string'} On instance['moduletool']: 'none'
View buildtest configuration (buildtest config view)
If you want to view buildtest configuration you can run buildtest config view which will print content of buildtest configuration.
buildtest config view
$ buildtest config view
───────────────────────── /tmp/tmpg3p4j9in/config.yml ──────────────────────────
1 system:
2 generic:
3 # specify a list of hostnames that is a regular expression where buildtest can run.
4 hostnames: ['.*']
5 # system description
6 description: Generic System
7 # specify module system to use. Supported module systems are [lmod, environment-modules, none]
8 moduletool: none
9
10 # specify size of job pool (https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool). This will configure the number of processes that can run in parallel.
11 # If not specified then buildtest will use all available cores on the system.
12 poolsize: 1
13
14 # maximum number of jobs that can run in parallel. If not specified, buildtest will run all jobs in parallel.
15 #max_jobs: 1
16
17 # test timeout in number of seconds
18 # timeout: 3600
19
20 # enable pagination for buildtest
21 pager: false
22
23 # options for buildtest buildspec find command
24 buildspecs:
25 # determine whether to rebuild buildspec cache
26 rebuild: false
27 # determine number of records to display
28 count: 15
29
30 # specify format fields
31 #format: name,description
32
33 # display output in terse mode
34 terse: false
35
36 # list of paths to search for buildspecs where it can be file or directory
37 # search: ["$BUILDTEST_ROOT/tutorials/containers", "$BUILDTEST_ROOT/examples", "demo.yml"]
38
39 # options for buildtest report command
40 report:
41 # number of records to display
42 count: 25
43
44 # specify format fields
45 format: name,id,state,runtime,returncode
46
47 # specify directory paths to search for binaries
48 #paths:
49 # directory path to search for slurm binaries.
50 #slurm: "/usr/bin"
51
52 # directory path to search for lsf binaries.
53 #lsf: "/usr/bin"
54
55 # directory path to search for pbs binaries.
56 #pbs: "/usr/bin"
57
58 # directory path to search for torque binaries.
59 #torque: "/usr/bin"
60
61 # path to docker directory
62 #docker: "/usr/local/bin"
63
64 # path to singularity directory
65 #singularity: "/usr/local/bin"
66
67 # path to podman directory
68 #podman: "/usr/local/bin"
69
70 # start of executor configuration
71 executors:
72 # local executor is used to submit jobs on local machine. In this example we have 4 executors: bash, sh, csh, zsh that will submit jobs using bash, sh, csh, zsh shell respectively.
73 local:
74 bash:
75 description: submit jobs on local machine using bash shell
76 shell: bash
77 sh:
78 description: submit jobs on local machine using sh shell
79 shell: sh
80 csh:
81 description: submit jobs on local machine using csh shell
82 shell: csh
83 zsh:
84 description: submit jobs on local machine using zsh shell
85 shell: zsh
86 # specify compiler declaration
87 compilers:
88 compiler:
89 # declaration of all gcc compilers
90 gcc:
91 # name of compiler
92 builtin_gcc:
93 cc: gcc
94 fc: gfortran
95 cxx: g++
96 # specify CDASH configuration when using 'buildtest cdash upload'
97 cdash:
98 # CDASH server
99 url: https://my.cdash.org/
100 # name of CDASH project where to push test reports
101 project: buildtest
102 # specify the site name which should generally by name of your HPC system. This is used by CDASH to group test results by site
103 site: generic
104 # specify the build name which is used to group test results by build name
105 buildname: tutorials
106
The --theme option can be used to change the color theme of the output. The default theme is monokai and list of
available themes can be retrieved with tab completion for option buildtest config view --theme.
Check path to buildtest configuration file (buildtest config path)
If you want to check path to buildtest configuration file you can run buildtest config path which will print path of buildtest configuration file.
$ buildtest config path
/tmp/tmpg3p4j9in/config.yml
Edit buildtest configuration (buildtest config edit)
The buildtest config edit command is used to open buildtest configuration file in your preferred editor. buildtest will
use the environment EDITOR to get the preffered editor; however, one can override the environment variable via command line option
buildtest --editor.
View Executors (buildtest config executors list)
You can use the command buildtest config executors list to view executors from buildtest
configuration file. Shown below is the command usage
buildtest config executors list --help
$ buildtest config executors list --help
usage: buildtest [options] [COMMANDS] config executors list
[-h] [-j | -y | -d | -i | -a]
options:
-h, --help show this help message and exit
-j, --json View executor in JSON format
-y, --yaml View executors in YAML format
-d, --disabled Show disabled executors
-i, --invalid Show invalid executors
-a, --all Show all executors
You can run buildtest config executors list without any options and it will report a list of named executors that
you would reference in buildspec using the executor property.
$ buildtest config executors list
generic.local.bash
generic.local.sh
generic.local.csh
generic.local.zsh
If you want to see the executor details, you may want to use --json or --yaml option which will show the executor settings in YAML or JSON format.
Shown below is an example output
buildtest config executors list --yaml
$ buildtest config executors list --yaml
executors:
local:
bash:
description: submit jobs on local machine using bash shell
shell: bash
csh:
description: submit jobs on local machine using csh shell
shell: csh
sh:
description: submit jobs on local machine using sh shell
shell: sh
zsh:
description: submit jobs on local machine using zsh shell
shell: zsh
buildtest config executors list --json
$ buildtest config executors list --json
{
"executors": {
"local": {
"bash": {
"description": "submit jobs on local machine using bash shell",
"shell": "bash"
},
"sh": {
"description": "submit jobs on local machine using sh shell",
"shell": "sh"
},
"csh": {
"description": "submit jobs on local machine using csh shell",
"shell": "csh"
},
"zsh": {
"description": "submit jobs on local machine using zsh shell",
"shell": "zsh"
}
}
}
}
Note
The command options for buildtest config executors list are mutually exclusive, so if you
specify multiple options you will get the following error.
$ buildtest config executors list --json --yaml
usage: buildtest [options] [COMMANDS] config executors list
[-h] [-j | -y | -d | -i | -a]
buildtest [options] [COMMANDS] config executors list: error: argument -y/--yaml: not allowed with argument -j/--json
Remove Executors (buildtest config executors remove)
The buildtest config executors remove command will remove an executor from buildtest configuration file.
The positional arguments are the name of the executor you want to remove. Tab completion is available to retrieve
all available executors (buildtest config executors list --all).
Shown below we see tab completion on available executors that can be removed.
buildtest config executors remove
generic.local.bash generic.local.csh generic.local.sh generic.local.zsh
Let’s try listing all executors and remove generic.local.zsh executor, you will notice after deletion,
the configuration file is updated and the executor is no longer present.
Removing executor ‘generic.local.zsh’
$ buildtest --config $BUILDTEST_CI_DIR/config.yml config executors list --all
generic.local.bash
generic.local.sh
generic.local.csh
generic.local.zsh
$ buildtest --config $BUILDTEST_CI_DIR/config.yml config executors remove generic.local.zsh
Removing executor: generic.local.zsh
Updating configuration file: /tmp/tmpg3p4j9in/config.yml
Upon deletion, buildtest will validate the configuration before writing the changes back to disk, to ensure the configuration is valid. Shown below we demonstrate an example where we attempt to remove all executors from the configuration file. Buildtest expects there is atleast 1 executor definition for local executor.
buildtest config executors remove generic.local.bash generic.local.sh generic.local.csh generic.local.zsh
$ buildtest config executors remove generic.local.bash generic.local.sh generic.local.csh generic.local.zsh
Removing executor: generic.local.bash
Removing executor: generic.local.sh
Removing executor: generic.local.csh
Unable to remove executor: generic.local.zsh because there are no executors of type: local
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ in/buildtest:38 in <module> │
│ │
│ 35 │
│ 36 if __name__ == "__main__": │
│ 37 │
│ ❱ 38 sys.exit(main()) │
│ 39 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ os = <module 'os' from │ │
│ │ '/home/docs/.asdf/installs/python/3.10.14/lib/python3.… │ │
│ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ uildtest/main.py:118 in main │
│ │
│ 115 │ │ if args.config in ["compilers", "co"]: │
│ 116 │ │ │ compiler_cmd(args, configuration) │
│ 117 │ │ else: │
│ ❱ 118 │ │ │ config_cmd( │
│ 119 │ │ │ │ command_args=args, │
│ 120 │ │ │ │ configuration=configuration, │
│ 121 │ │ │ │ editor=buildtest_editor, │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='config', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ config='executors', executors='remove', │ │
│ │ executor_names=['generic.local.bash', │ │
│ │ 'generic.local.sh', 'generic.local.csh', │ │
│ │ 'generic.local.zsh']) │ │
│ │ buildtest_editor = '/usr/bin/vim' │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f448473ed70> │ │
│ │ parser = <buildtest.cli.BuildTestParser object at │ │
│ │ 0x7f44868466b0> │ │
│ │ report_file = None │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f448473e3b0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ uildtest/cli/config.py:27 in config_cmd │
│ │
│ 24 │ │
│ 25 │ handle_view_command(command_args, configuration) │
│ 26 │ handle_profiles_command(command_args, configuration) │
│ ❱ 27 │ handle_executors_command(command_args, configuration) │
│ 28 │ handle_validate_command(command_args, configuration, system) │
│ 29 │ handle_systems_command(command_args, configuration) │
│ 30 │ handle_edit_command(command_args, configuration, editor) │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ command_args = Namespace(subcommands='config', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ config='executors', executors='remove', │ │
│ │ executor_names=['generic.local.bash', │ │
│ │ 'generic.local.sh', 'generic.local.csh', │ │
│ │ 'generic.local.zsh']) │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f448473ed70> │ │
│ │ editor = '/usr/bin/vim' │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f448473e3b0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ uildtest/cli/config.py:66 in handle_executors_command │
│ │
│ 63 │ │ │ │ display_all=command_args.all, │
│ 64 │ │ │ ) │
│ 65 │ │ if command_args.executors in ["remove", "rm"]: │
│ ❱ 66 │ │ │ remove_executors(configuration, command_args.executor_name │
│ 67 │
│ 68 │
│ 69 def handle_validate_command(command_args, configuration, system): │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildexecutor = [buildtest-executor] │ │
│ │ command_args = Namespace(subcommands='config', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ config='executors', executors='remove', │ │
│ │ executor_names=['generic.local.bash', │ │
│ │ 'generic.local.sh', 'generic.local.csh', │ │
│ │ 'generic.local.zsh']) │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f448473ed70> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ uildtest/cli/config.py:383 in remove_executors │
│ │
│ 380 │ │ console.print(f"Removing executor: {name}") │
│ 381 │ │ write_back = True │
│ 382 │ │
│ ❱ 383 │ custom_validator( │
│ 384 │ │ configuration.config, schema_table["settings.schema.json"]["re │
│ 385 │ ) │
│ 386 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f448473ed70> │ │
│ │ exec_name = 'zsh' │ │
│ │ exec_type = 'local' │ │
│ │ executor_names = [ │ │
│ │ │ 'generic.local.bash', │ │
│ │ │ 'generic.local.sh', │ │
│ │ │ 'generic.local.csh', │ │
│ │ │ 'generic.local.zsh' │ │
│ │ ] │ │
│ │ name = 'generic.local.zsh' │ │
│ │ write_back = True │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/b │
│ uildtest/schemas/defaults.py:105 in custom_validator │
│ │
│ 102 │ assert isinstance(schema, dict) │
│ 103 │ │
│ 104 │ validator = Draft7Validator(schema, registry=registry) │
│ ❱ 105 │ validator.validate(recipe) │
│ 106 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ recipe = { │ │
│ │ │ 'system': { │ │
│ │ │ │ 'generic': { │ │
│ │ │ │ │ 'hostnames': ['.*'], │ │
│ │ │ │ │ 'description': 'Generic System', │ │
│ │ │ │ │ 'moduletool': 'none', │ │
│ │ │ │ │ 'poolsize': 1, │ │
│ │ │ │ │ 'pager': False, │ │
│ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ 'rebuild': False, │ │
│ │ │ │ │ │ 'count': 15, │ │
│ │ │ │ │ │ 'terse': False │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'report': { │ │
│ │ │ │ │ │ 'count': 25, │ │
│ │ │ │ │ │ 'format': 'name,id,state,runtime,returncode' │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'executors': {}, │ │
│ │ │ │ │ 'compilers': { │ │
│ │ │ │ │ │ 'compiler': { │ │
│ │ │ │ │ │ │ 'gcc': { │ │
│ │ │ │ │ │ │ │ 'builtin_gcc': { │ │
│ │ │ │ │ │ │ │ │ 'cc': 'gcc', │ │
│ │ │ │ │ │ │ │ │ 'fc': 'gfortran', │ │
│ │ │ │ │ │ │ │ │ 'cxx': 'g++' │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ } │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'cdash': { │ │
│ │ │ │ │ │ 'url': 'https://my.cdash.org/', │ │
│ │ │ │ │ │ 'project': 'buildtest', │ │
│ │ │ │ │ │ 'site': 'generic', │ │
│ │ │ │ │ │ 'buildname': 'tutorials' │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ } │ │
│ │ } │ │
│ │ schema = { │ │
│ │ │ '$id': 'settings.schema.json', │ │
│ │ │ '$schema': 'http://json-schema.org/draft-07/schema#', │ │
│ │ │ 'title': 'buildtest configuration schema', │ │
│ │ │ 'type': 'object', │ │
│ │ │ 'required': ['system'], │ │
│ │ │ 'additionalProperties': False, │ │
│ │ │ 'properties': { │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'patternProperties': { │ │
│ │ │ │ │ │ '^.*$': {'$ref': '#/definitions/system'} │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ }, │ │
│ │ │ 'definitions': { │ │
│ │ │ │ 'terse': { │ │
│ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ 'description': 'A boolean to determine whether │ │
│ │ to enable terse mode' │ │
│ │ │ │ }, │ │
│ │ │ │ 'format': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Determine the format fields to │ │
│ │ display when viewing table results' │ │
│ │ │ │ }, │ │
│ │ │ │ 'count': { │ │
│ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ 'description': 'Determine number of records to │ │
│ │ display in a table' │ │
│ │ │ │ }, │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'required': [ │ │
│ │ │ │ │ │ 'executors', │ │
│ │ │ │ │ │ 'moduletool', │ │
│ │ │ │ │ │ 'hostnames', │ │
│ │ │ │ │ │ 'compilers', │ │
│ │ │ │ │ │ 'buildspecs', │ │
│ │ │ │ │ │ 'report' │ │
│ │ │ │ │ ], │ │
│ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ 'hostnames': { │ │
│ │ │ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ │ │ 'description': 'Specify a list of │ │
│ │ hostnames to check where buildtest can run for the given │ │
│ │ syste'+8, │ │
│ │ │ │ │ │ │ 'items': {'type': 'string'} │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'description': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'system description │ │
│ │ field' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'poolsize': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Specify size of Process │ │
│ │ Pool for parallel processing using ``multiprocessing.Poo'+3 │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'max_jobs': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Maximum number of jobs │ │
│ │ that can be run at a given time for a particular executor' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'testdir': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify full path to │ │
│ │ test directory where buildtest will write tests.' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'logdir': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify location where │ │
│ │ buildtest will write log files' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'moduletool': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify modules tool │ │
│ │ used for interacting with ``module`` command. ', │ │
│ │ │ │ │ │ │ 'enum': [ │ │
│ │ │ │ │ │ │ │ 'environment-modules', │ │
│ │ │ │ │ │ │ │ 'lmod', │ │
│ │ │ │ │ │ │ │ 'none' │ │
│ │ │ │ │ │ │ ] │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'timeout': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Specify timeout duration │ │
│ │ in number of seconds' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'pager': { │ │
│ │ │ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ │ │ 'description': 'A boolean to determine │ │
│ │ whether to enable paging when viewing buildspec cache' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ │ │ 'description': 'Specify configuration │ │
│ │ for ``buildtest buildspec`` command', │ │
│ │ │ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ │ │ 'rebuild': { │ │
│ │ │ │ │ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ │ │ │ │ 'description': 'A boolean to │ │
│ │ determine whether to rebuild buildspec cache' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'count': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/count' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'format': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/format' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'terse': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/terse' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'search': { │ │
│ │ │ │ │ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ │ │ │ │ 'items': {'type': 'string'}, │ │
│ │ │ │ │ │ │ │ │ 'description': 'Specify a list │ │
│ │ of file paths to search buildspecs. This field can be used │ │
│ │ with `'+103 │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ ... +6 │ │
│ │ │ │ │ } │ │
│ │ │ │ }, │ │
│ │ │ │ 'cc': { │ │
│ │ │ │ │ 'description': 'Specify path to C compiler │ │
│ │ wrapper. You may specify a compiler wrapper such as `'+101, │ │
│ │ │ │ │ 'type': 'string' │ │
│ │ │ │ }, │ │
│ │ │ │ 'cxx': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Specify path to C++ compiler │ │
│ │ wrapper. You may specify a compiler wrapper such as'+103 │ │
│ │ │ │ }, │ │
│ │ │ │ 'fc': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Specify path to Fortran compiler │ │
│ │ wrapper. You may specify a compiler wrapper suc'+112 │ │
│ │ │ │ }, │ │
│ │ │ │ 'compiler_section': { │ │
│ │ │ │ │ 'description': 'A compiler section is composed │ │
│ │ of ``cc``, ``cxx`` and ``fc`` wrapper these are r'+50, │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ 'required': ['cc', 'cxx', 'fc'], │ │
│ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ 'cc': {'$ref': '#/definitions/cc'}, │ │
│ │ │ │ │ │ 'cxx': {'$ref': '#/definitions/cxx'}, │ │
│ │ │ │ │ │ 'fc': {'$ref': '#/definitions/fc'}, │ │
│ │ │ │ │ │ 'module': {'$ref': '#/definitions/module'} │ │
│ │ │ │ │ } │ │
│ │ │ │ }, │ │
│ │ │ │ 'unique_string_array': { │ │
│ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ 'uniqueItems': True, │ │
│ │ │ │ │ 'items': {'type': 'string'} │ │
│ │ │ │ }, │ │
│ │ │ │ 'disable': { │ │
│ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ 'description': 'Disable an executor' │ │
│ │ │ │ }, │ │
│ │ │ │ ... +11 │ │
│ │ │ } │ │
│ │ } │ │
│ │ validator = Draft7Validator( │ │
│ │ │ schema={'$id': 'settings.schema.json', '$schema': │ │
│ │ 'http://json-...ft-07/schema#', 'additionalProperties': │ │
│ │ False, 'definitions': {'account': {'description': 'Specify │ │
│ │ Job ...ing resources', 'type': 'string'}, 'cc': │ │
│ │ {'description': 'Specify path...iler wrapper.', 'type': │ │
│ │ 'string'}, 'compiler_section': {'additionalProperties': │ │
│ │ False, 'description': 'A compiler s...iler wrapper.', │ │
│ │ 'properties': {'cc': {'$ref': '#/definitions/cc'}, 'cxx': │ │
│ │ {'$ref': '#/definitions/cxx'}, 'fc': {'$ref': │ │
│ │ '#/definitions/fc'}, 'module': {'$ref': │ │
│ │ '#/definitions/module'}}, 'required': ['cc', 'cxx', 'fc'], │ │
│ │ ...}, 'container': {'additionalProperties': False, │ │
│ │ 'properties': {'before_script': {'#ref': │ │
│ │ '#/definitions/script', 'description': 'The │ │
│ │ ``before...active shell.'}, 'description': {'$ref': │ │
│ │ '#/definitions/description'}, 'disable': {'$ref': │ │
│ │ '#/definitions/disable'}, 'image': {'description': 'Specify │ │
│ │ cont...r container. ', 'type': 'string'}, ...}, 'type': │ │
│ │ 'object'}, ...}, ...}, │ │
│ │ │ format_checker=None │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/devel/lib/py │
│ thon3.10/site-packages/jsonschema/validators.py:451 in validate │
│ │
│ 448 │ │ │
│ 449 │ │ def validate(self, *args, **kwargs): │
│ 450 │ │ │ for error in self.iter_errors(*args, **kwargs): │
│ ❱ 451 │ │ │ │ raise error │
│ 452 │ │ │
│ 453 │ │ def is_type(self, instance, type): │
│ 454 │ │ │ try: │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = ( │ │
│ │ │ { │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'generic': { │ │
│ │ │ │ │ │ 'hostnames': ['.*'], │ │
│ │ │ │ │ │ 'description': 'Generic System', │ │
│ │ │ │ │ │ 'moduletool': 'none', │ │
│ │ │ │ │ │ 'poolsize': 1, │ │
│ │ │ │ │ │ 'pager': False, │ │
│ │ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ │ 'rebuild': False, │ │
│ │ │ │ │ │ │ 'count': 15, │ │
│ │ │ │ │ │ │ 'terse': False │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'report': { │ │
│ │ │ │ │ │ │ 'count': 25, │ │
│ │ │ │ │ │ │ 'format': │ │
│ │ 'name,id,state,runtime,returncode' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'executors': {}, │ │
│ │ │ │ │ │ 'compilers': { │ │
│ │ │ │ │ │ │ 'compiler': { │ │
│ │ │ │ │ │ │ │ 'gcc': { │ │
│ │ │ │ │ │ │ │ │ 'builtin_gcc': { │ │
│ │ │ │ │ │ │ │ │ │ 'cc': 'gcc', │ │
│ │ │ │ │ │ │ │ │ │ 'fc': 'gfortran', │ │
│ │ │ │ │ │ │ │ │ │ 'cxx': 'g++' │ │
│ │ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'cdash': { │ │
│ │ │ │ │ │ │ 'url': 'https://my.cdash.org/', │ │
│ │ │ │ │ │ │ 'project': 'buildtest', │ │
│ │ │ │ │ │ │ 'site': 'generic', │ │
│ │ │ │ │ │ │ 'buildname': 'tutorials' │ │
│ │ │ │ │ │ } │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ }, │ │
│ │ ) │ │
│ │ error = <ValidationError: "'local' is a required property"> │ │
│ │ kwargs = {} │ │
│ │ self = Draft7Validator( │ │
│ │ │ schema={'$id': 'settings.schema.json', '$schema': │ │
│ │ 'http://json-...ft-07/schema#', 'additionalProperties': False, │ │
│ │ 'definitions': {'account': {'description': 'Specify Job ...ing │ │
│ │ resources', 'type': 'string'}, 'cc': {'description': 'Specify │ │
│ │ path...iler wrapper.', 'type': 'string'}, 'compiler_section': │ │
│ │ {'additionalProperties': False, 'description': 'A compiler │ │
│ │ s...iler wrapper.', 'properties': {'cc': {'$ref': │ │
│ │ '#/definitions/cc'}, 'cxx': {'$ref': '#/definitions/cxx'}, │ │
│ │ 'fc': {'$ref': '#/definitions/fc'}, 'module': {'$ref': │ │
│ │ '#/definitions/module'}}, 'required': ['cc', 'cxx', 'fc'], │ │
│ │ ...}, 'container': {'additionalProperties': False, │ │
│ │ 'properties': {'before_script': {'#ref': │ │
│ │ '#/definitions/script', 'description': 'The ``before...active │ │
│ │ shell.'}, 'description': {'$ref': '#/definitions/description'}, │ │
│ │ 'disable': {'$ref': '#/definitions/disable'}, 'image': │ │
│ │ {'description': 'Specify cont...r container. ', 'type': │ │
│ │ 'string'}, ...}, 'type': 'object'}, ...}, ...}, │ │
│ │ │ format_checker=None │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
ValidationError: 'local' is a required property
Failed validating 'required' in
schema['properties']['system']['patternProperties']['^.*$']['properties']['execu
tors']:
{'type': 'object',
'additionalProperties': False,
'description': 'The executor section is used for declaring your '
'executors that are responsible for running jobs. The '
'executor section can be ``local``, ``lsf``, '
'``slurm``. The executors are referenced in buildspec '
'using ``executor`` field.',
'required': ['local'],
'properties': {'defaults': {'type': 'object',
'description': 'Specify default executor '
'settings for all '
'executors',
'additionalProperties': False,
'properties': {'pollinterval': {'type':
'integer',
'description':
'Specify '
'poll '
'interval '
'in '
'seconds '
'after '
'job '
'submission, '
'where '
'buildtest '
'will '
'sleep '
'and '
'poll '
'all '
'jobs '
'for '
'job '
'states. '
'This '
'field '
'can '
'be '
'configured '
'based '
'on '
'your '
'preference. '
'Excessive '
'polling '
'every '
'few '
'seconds '
'can '
'result '
'in '
'system '
'degradation. ',
'minimum': 1,
'default': 30},
'maxpendtime': {'$ref':
'#/definitions/maxpendtime'},
'account': {'$ref':
'#/definitions/account'}}},
'local': {'type': 'object',
'description': 'The ``local`` section is '
'used for declaring local '
'executors for running jobs '
'on local machine',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/local'},
'minProperties': 1},
'lsf': {'type': 'object',
'description': 'The ``lsf`` section is used '
'for declaring LSF executors '
'for running jobs using LSF '
'scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/lsf'}},
'slurm': {'type': 'object',
'description': 'The ``slurm`` section is '
'used for declaring Slurm '
'executors for running jobs '
'using Slurm scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/slurm'}},
'pbs': {'type': 'object',
'description': 'The ``pbs`` section is used '
'for declaring PBS executors '
'for running jobs using PBS '
'scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/pbs'}},
'torque': {'type': 'object',
'description': 'The ``torque`` section is '
'used for declaring '
'PBS/Torque executors for '
'running jobs using Torque '
'scheduler',
'propertyNames': {'pattern':
'^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/pbs'}},
'container': {'type': 'object',
'description': 'The ``container`` '
'section is used for '
'declaring container '
'executors for running '
'jobs using container '
'technology',
'propertyNames': {'pattern':
'^[A-Za-z0-9_.-]+$'},
'additionalProperties': {'$ref':
'#/definitions/container'}}}}
On instance['system']['generic']['executors']:
{}
View Registered Systems (buildtest config systems)
Your buildtest configuration may compose of one or more systems since you can define multiple systems
in a single configuration file to run buildtest for different HPC clusters. You can use
buildtest config systems to report all system details defined in your configuration file.
In this example below we should the generic system. If you have multiple entries, you will see one
entry per system record.
$ buildtest config systems
System Summary (Configuration=/tmp/tmpg3p4j9in/config.yml)
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ system ┃ description ┃ moduletool ┃ hostnames ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ generic │ Generic System │ none │ .* │
└───────────────┴─────────────────────────┴───────────────────┴────────────────┘