Buildtest Command Reference¶
buildtest is a HPC testing framework for building and running tests.
usage: buildtest [options] [COMMANDS]
Named Arguments¶
- -V, --version
show program’s version number and exit
- -c, --config
Specify Path to Configuration File
- -d, --debug
Print debug messages to screen
Default: False
- --color
Possible choices: on, off
Enable or disable color
Default: “on”
COMMANDS¶
Possible choices: build, buildspec, config, report, inspect, history, edit, schema, cdash, docs, schemadocs, help
Sub-commands:¶
build¶
Build and Run test
buildtest build [-h] [-b BUILDSPEC] [-x EXCLUDE] [-e EXECUTOR] [-t TAGS]
[-f FILTER] [--helpfilter] [-k]
[--max-pend-time MAX_PEND_TIME]
[--poll-interval POLL_INTERVAL] [--rebuild REBUILD]
[-r REPORT] [-s {parse,build}] [--testdir TESTDIR]
discover¶
select buildspecs
- -b, --buildspec
Specify a buildspec (file or directory) to build. A buildspec must end in ‘.yml’ extension.
- -x, --exclude
Exclude one or more buildspecs (file or directory) from processing. A buildspec must end in ‘.yml’ extension.
- -e, --executor
Discover buildspecs by executor name found in buildspec cache
- -t, --tags
Discover buildspecs by tags found in buildspec cache
filter¶
Filter tests
- -f, --filter
Filter buildspec based on tags, type, or maintainers. Usage: –filter key1=val1,key2=val2
- --helpfilter
Show available filter fields used with –filter option
Default: False
extra¶
All extra options
- -k, --keep-stage-dir
Keep stage directory after job completion.
Default: False
- --max-pend-time
Specify Maximum Pending Time (sec) for job before cancelling job. This only applies for batch job submission.
- --poll-interval
Specify Poll Interval (sec) for polling batch jobs
- --rebuild
Rebuild test X number of times. Must be a positive number between [1-50]
- -r, --report
Specify a report file where tests will be written.
- -s, --stage
Possible choices: parse, build
control behavior of buildtest build
- --testdir
Specify a custom test directory where to write tests. This overrides configuration file and default location.
buildspec¶
Buildspec Interface
buildtest buildspec [-h] ...
Sub-commands:¶
find¶
Query information from buildspecs cache
buildtest buildspec find [-h] [-b] [-e] [--group-by-tags]
[--group-by-executor] [-m] [-mb] [-p] [-t]
[--filter FILTER] [--format FORMAT] [--helpfilter]
[--helpformat] [-n] [--terse] [-r] [--root ROOT]
...
Positional Arguments¶
Possible choices: invalid
Named Arguments¶
- -r, --rebuild
Rebuild buildspec cache and find all buildspecs again
Default: False
- --root
Specify root buildspecs (directory) path to load buildspecs into buildspec cache.
filter and format¶
filter and format options
- --filter
Filter buildspec cache with filter fields in format –filter key1=val1,key2=val2
- --format
Format buildspec cache with format fields in format –format field1,field2,…
- --helpfilter
Show Filter fields for –filter option for filtering buildspec cache output
Default: False
- --helpformat
Show Format fields for –format option for formatting buildspec cache output
Default: False
terse¶
terse options
- -n, --no-header
Print output without header in terse output
Default: False
- --terse
Print output in machine readable format
Default: False
query¶
query options to retrieve from buildspec cache
- -b, --buildspec
Get all buildspec files from cache
Default: False
- -e, --executors
get all unique executors from buildspecs
Default: False
- --group-by-tags
Group tests by tag name
Default: False
- --group-by-executor
Group tests by executor name
Default: False
- -m, --maintainers
Get all maintainers for all buildspecs
Default: False
- -mb, --maintainers-by-buildspecs
Show maintainers breakdown by buildspecs
Default: False
- -p, --paths
print all root buildspec paths
Default: False
- -t, --tags
List all available tags
Default: False
show¶
Show content of buildspec file
buildtest buildspec show [-h] name
Positional Arguments¶
- name
Show content of buildspec based on test name
validate¶
Validate buildspecs with JSON Schema
buildtest buildspec validate [-h] [-b BUILDSPEC] [-x EXCLUDE] [-e EXECUTOR]
[-t TAG]
Named Arguments¶
- -b, --buildspec
Specify path to buildspec (file, or directory) to validate
- -x, --exclude
Specify path to buildspec to exclude (file or directory) during validation
- -e, --executor
Specify buildspecs by executor name to validate
- -t, --tag
Specify buildspecs by tag name to validate
config¶
Query buildtest configuration
buildtest config [-h] ...
subcommands¶
Query information from buildtest configuration file
Possible choices: compilers, executors, summary, systems, validate, view
Sub-commands:¶
compilers¶
Search compilers
buildtest config compilers [-h] [-j] [-y] ...
Named Arguments¶
- -j, --json
List compiler details in JSON format
Default: False
- -y, --yaml
List compiler details in YAML format
Default: False
report¶
Query test report
buildtest report [-h] [--filter FILTER] [--format FORMAT] [--helpfilter]
[--helpformat] [--latest] [--oldest] [-n] [-r REPORT] [-t]
...
Named Arguments¶
- --filter
Filter report by filter fields. The filter fields must be a key=value pair and multiple fields can be comma separated in the following format: –filter key1=val1,key2=val2 . For list of filter fields run: –helpfilter.
- --format
format field for printing purposes. For more details see –helpformat for list of available fields. Fields must be separated by comma (usage: –format <field1>,<field2>,…)
- --helpfilter
List available filter fields to be used with –filter option
Default: False
- --helpformat
List of available format fields
Default: False
- --latest
Retrieve latest record of particular test
Default: False
- --oldest
Retrieve oldest record of particular test
Default: False
- -n, --no-header
Don’t print headers column used with terse option (–terse).
Default: False
- -r, --report
Specify a report file to read
Default: “/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json”
- -t, --terse
Print output in machine readable format
Default: False
subcommands¶
Fetch test results from report file and print them in table format
Possible choices: clear, list, summary
inspect¶
Inspect a test based on NAME or ID
buildtest inspect [-h] [-r REPORT] ...
Named Arguments¶
- -r, --report
Specify a report file to load when inspecting test
subcommands¶
Inspect Test result based on Test ID or Test Name
Possible choices: buildspec, id, name, query, list
Sub-commands:¶
buildspec¶
Inspect a test based on buildspec
buildtest inspect buildspec [-h] [-a] [buildspec [buildspec ...]]
Positional Arguments¶
- buildspec
List of buildspecs to query
Named Arguments¶
- -a, --all
Fetch all records for a given test
Default: False
name¶
Specify name of test
buildtest inspect name [-h] [-a] [name [name ...]]
Positional Arguments¶
- name
Name of test
Named Arguments¶
- -a, --all
Fetch all test records for a given test name
Default: False
query¶
Query fields from record
buildtest inspect query [-h] [-b] [-d {first,last,all}] [-e] [-o] [-t]
[name [name ...]]
Positional Arguments¶
- name
Name of test
Named Arguments¶
- -b, --buildscript
Print build script
Default: False
- -d, --display
Possible choices: first, last, all
Determine how records are fetched, by default it will report the last record of the test.
Default: “last”
- -e, --error
Print error file
Default: False
- -o, --output
Print output file
Default: False
- -t, --testpath
Print content of testpath
Default: False
history¶
Query build history
buildtest history [-h] ...
edit¶
Edit a buildspec and validate with schema file
buildtest edit [-h] buildspec
Positional Arguments¶
- buildspec
Open buildspec in editor and validate upon closing file
schema¶
List schema contents and examples
buildtest schema [-h] [-e] [-j] [-n Schema Name]
Named Arguments¶
- -e, --example
Show schema examples
Default: False
- -j, --json
Display json schema file
Default: False
- -n, --name
Possible choices: global.schema.json, definitions.schema.json, settings.schema.json, compiler-v1.0.schema.json, spack-v1.0.schema.json, script-v1.0.schema.json
show schema by name (e.g., script)
GitHub: https://github.com/buildtesters/buildtest Documentation: https://buildtest.readthedocs.io/en/latest/index.html Schema Documentation: https://buildtesters.github.io/buildtest/ Slack: http://hpcbuildtest.slack.com/
Please report issues at https://github.com/buildtesters/buildtest/issues
Copyright (c) 2021, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy), Shahzeb Siddiqui, and Vanessa Sochat. All rights reserved.