Buildtest Command Reference¶

buildtest is a HPC testing framework for building and running tests.

usage: buildtest [options] [COMMANDS]

Named Arguments¶

-V, --version

show program’s version number and exit

-c, --config

Specify Path to Configuration File

-d, --debug

Print debug messages to screen

Default: False

COMMANDS¶

: Possible choices: build, history, buildspec, config, report, inspect, schema, cdash, docs, schemadocs, help

Sub-commands:¶

build¶

Build and Run test

buildtest build [-h] [-b BUILDSPEC] [-x EXCLUDE] [-t TAGS] [-ft FILTER_TAGS]
                [-e EXECUTOR] [-s {parse,build}] [--testdir TESTDIR]
                [--rebuild REBUILD] [-r REPORT]
                [--max-pend-time MAX_PEND_TIME]
                [--poll-interval POLL_INTERVAL] [-k]

Named Arguments¶

-b, --buildspec

Specify a buildspec (file or directory) to build. A buildspec must end in ‘.yml’ extension.

-x, --exclude

Exclude one or more buildspecs (file or directory) from processing. A buildspec must end in ‘.yml’ extension.

-t, --tags

Discover buildspecs by tags found in buildspec cache

-ft, --filter-tags

Filter buildspecs by tags when building tests.

-e, --executor

Discover buildspecs by executor name found in buildspec cache

-s, --stage

Possible choices: parse, build

control behavior of buildtest build

--testdir

Specify a custom test directory where to write tests. This overrides configuration file and default location.

--rebuild

Rebuild test X number of times. Must be a positive number between [1-50]

-r, --report

Specify a report file where tests will be written.

--max-pend-time

Specify Maximum Pending Time (sec) for job before cancelling job. This only applies for batch job submission.

--poll-interval

Specify Poll Interval (sec) for polling batch jobs

-k, --keep-stage-dir

Keep stage directory after job completion.

Default: False

history¶

Query build history

buildtest history [-h]  ...

subcommands¶

Query build history file

: Possible choices: list, query

Sub-commands:¶

list¶

List a summary of all builds

buildtest history list [-h]

query¶

Query information for a particular build

buildtest history query [-h] [-l] id

Positional Arguments¶

id: Select a build ID

Named Arguments¶

-l, --log

Display logfile for corresponding build id

Default: False

buildspec¶

Options for querying buildspec cache

buildtest buildspec [-h]  ...

subcommands¶

Find buildspec from cache file

: Possible choices: find, validate

Sub-commands:¶

find¶

find all buildspecs

buildtest buildspec find [-h] [--root ROOT] [-r] [-t] [-b] [-e] [-p]
                         [--group-by-tags] [--group-by-executor] [-m] [-mb]
                         [--filter FILTER] [--format FORMAT] [--helpfilter]
                         [--helpformat] [--terse]

Named Arguments¶

--root

Specify root buildspecs (directory) path to load buildspecs into buildspec cache.

-r, --rebuild

Rebuild buildspec cache and find all buildspecs again

Default: False

-t, --tags

List all available tags

Default: False

-b, --buildspec

Get all buildspec files from cache

Default: False

-e, --executors

get all unique executors from buildspecs

Default: False

-p, --paths

print all root buildspec paths

Default: False

--group-by-tags

Group tests by tag name

Default: False

--group-by-executor

Group tests by executor name

Default: False

-m, --maintainers

Get all maintainers for all buildspecs

Default: False

-mb, --maintainers-by-buildspecs

Show maintainers breakdown by buildspecs

Default: False

--filter

Filter buildspec cache with filter fields in format –filter key1=val1,key2=val2

--format

Format buildspec cache with format fields in format –format field1,field2,…

--helpfilter

Show Filter fields for –filter option for filtering buildspec cache output

Default: False

--helpformat

Show Format fields for –format option for formatting buildspec cache output

Default: False

--terse

Print output in machine readable format

Default: False

validate¶

Validate buildspecs

buildtest buildspec validate [-h] [-b BUILDSPEC] [-x EXCLUDE] [-t TAG]
                             [-e EXECUTOR]

Named Arguments¶

-b, --buildspec: Specify path to buildspec (file, or directory) to validate
-x, --exclude: Specify path to buildspec to exclude (file or directory) during validation
-t, --tag: Specify buildspecs by tag name to validate
-e, --executor: Specify buildspecs by executor name to validate

config¶

Query buildtest configuration

buildtest config [-h]  ...

subcommands¶

Query information from buildtest configuration file

: Possible choices: executors, view, validate, summary, systems, compilers

Sub-commands:¶

executors¶

Query executors from buildtest configuration

buildtest config executors [-h] [-j] [-y]

Named Arguments¶

-j, --json

View executor in JSON format

Default: False

-y, --yaml

View executors in YAML format

Default: False

view¶

View Buildtest Configuration File

buildtest config view [-h]

validate¶

Validate buildtest settings file with schema.

buildtest config validate [-h]

summary¶

Provide summary of buildtest settings.

buildtest config summary [-h]

systems¶

List all available systems

buildtest config systems [-h]

compilers¶

search or find compilers

buildtest config compilers [-h] [-j] [-y]  ...

Named Arguments¶

-j, --json

List compiler details in JSON format

Default: False

-y, --yaml

List compiler details in YAML format

Default: False

subcommands¶

Find new compilers and add them to detected compiler section

: Possible choices: find

Sub-commands:¶

find¶

Find compilers

buildtest config compilers find [-h] [-d]

Named Arguments¶

-d, --debug

Display Debugging output when finding compilers

Default: False

report¶

Query test report

buildtest report [-h] [--helpformat] [--helpfilter] [--format FORMAT]
                 [--filter FILTER] [--latest] [--oldest] [-r REPORT]
                 ...

Named Arguments¶

--helpformat

List of available format fields

Default: False

--helpfilter

List available filter fields to be used with –filter option

Default: False

--format

format field for printing purposes. For more details see –helpformat for list of available fields. Fields must be separated by comma (usage: –format <field1>,<field2>,…)

--filter

Filter report by filter fields. The filter fields must be a key=value pair and multiple fields can be comma separated in the following format: –filter key1=val1,key2=val2 . For list of filter fields run: –helpfilter.

--latest

Retrieve latest record of particular test

Default: False

--oldest

Retrieve oldest record of particular test

Default: False

-r, --report

Specify a report file to read

Default: “/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.0/var/report.json”

subcommands¶

Fetch test results from report file and print them in table format

: Possible choices: clear, list

Sub-commands:¶

clear¶

delete report file

buildtest report clear [-h]

list¶

List all report files

buildtest report list [-h]

inspect¶

Inspect a test based on NAME or ID

buildtest inspect [-h] [-r REPORT]  ...

Named Arguments¶

-r, --report: Specify a report file to load when inspecting test

subcommands¶

Inspect Test result based on Test ID or Test Name

: Possible choices: name, id, list

Sub-commands:¶

name¶

Specify name of test

buildtest inspect name [-h] [name [name ...]]

Positional Arguments¶

name: Name of test

id¶

Specify a Test ID

buildtest inspect id [-h] [id [id ...]]

Positional Arguments¶

id: Test ID

list¶

List all test ids

buildtest inspect list [-h] [-p]

Named Arguments¶

-p, --parse

Print output in parseable format

Default: False

schema¶

List schema contents and examples

buildtest schema [-h] [-n Schema Name] [-e] [-j]

Named Arguments¶

-n, --name

Possible choices: global.schema.json, definitions.schema.json, settings.schema.json, compiler-v1.0.schema.json, spack-v1.0.schema.json, script-v1.0.schema.json

show schema by name (e.g., script)

-e, --example

Show schema examples

Default: False

-j, --json

Display json schema file

Default: False

cdash¶

Upload test to CDASH server

buildtest cdash [-h]  ...

subcommands¶

buildtest CDASH integeration

: Possible choices: view, upload

Sub-commands:¶

view¶

Open CDASH project in webbrowser

buildtest cdash view [-h] [--url URL]

Named Arguments¶

--url: Specify a url to CDASH project

upload¶

Upload Test to CDASH server

buildtest cdash upload [-h] [--site SITE] [-r REPORT] buildname

Positional Arguments¶

buildname: Specify Build Name reported in CDASH

Named Arguments¶

--site: Specify site name reported in CDASH
-r, --report: Path to report file to upload test results

docs¶

Open buildtest docs in browser

buildtest docs [-h]

schemadocs¶

Open buildtest schema docs in browser

buildtest schemadocs [-h]

help¶

buildtest command guide

buildtest help [-h]

GitHub: https://github.com/buildtesters/buildtest Documentation: https://buildtest.readthedocs.io/en/latest/index.html Schema Documentation: https://buildtesters.github.io/buildtest/ Slack: http://hpcbuildtest.slack.com/

Please report issues at https://github.com/buildtesters/buildtest/issues

Copyright (c) 2021, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy), Shahzeb Siddiqui, and Vanessa Sochat. All rights reserved.