Query Test Report
buildtest keeps track of all tests and results in a JSON file. This file is read by buildtest report command to extract certain fields from JSON file and display them in table format. Shown below is command usage to query test reports.
buildtest report --help
$ buildtest report --help
usage: buildtest [options] [COMMANDS] report [-h] [--pager] [--row-count]
[--terse] [-n] [-c COUNT]
[--filter FILTER] [--helpfilter]
[--filterfields] [--helpformat]
[--formatfields] [--latest]
[--oldest] [-s START] [-e END]
[-f | -p] [--format FORMAT | -d]
...
optional arguments:
-h, --help show this help message and exit
--pager Enable PAGING when viewing result
--row-count Display number of rows from query shown in table
--terse Print output in machine readable format
-n, --no-header Do not print header columns in terse output (--terse)
-c COUNT, --count COUNT
Retrieve limited number of rows that get printed
-f, --fail Retrieve all FAIL tests
-p, --pass Retrieve all PASS tests
--format FORMAT Format field for printing purposes. Fields must be
separated by commas (e.g., --format
field1,field2,...).
-d, --detailed Print a detailed summary of the test results
subcommands:
Fetch test results from the report file and print them in table format
clear (c) Remove all report files
list (ls) List all report files
path (p) Print full path to the report file being used
summary (sm) Summarize test report
filter:
--filter FILTER Filter report by filter fields. The filter fields must
be in key=value format and multiple fields can be
comma separated (e.g., --filter key1=val1,key2=val2).
For a list of filter fields, run --helpfilter.
--helpfilter List available filter fields to be used with --filter
option
--filterfields Print raw filter fields for --filter option to filter
the report
format:
--helpformat List available format fields to be used with --format
option
--formatfields Print raw format fields for --format option to format
the report
extra:
--latest Retrieve latest record of a particular test
--oldest Retrieve oldest record of a particular test
-s START, --start START
Filter test by starttime
-e END, --end END Filter test by endtime
You may run buildtest report without any option, and buildtest will display all test results
with default format fields. To see a list of all format fields, click here.
buildtest report
$ buildtest report
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ name ┃ id ┃ state ┃ runtime ┃ returncode ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│ variables_bash │ 136a82f6 │ PASS │ 0.01711 │ 0 │
│ variables_bash │ 016ed495 │ PASS │ 0.016156 │ 0 │
│ variables_bash │ 9048490a │ PASS │ 0.014824 │ 0 │
│ variables_bash │ 89eac98f │ PASS │ 0.01483 │ 0 │
│ variables_bash │ 1cfe762d │ PASS │ 0.019083 │ 0 │
│ variables_bash │ bf470cea │ PASS │ 0.019896 │ 0 │
│ unskipped │ b6c8452f │ PASS │ 0.010637 │ 0 │
│ hello_world │ dbddca39 │ PASS │ 0.008793 │ 0 │
│ hello_world │ 7dd62a7a │ PASS │ 0.009604 │ 0 │
│ hello_world │ 50c2fb9c │ PASS │ 0.008717 │ 0 │
│ hello_world │ f3cc9e60 │ PASS │ 0.009119 │ 0 │
│ hello_world │ adc9f8bf │ PASS │ 0.008566 │ 0 │
│ hello_world │ 099274a7 │ PASS │ 0.00842 │ 0 │
│ hello_world │ dc8965c0 │ PASS │ 0.008419 │ 0 │
│ hello_world │ c4fac494 │ PASS │ 0.008446 │ 0 │
│ hello_world │ da034607 │ PASS │ 0.008779 │ 0 │
...
You can run buildtest report path command to get the full path to the report file being used. This is helpful when the report file
is not known in that case one can print the file by doing cat $(buildtest report path).
buildtest report path
$ buildtest report path
/tmp/tmpr88132k2/var/report.json
The buildtest report command will fetch all records and display them in a table format, if you want to know how many records are displayed in the table you can use buildtest report --row-count which will display
a raw count of records in the table. In the example below we will run buildtest report --row-count which will show number of tests in the report.
buildtest report --row-count
$ buildtest report --row-count
25
You may find it useful to fetch all failed records and determine pass/fail criteria as part of your shell script, this can be done by running buildtest report --row-count --fail as shown below.
buildtest report --row-count --fail
$ buildtest report --row-count --fail
25
Note
buildtest rt is an alias for buildtest report command.
Detailed Reports (buildtest report --detailed)
The buildtest report –detailed option can be used to show a detailed test report that may be of interest when
examining a test. This option is synonymous to running buildtest report --format name,id,user,state,returncode,
runtime,outfile,errfile,buildspec.
Shown below is an example output.
buildtest report --detailed
$ buildtest report --detailed
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━┳━━━━━━━━┳━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ return ┃ runtim ┃ outfil ┃ errfil ┃ buildsp ┃
┃ name ┃ id ┃ user ┃ state ┃ code ┃ e ┃ e ┃ e ┃ ec ┃
┡━━━━━━━━╇━━━━━━━━╇━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│ variab │ 136a82 │ docs │ PASS │ 0 │ 0.0171 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ f6 │ │ │ │ 1 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ 136a82 │ 136a82 │ utorial │
│ │ │ │ │ │ │ f6/var │ f6/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ variab │ 016ed4 │ docs │ PASS │ 0 │ 0.0161 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ 95 │ │ │ │ 56 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ 016ed4 │ 016ed4 │ utorial │
│ │ │ │ │ │ │ 95/var │ 95/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ variab │ 904849 │ docs │ PASS │ 0 │ 0.0148 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ 0a │ │ │ │ 24 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ 904849 │ 904849 │ utorial │
│ │ │ │ │ │ │ 0a/var │ 0a/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ variab │ 89eac9 │ docs │ PASS │ 0 │ 0.0148 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ 8f │ │ │ │ 3 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ 89eac9 │ 89eac9 │ utorial │
│ │ │ │ │ │ │ 8f/var │ 8f/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ variab │ 1cfe76 │ docs │ PASS │ 0 │ 0.0190 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ 2d │ │ │ │ 83 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ 1cfe76 │ 1cfe76 │ utorial │
│ │ │ │ │ │ │ 2d/var │ 2d/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ variab │ bf470c │ docs │ PASS │ 0 │ 0.0198 │ /tmp/t │ /tmp/t │ /home/d │
│ les_ba │ ea │ │ │ │ 96 │ mpr881 │ mpr881 │ ocs/che │
│ sh │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/va │ ash/va │ buildte │
│ │ │ │ │ │ │ rs/var │ rs/var │ st/chec │
│ │ │ │ │ │ │ iables │ iables │ kouts/s │
│ │ │ │ │ │ │ _bash/ │ _bash/ │ table/t │
│ │ │ │ │ │ │ bf470c │ bf470c │ utorial │
│ │ │ │ │ │ │ ea/var │ ea/var │ s/vars. │
│ │ │ │ │ │ │ iables │ iables │ yml │
│ │ │ │ │ │ │ _bash. │ _bash. │ │
│ │ │ │ │ │ │ out │ err │ │
│ unskip │ b6c845 │ docs │ PASS │ 0 │ 0.0106 │ /tmp/t │ /tmp/t │ /home/d │
│ ped │ 2f │ │ │ │ 37 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/sk │ ash/sk │ buildte │
│ │ │ │ │ │ │ ip_tes │ ip_tes │ st/chec │
│ │ │ │ │ │ │ ts/uns │ ts/uns │ kouts/s │
│ │ │ │ │ │ │ kipped │ kipped │ table/t │
│ │ │ │ │ │ │ /b6c84 │ /b6c84 │ utorial │
│ │ │ │ │ │ │ 52f/un │ 52f/un │ s/skip_ │
│ │ │ │ │ │ │ skippe │ skippe │ tests.y │
│ │ │ │ │ │ │ d.out │ d.err │ ml │
│ hello_ │ dbddca │ docs │ PASS │ 0 │ 0.0087 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 39 │ │ │ │ 93 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/db │ rld/db │ utorial │
│ │ │ │ │ │ │ ddca39 │ ddca39 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ 7dd62a │ docs │ PASS │ 0 │ 0.0096 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 7a │ │ │ │ 04 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/7d │ rld/7d │ utorial │
│ │ │ │ │ │ │ d62a7a │ d62a7a │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ 50c2fb │ docs │ PASS │ 0 │ 0.0087 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 9c │ │ │ │ 17 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/50 │ rld/50 │ utorial │
│ │ │ │ │ │ │ c2fb9c │ c2fb9c │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ f3cc9e │ docs │ PASS │ 0 │ 0.0091 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 60 │ │ │ │ 19 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/f3 │ rld/f3 │ utorial │
│ │ │ │ │ │ │ cc9e60 │ cc9e60 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ adc9f8 │ docs │ PASS │ 0 │ 0.0085 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ bf │ │ │ │ 66 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/ad │ rld/ad │ utorial │
│ │ │ │ │ │ │ c9f8bf │ c9f8bf │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ 099274 │ docs │ PASS │ 0 │ 0.0084 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ a7 │ │ │ │ 2 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/09 │ rld/09 │ utorial │
│ │ │ │ │ │ │ 9274a7 │ 9274a7 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ dc8965 │ docs │ PASS │ 0 │ 0.0084 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ c0 │ │ │ │ 19 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/dc │ rld/dc │ utorial │
│ │ │ │ │ │ │ 8965c0 │ 8965c0 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ c4fac4 │ docs │ PASS │ 0 │ 0.0084 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 94 │ │ │ │ 46 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/c4 │ rld/c4 │ utorial │
│ │ │ │ │ │ │ fac494 │ fac494 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ hello_ │ da0346 │ docs │ PASS │ 0 │ 0.0087 │ /tmp/t │ /tmp/t │ /home/d │
│ world │ 07 │ │ │ │ 79 │ mpr881 │ mpr881 │ ocs/che │
│ │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/he │ ash/he │ buildte │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ st/chec │
│ │ │ │ │ │ │ rld/he │ rld/he │ kouts/s │
│ │ │ │ │ │ │ llo_wo │ llo_wo │ table/t │
│ │ │ │ │ │ │ rld/da │ rld/da │ utorial │
│ │ │ │ │ │ │ 034607 │ 034607 │ s/hello │
│ │ │ │ │ │ │ /hello │ /hello │ _world. │
│ │ │ │ │ │ │ _world │ _world │ yml │
│ │ │ │ │ │ │ .out │ .err │ │
│ ulimit │ 90019e │ docs │ PASS │ 0 │ 0.0085 │ /tmp/t │ /tmp/t │ /home/d │
│ _cputi │ 12 │ │ │ │ 09 │ mpr881 │ mpr881 │ ocs/che │
│ me_unl │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ imited │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _cputi │ _cputi │ table/g │
│ │ │ │ │ │ │ me_unl │ me_unl │ eneral_ │
│ │ │ │ │ │ │ imited │ imited │ tests/c │
│ │ │ │ │ │ │ /90019 │ /90019 │ onfigur │
│ │ │ │ │ │ │ e12/ul │ e12/ul │ ation/u │
│ │ │ │ │ │ │ imit_c │ imit_c │ limits. │
│ │ │ │ │ │ │ putime │ putime │ yml │
│ │ │ │ │ │ │ _unlim │ _unlim │ │
│ │ │ │ │ │ │ ited.o │ ited.e │ │
│ │ │ │ │ │ │ ut │ rr │ │
│ ulimit │ 56c91a │ docs │ PASS │ 0 │ 0.0087 │ /tmp/t │ /tmp/t │ /home/d │
│ _cputi │ 09 │ │ │ │ 83 │ mpr881 │ mpr881 │ ocs/che │
│ me_unl │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ imited │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _cputi │ _cputi │ table/g │
│ │ │ │ │ │ │ me_unl │ me_unl │ eneral_ │
│ │ │ │ │ │ │ imited │ imited │ tests/c │
│ │ │ │ │ │ │ /56c91 │ /56c91 │ onfigur │
│ │ │ │ │ │ │ a09/ul │ a09/ul │ ation/u │
│ │ │ │ │ │ │ imit_c │ imit_c │ limits. │
│ │ │ │ │ │ │ putime │ putime │ yml │
│ │ │ │ │ │ │ _unlim │ _unlim │ │
│ │ │ │ │ │ │ ited.o │ ited.e │ │
│ │ │ │ │ │ │ ut │ rr │ │
│ ulimit │ cfb440 │ docs │ PASS │ 0 │ 0.0085 │ /tmp/t │ /tmp/t │ /home/d │
│ _cputi │ bb │ │ │ │ 34 │ mpr881 │ mpr881 │ ocs/che │
│ me_unl │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ imited │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _cputi │ _cputi │ table/g │
│ │ │ │ │ │ │ me_unl │ me_unl │ eneral_ │
│ │ │ │ │ │ │ imited │ imited │ tests/c │
│ │ │ │ │ │ │ /cfb44 │ /cfb44 │ onfigur │
│ │ │ │ │ │ │ 0bb/ul │ 0bb/ul │ ation/u │
│ │ │ │ │ │ │ imit_c │ imit_c │ limits. │
│ │ │ │ │ │ │ putime │ putime │ yml │
│ │ │ │ │ │ │ _unlim │ _unlim │ │
│ │ │ │ │ │ │ ited.o │ ited.e │ │
│ │ │ │ │ │ │ ut │ rr │ │
│ ulimit │ ab3ccb │ docs │ FAIL │ 0 │ 0.0085 │ /tmp/t │ /tmp/t │ /home/d │
│ _stack │ 43 │ │ │ │ 82 │ mpr881 │ mpr881 │ ocs/che │
│ size_u │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ nlimit │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ ed │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _stack │ _stack │ table/g │
│ │ │ │ │ │ │ size_u │ size_u │ eneral_ │
│ │ │ │ │ │ │ nlimit │ nlimit │ tests/c │
│ │ │ │ │ │ │ ed/ab3 │ ed/ab3 │ onfigur │
│ │ │ │ │ │ │ ccb43/ │ ccb43/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _stack │ _stack │ yml │
│ │ │ │ │ │ │ size_u │ size_u │ │
│ │ │ │ │ │ │ nlimit │ nlimit │ │
│ │ │ │ │ │ │ ed.out │ ed.err │ │
│ ulimit │ 6d7595 │ docs │ FAIL │ 0 │ 0.0090 │ /tmp/t │ /tmp/t │ /home/d │
│ _stack │ 12 │ │ │ │ 54 │ mpr881 │ mpr881 │ ocs/che │
│ size_u │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ nlimit │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ ed │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _stack │ _stack │ table/g │
│ │ │ │ │ │ │ size_u │ size_u │ eneral_ │
│ │ │ │ │ │ │ nlimit │ nlimit │ tests/c │
│ │ │ │ │ │ │ ed/6d7 │ ed/6d7 │ onfigur │
│ │ │ │ │ │ │ 59512/ │ 59512/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _stack │ _stack │ yml │
│ │ │ │ │ │ │ size_u │ size_u │ │
│ │ │ │ │ │ │ nlimit │ nlimit │ │
│ │ │ │ │ │ │ ed.out │ ed.err │ │
│ ulimit │ e79714 │ docs │ FAIL │ 0 │ 0.0184 │ /tmp/t │ /tmp/t │ /home/d │
│ _stack │ 01 │ │ │ │ 94 │ mpr881 │ mpr881 │ ocs/che │
│ size_u │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ nlimit │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ ed │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _stack │ _stack │ table/g │
│ │ │ │ │ │ │ size_u │ size_u │ eneral_ │
│ │ │ │ │ │ │ nlimit │ nlimit │ tests/c │
│ │ │ │ │ │ │ ed/e79 │ ed/e79 │ onfigur │
│ │ │ │ │ │ │ 71401/ │ 71401/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _stack │ _stack │ yml │
│ │ │ │ │ │ │ size_u │ size_u │ │
│ │ │ │ │ │ │ nlimit │ nlimit │ │
│ │ │ │ │ │ │ ed.out │ ed.err │ │
│ ulimit │ 04e97c │ docs │ FAIL │ 0 │ 0.0088 │ /tmp/t │ /tmp/t │ /home/d │
│ _filed │ 83 │ │ │ │ 68 │ mpr881 │ mpr881 │ ocs/che │
│ escrip │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ tor_40 │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ 96 │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _filed │ _filed │ table/g │
│ │ │ │ │ │ │ escrip │ escrip │ eneral_ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ tests/c │
│ │ │ │ │ │ │ 96/04e │ 96/04e │ onfigur │
│ │ │ │ │ │ │ 97c83/ │ 97c83/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _filed │ _filed │ yml │
│ │ │ │ │ │ │ escrip │ escrip │ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ │
│ │ │ │ │ │ │ 96.out │ 96.err │ │
│ ulimit │ a82daa │ docs │ FAIL │ 0 │ 0.0091 │ /tmp/t │ /tmp/t │ /home/d │
│ _filed │ 81 │ │ │ │ 92 │ mpr881 │ mpr881 │ ocs/che │
│ escrip │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ tor_40 │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ 96 │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _filed │ _filed │ table/g │
│ │ │ │ │ │ │ escrip │ escrip │ eneral_ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ tests/c │
│ │ │ │ │ │ │ 96/a82 │ 96/a82 │ onfigur │
│ │ │ │ │ │ │ daa81/ │ daa81/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _filed │ _filed │ yml │
│ │ │ │ │ │ │ escrip │ escrip │ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ │
│ │ │ │ │ │ │ 96.out │ 96.err │ │
│ ulimit │ 80114b │ docs │ FAIL │ 0 │ 0.0087 │ /tmp/t │ /tmp/t │ /home/d │
│ _filed │ e3 │ │ │ │ 87 │ mpr881 │ mpr881 │ ocs/che │
│ escrip │ │ │ │ │ │ 32k2/v │ 32k2/v │ ckouts/ │
│ tor_40 │ │ │ │ │ │ ar/tes │ ar/tes │ readthe │
│ 96 │ │ │ │ │ │ ts/gen │ ts/gen │ docs.or │
│ │ │ │ │ │ │ eric.l │ eric.l │ g/user_ │
│ │ │ │ │ │ │ ocal.b │ ocal.b │ builds/ │
│ │ │ │ │ │ │ ash/ul │ ash/ul │ buildte │
│ │ │ │ │ │ │ imits/ │ imits/ │ st/chec │
│ │ │ │ │ │ │ ulimit │ ulimit │ kouts/s │
│ │ │ │ │ │ │ _filed │ _filed │ table/g │
│ │ │ │ │ │ │ escrip │ escrip │ eneral_ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ tests/c │
│ │ │ │ │ │ │ 96/801 │ 96/801 │ onfigur │
│ │ │ │ │ │ │ 14be3/ │ 14be3/ │ ation/u │
│ │ │ │ │ │ │ ulimit │ ulimit │ limits. │
│ │ │ │ │ │ │ _filed │ _filed │ yml │
│ │ │ │ │ │ │ escrip │ escrip │ │
│ │ │ │ │ │ │ tor_40 │ tor_40 │ │
│ │ │ │ │ │ │ 96.out │ 96.err │ │
└────────┴────────┴──────┴───────┴────────┴────────┴────────┴────────┴─────────┘
Note
The --detailed and --format options are mutually exclusive options because both options
will alter the format fields when displaying test results.
You will get the following error if you specify both options as shown below
buildtest report --detailed --format name
$ buildtest report --detailed --format name usage: buildtest [options] [COMMANDS] report [-h] [--pager] [--row-count] [--terse] [-n] [-c COUNT] [--filter FILTER] [--helpfilter] [--filterfields] [--helpformat] [--formatfields] [--latest] [--oldest] [-s START] [-e END] [-f | -p] [--format FORMAT | -d] ... buildtest [options] [COMMANDS] report: error: argument --format: not allowed with argument -d/--detailed
Format Reports (buildtest report --format)
The buildtest report command displays a default format fields that can be changed using the
--format option. The report file (JSON) contains many more fields and we expose some of the fields
with the –format option. To see a list of available format fields you can run buildtest report --helpformat.
This option will list all format fields with their description.
buildtest report --helpformat
$ buildtest report --helpformat
Format Fields
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Field ┃ Description ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ buildspec │ Buildspec File │
│ buildenv │ Show build environment file for test │
│ command │ Command executed │
│ compiler │ Retrieve compiler used for test │
│ endtime │ End Time for test │
│ errfile │ Error File │
│ executor │ Name of executor used for running test │
│ hostname │ Hostname of machine where job was submitted from │
│ full_id │ Fully qualified build identifier │
│ id │ Unique build identifier │
│ metrics │ List all metrics for test │
│ name │ Name of test │
│ outfile │ Output file │
│ returncode │ Return Code │
│ runtime │ Total runtime in seconds │
│ schemafile │ Schema file used for validating test │
│ starttime │ Start time of test │
│ state │ State of test (PASS/FAIL) │
│ tags │ Tag name │
│ testroot │ Root of test directory │
│ testpath │ Path to test │
│ user │ User who ran the test │
└────────────┴──────────────────────────────────────────────────┘
To print just the raw format fields use the command buildtest report --formatfields.
buildtest report --formatfields
$ buildtest report --formatfields
buildspec
buildenv
command
compiler
endtime
errfile
executor
hostname
full_id
id
metrics
name
outfile
returncode
runtime
schemafile
starttime
state
tags
testroot
testpath
user
Format Field Usage
The --format field are specified in comma separated format (i.e --format <field1>,<field2>).
In this example we format table by fields --format id,executor,state,returncode.
buildtest rt --format name,id,executor,state,returncode
$ buildtest rt --format name,id,executor,state,returncode
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃ name ┃ id ┃ executor ┃ state ┃ returncode ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│ variables_bash │ 136a82f6 │ generic.local.bash │ PASS │ 0 │
│ variables_bash │ 016ed495 │ generic.local.bash │ PASS │ 0 │
│ variables_bash │ 9048490a │ generic.local.bash │ PASS │ 0 │
│ variables_bash │ 89eac98f │ generic.local.bash │ PASS │ 0 │
│ variables_bash │ 1cfe762d │ generic.local.bash │ PASS │ 0 │
│ variables_bash │ bf470cea │ generic.local.bash │ PASS │ 0 │
│ unskipped │ b6c8452f │ generic.local.bash │ PASS │ 0 │
│ hello_world │ dbddca39 │ generic.local.bash │ PASS │ 0 │
│ hello_world │ 7dd62a7a │ generic.local.bash │ PASS │ 0 │
│ hello_world │ 50c2fb9c │ generic.local.bash │ PASS │ 0 │
│ hello_world │ f3cc9e60 │ generic.local.bash │ PASS │ 0 │
│ hello_world │ adc9f8bf │ generic.local.bash │ PASS │ 0 │
│ hello_world │ 099274a7 │ generic.local.bash │ PASS │ 0 │
│ hello_world │ dc8965c0 │ generic.local.bash │ PASS │ 0 │
│ hello_world │ c4fac494 │ generic.local.bash │ PASS │ 0 │
│ hello_world │ da034607 │ generic.local.bash │ PASS │ 0 │
│ ulimit_cputime_unlimite │ 90019e12 │ generic.local.bash │ PASS │ 0 │
...
Filter Reports (buildtest report --filter)
The buildtest report command will display all tests results, which can be quite long depending on number of tests
so therefore we need a mechanism to filter the test results. The --filter option can be used
to filter out tests in the output based on filter fields. First, lets see the available filter fields
by run buildtest report --helpfilter which shows a list of filter fields and their description.
buildtest report --helpfilter
$ buildtest report --helpfilter
Filter Fields
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Field ┃ Description ┃ Expected Value ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ buildspec │ Filter by buildspec file │ FILE │
│ name │ Filter by test name │ STRING │
│ executor │ Filter by executor name │ STRING │
│ returncode │ Filter tests by returncode │ INT │
│ state │ Filter by test state │ PASS/FAIL │
│ tags │ Filter tests by tag name │ STRING │
└────────────┴────────────────────────────┴────────────────┘
To print just the raw filter fields use the command buildtest report --filterfields.
buildtest report --filterfields
$ buildtest report --filterfields
buildspec
name
executor
returncode
state
tags
The --filter option expects arguments in key=value format. You can
specify multiple filter delimited by comma. buildtest will treat multiple
filters as logical AND operation. The filter option can be used with
--format field. Let’s see some examples to illustrate the point.
Filter by returncode (--filter returncode)
If you want to retrieve all tests with a given returncode, we can use the returncode
property. For instance, let’s retrieve all tests with returncode of 2 by setting --filter returncode=2.
buildtest rt --filter returncode=2 --format=name,id,returncode
$ buildtest rt --filter returncode=2 --format=name,id,returncode
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ name ┃ id ┃ returncode ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│ returncode_list_mismatch │ b718c4fc │ 2 │
│ returncode_list_mismatch │ b0e727c4 │ 2 │
│ returncode_list_mismatch │ c8357558 │ 2 │
│ returncode_list_mismatch │ 755a0f50 │ 2 │
│ returncode_list_mismatch │ 3829cb31 │ 2 │
│ returncode_list_mismatch │ 89e9896c │ 2 │
│ returncode_list_mismatch │ 6887dc33 │ 2 │
│ linux_strict_test │ 8bbbb6ee │ 2 │
└──────────────────────────┴──────────┴────────────┘
Note
buildtest automatically converts returncode to integer when matching returncode, so --filter returncode="2" will work too
Filter by test name (--filter name)
If you want to filter by test name, use the name attribute in filter option. Let’s assume
we want to filter all tests by name exit1_pass, this can be achieved by setting filter
field as follows: --filter name=exit1_pass. Shown below is an example using name filter field
to filter test results.
buildtest rt --filter name=exit1_pass --format=name,id,returncode,state
$ buildtest rt --filter name=exit1_pass --format=name,id,returncode,state
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┓
┃ name ┃ id ┃ returncode ┃ state ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━┩
│ exit1_pass │ 32962fda │ 1 │ PASS │
│ exit1_pass │ dd622c9c │ 1 │ PASS │
│ exit1_pass │ dc74afeb │ 1 │ PASS │
│ exit1_pass │ 995b6eee │ 1 │ PASS │
│ exit1_pass │ 8fb8c597 │ 1 │ PASS │
│ exit1_pass │ 986d96cd │ 1 │ PASS │
│ exit1_pass │ be59c833 │ 1 │ PASS │
│ exit1_pass │ c035afa0 │ 1 │ PASS │
└────────────┴──────────┴────────────┴───────┘
Filter by buildspec (--filter buildspec)
Likewise, we can filter results by buildspec file using buildspec attribute via
--filter buildspec=<file>. The buildspec attribute must resolve to a file path which can be
relative or absolute path. buildtest will resolve path (absolute path) and find the appropriate
tests that belong to the buildspec file. If file doesn’t exist or is not found in cache it will raise an error.
buildtest rt --filter buildspec=tutorials/python-hello.yml --format=name,id,state,buildspec
$ buildtest rt --filter buildspec=tutorials/python-hello.yml --format=name,id,state,buildspec
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃ state ┃ buildspec ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python_hello │ 342a6091 │ PASS │ /home/docs/checkouts/readthedocs.org/user_ │
│ │ │ │ builds/buildtest/checkouts/stable/tutorial │
│ │ │ │ s/python-hello.yml │
│ python_hello │ 021e0f61 │ PASS │ /home/docs/checkouts/readthedocs.org/user_ │
│ │ │ │ builds/buildtest/checkouts/stable/tutorial │
│ │ │ │ s/python-hello.yml │
│ python_hello │ 7fbd1111 │ PASS │ /home/docs/checkouts/readthedocs.org/user_ │
│ │ │ │ builds/buildtest/checkouts/stable/tutorial │
│ │ │ │ s/python-hello.yml │
│ python_hello │ 293e8e6a │ PASS │ /home/docs/checkouts/readthedocs.org/user_ │
│ │ │ │ builds/buildtest/checkouts/stable/tutorial │
│ │ │ │ s/python-hello.yml │
└──────────────┴──────────┴───────┴────────────────────────────────────────────┘
Filter by test state (--filter state)
If you want to filter results by test state, use the state property. This can be
useful if you want to know all pass or failed tests. The state property expects
value of [PASS|FAIL] since these are the two recorded test states marked by buildtest.
We can also pass multiple filter fields for instance if we want to find all FAIL
tests for executor generic.local.sh we can do the following.
buildtest rt --filter state=FAIL,executor=generic.local.sh --format=name,id,state,executor
$ buildtest rt --filter state=FAIL,executor=generic.local.sh --format=name,id,state,executor
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━┳━━━━┳━━━━━━━┳━━━━━━━━━━┓
┃ name ┃ id ┃ state ┃ executor ┃
┡━━━━━━╇━━━━╇━━━━━━━╇━━━━━━━━━━┩
└──────┴────┴───────┴──────────┘
Filter Exception Cases
The returncode filter field expects an integer value, so if you try a non-integer
returncode you will get the following message
buildtest rt --filter returncode=1.5
$ buildtest rt --filter returncode=1.5
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ bin/buildtest:38 in <module> │
│ │
│ 35 │
│ 36 if __name__ == "__main__": │
│ 37 │
│ ❱ 38 sys.exit(main()) │
│ 39 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ main = <function main at 0x7ff4b9e80280> │ │
│ │ os = <module 'os' from │ │
│ │ '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9… │ │
│ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/main.py:128 in main │
│ │
│ 125 │ │
│ 126 │ # buildtest report │
│ 127 │ elif args.subcommands in ["report", "rt"]: │
│ ❱ 128 │ │ report_cmd(args, configuration=configuration, report_file=repo │
│ 129 │ │
│ 130 │ elif args.subcommands == "path": │
│ 131 │ │ path_cmd( │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ pager=False, row_count=False, terse=False, │ │
│ │ no_header=False, count=None, report_subcommand=None, │ │
│ │ filter={'returncode': '1.5'}, helpfilter=False, │ │
│ │ filterfields=False, helpformat=False, │ │
│ │ formatfields=False, latest=False, oldest=False, │ │
│ │ start=None, end=None, fail=False, passed=False, │ │
│ │ format=None, detailed=False) │ │
│ │ buildtest_editor = '/usr/bin/vim' │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7ff4b7f67ca0> │ │
│ │ parser = <buildtest.cli.BuildTestParser object at │ │
│ │ 0x7ff4b9fcca30> │ │
│ │ report_file = None │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7ff4b7f67c40> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:778 in report_cmd │
│ │
│ 775 │ │ list_report() │
│ 776 │ │ return │
│ 777 │ │
│ ❱ 778 │ results = Report( │
│ 779 │ │ configuration=configuration, │
│ 780 │ │ filter=args.filter, │
│ 781 │ │ format=args.format, │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, pager=False, │ │
│ │ row_count=False, terse=False, no_header=False, │ │
│ │ count=None, report_subcommand=None, │ │
│ │ filter={'returncode': '1.5'}, helpfilter=False, │ │
│ │ filterfields=False, helpformat=False, │ │
│ │ formatfields=False, latest=False, oldest=False, │ │
│ │ start=None, end=None, fail=False, passed=False, │ │
│ │ format=None, detailed=False) │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7ff4b7f67ca0> │ │
│ │ consoleColor = 'default' │ │
│ │ pager = False │ │
│ │ report_file = None │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:98 in __init__ │
│ │
│ 95 │ │ self.set_report_parameters(**kwargs) │
│ 96 │ │ │
│ 97 │ │ self.report = self.load_report() │
│ ❱ 98 │ │ self.validate_filter_and_format_fields() │
│ 99 │ │ self.filter_buildspecs_from_report() │
│ 100 │ │ │
│ 101 │ │ self.process_report() │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7ff4b7f67ca0> │ │
│ │ kwargs = { │ │
│ │ │ 'filter': {'returncode': '1.5'}, │ │
│ │ │ 'format': None, │ │
│ │ │ 'start': None, │ │
│ │ │ 'end': None, │ │
│ │ │ 'failure': False, │ │
│ │ │ 'passed': False, │ │
│ │ │ 'latest': False, │ │
│ │ │ 'oldest': False, │ │
│ │ │ 'report_file': None, │ │
│ │ │ 'count': None, │ │
│ │ │ ... +1 │ │
│ │ } │ │
│ │ self = <buildtest.cli.report.Report object at 0x7ff4b7f5ee80> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:141 in validate_filter_and_format_fields │
│ │
│ 138 │ │ return self.report │
│ 139 │ │
│ 140 │ def validate_filter_and_format_fields(self): │
│ ❱ 141 │ │ self._check_filter_fields() │
│ 142 │ │ self._check_format_fields() │
│ 143 │ │ self._check_start_and_end_fields() │
│ 144 │
│ │
│ ╭─────────────────────────── locals ────────────────────────────╮ │
│ │ self = <buildtest.cli.report.Report object at 0x7ff4b7f5ee80> │ │
│ ╰───────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:166 in _check_filter_fields │
│ │
│ 163 │ │ │ │ │ ) │
│ 164 │ │ │ │ │
│ 165 │ │ │ │ if key == "returncode" and not is_int(self.filter[key] │
│ ❱ 166 │ │ │ │ │ raise BuildTestError( │
│ 167 │ │ │ │ │ │ f"Invalid returncode:{self.filter[key]} must b │
│ 168 │ │ │ │ │ ) │
│ 169 │
│ │
│ ╭─────────────────────────── locals ────────────────────────────╮ │
│ │ key = 'returncode' │ │
│ │ self = <buildtest.cli.report.Report object at 0x7ff4b7f5ee80> │ │
│ ╰───────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
BuildTestError: 'Invalid returncode:1.5 must be an integer'
The state filter field expects value of PASS or FAIL so if you specify an
invalid state you will get an error as follows.
buildtest rt --filter state=UNKNOWN
$ buildtest rt --filter state=UNKNOWN
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ bin/buildtest:38 in <module> │
│ │
│ 35 │
│ 36 if __name__ == "__main__": │
│ 37 │
│ ❱ 38 sys.exit(main()) │
│ 39 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ main = <function main at 0x7f3500110280> │ │
│ │ os = <module 'os' from │ │
│ │ '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9… │ │
│ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/main.py:128 in main │
│ │
│ 125 │ │
│ 126 │ # buildtest report │
│ 127 │ elif args.subcommands in ["report", "rt"]: │
│ ❱ 128 │ │ report_cmd(args, configuration=configuration, report_file=repo │
│ 129 │ │
│ 130 │ elif args.subcommands == "path": │
│ 131 │ │ path_cmd( │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ pager=False, row_count=False, terse=False, │ │
│ │ no_header=False, count=None, report_subcommand=None, │ │
│ │ filter={'state': 'UNKNOWN'}, helpfilter=False, │ │
│ │ filterfields=False, helpformat=False, │ │
│ │ formatfields=False, latest=False, oldest=False, │ │
│ │ start=None, end=None, fail=False, passed=False, │ │
│ │ format=None, detailed=False) │ │
│ │ buildtest_editor = '/usr/bin/vim' │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f34fe1f6ca0> │ │
│ │ parser = <buildtest.cli.BuildTestParser object at │ │
│ │ 0x7f350025ba30> │ │
│ │ report_file = None │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f34fe1f6c40> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:778 in report_cmd │
│ │
│ 775 │ │ list_report() │
│ 776 │ │ return │
│ 777 │ │
│ ❱ 778 │ results = Report( │
│ 779 │ │ configuration=configuration, │
│ 780 │ │ filter=args.filter, │
│ 781 │ │ format=args.format, │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, pager=False, │ │
│ │ row_count=False, terse=False, no_header=False, │ │
│ │ count=None, report_subcommand=None, filter={'state': │ │
│ │ 'UNKNOWN'}, helpfilter=False, filterfields=False, │ │
│ │ helpformat=False, formatfields=False, latest=False, │ │
│ │ oldest=False, start=None, end=None, fail=False, │ │
│ │ passed=False, format=None, detailed=False) │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f34fe1f6ca0> │ │
│ │ consoleColor = 'default' │ │
│ │ pager = False │ │
│ │ report_file = None │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:99 in __init__ │
│ │
│ 96 │ │ │
│ 97 │ │ self.report = self.load_report() │
│ 98 │ │ self.validate_filter_and_format_fields() │
│ ❱ 99 │ │ self.filter_buildspecs_from_report() │
│ 100 │ │ │
│ 101 │ │ self.process_report() │
│ 102 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f34fe1f6ca0> │ │
│ │ kwargs = { │ │
│ │ │ 'filter': {'state': 'UNKNOWN'}, │ │
│ │ │ 'format': None, │ │
│ │ │ 'start': None, │ │
│ │ │ 'end': None, │ │
│ │ │ 'failure': False, │ │
│ │ │ 'passed': False, │ │
│ │ │ 'latest': False, │ │
│ │ │ 'oldest': False, │ │
│ │ │ 'report_file': None, │ │
│ │ │ 'count': None, │ │
│ │ │ ... +1 │ │
│ │ } │ │
│ │ self = <buildtest.cli.report.Report object at 0x7f34fe1ede80> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:290 in filter_buildspecs_from_report │
│ │
│ 287 │ │ │ self.validate_buildspec_filter() │
│ 288 │ │ │
│ 289 │ │ if self.filter.get("state"): │
│ ❱ 290 │ │ │ self.validate_state_filter() │
│ 291 │ │
│ 292 │ def filter_by_start_end(self, test): │
│ 293 │ │ """This method will return a boolean (True/False) to check if │
│ │
│ ╭─────────────────────────── locals ────────────────────────────╮ │
│ │ self = <buildtest.cli.report.Report object at 0x7f34fe1ede80> │ │
│ ╰───────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:272 in validate_state_filter │
│ │
│ 269 │ │
│ 270 │ def validate_state_filter(self): │
│ 271 │ │ if self.filter["state"] not in [PASS, FAIL]: │
│ ❱ 272 │ │ │ raise BuildTestError( │
│ 273 │ │ │ │ f"filter argument 'state' must be 'PASS' or 'FAIL' got │
│ 274 │ │ │ ) │
│ 275 │
│ │
│ ╭─────────────────────────── locals ────────────────────────────╮ │
│ │ self = <buildtest.cli.report.Report object at 0x7f34fe1ede80> │ │
│ ╰───────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
BuildTestError: "filter argument 'state' must be 'PASS' or 'FAIL' got value
UNKNOWN"
The buildspec field expects a valid file path, it can be an absolute or relative
path, buildtest will resolve absolute path and check if file exist and is in the report
file. If it’s an invalid file we get an error such as
buildtest rt --filter buildspec=/path/to/invalid.yml
$ buildtest rt --filter buildspec=/path/to/invalid.yml
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ bin/buildtest:38 in <module> │
│ │
│ 35 │
│ 36 if __name__ == "__main__": │
│ 37 │
│ ❱ 38 sys.exit(main()) │
│ 39 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ main = <function main at 0x7f905cfec280> │ │
│ │ os = <module 'os' from │ │
│ │ '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9… │ │
│ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/main.py:128 in main │
│ │
│ 125 │ │
│ 126 │ # buildtest report │
│ 127 │ elif args.subcommands in ["report", "rt"]: │
│ ❱ 128 │ │ report_cmd(args, configuration=configuration, report_file=repo │
│ 129 │ │
│ 130 │ elif args.subcommands == "path": │
│ 131 │ │ path_cmd( │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, │ │
│ │ pager=False, row_count=False, terse=False, │ │
│ │ no_header=False, count=None, report_subcommand=None, │ │
│ │ filter={'buildspec': '/path/to/invalid.yml'}, │ │
│ │ helpfilter=False, filterfields=False, │ │
│ │ helpformat=False, formatfields=False, latest=False, │ │
│ │ oldest=False, start=None, end=None, fail=False, │ │
│ │ passed=False, format=None, detailed=False) │ │
│ │ buildtest_editor = '/usr/bin/vim' │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f905b0d3cd0> │ │
│ │ parser = <buildtest.cli.BuildTestParser object at │ │
│ │ 0x7f905d138a30> │ │
│ │ report_file = None │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f905b0d3c70> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:778 in report_cmd │
│ │
│ 775 │ │ list_report() │
│ 776 │ │ return │
│ 777 │ │
│ ❱ 778 │ results = Report( │
│ 779 │ │ configuration=configuration, │
│ 780 │ │ filter=args.filter, │
│ 781 │ │ format=args.format, │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(subcommands='rt', configfile=None, │ │
│ │ debug=False, loglevel='DEBUG', editor=None, │ │
│ │ view_log=False, logpath=False, print_log=False, │ │
│ │ color=None, no_color=False, helpcolor=False, │ │
│ │ report=None, listopts=False, verbose=False, pager=False, │ │
│ │ row_count=False, terse=False, no_header=False, │ │
│ │ count=None, report_subcommand=None, filter={'buildspec': │ │
│ │ '/path/to/invalid.yml'}, helpfilter=False, │ │
│ │ filterfields=False, helpformat=False, │ │
│ │ formatfields=False, latest=False, oldest=False, │ │
│ │ start=None, end=None, fail=False, passed=False, │ │
│ │ format=None, detailed=False) │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f905b0d3cd0> │ │
│ │ consoleColor = 'default' │ │
│ │ pager = False │ │
│ │ report_file = None │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:99 in __init__ │
│ │
│ 96 │ │ │
│ 97 │ │ self.report = self.load_report() │
│ 98 │ │ self.validate_filter_and_format_fields() │
│ ❱ 99 │ │ self.filter_buildspecs_from_report() │
│ 100 │ │ │
│ 101 │ │ self.process_report() │
│ 102 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f905b0d3cd0> │ │
│ │ kwargs = { │ │
│ │ │ 'filter': {'buildspec': '/path/to/invalid.yml'}, │ │
│ │ │ 'format': None, │ │
│ │ │ 'start': None, │ │
│ │ │ 'end': None, │ │
│ │ │ 'failure': False, │ │
│ │ │ 'passed': False, │ │
│ │ │ 'latest': False, │ │
│ │ │ 'oldest': False, │ │
│ │ │ 'report_file': None, │ │
│ │ │ 'count': None, │ │
│ │ │ ... +1 │ │
│ │ } │ │
│ │ self = <buildtest.cli.report.Report object at 0x7f905b0c8bb0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:287 in filter_buildspecs_from_report │
│ │
│ 284 │ │ │ return │
│ 285 │ │ │
│ 286 │ │ if self.filter.get("buildspec"): │
│ ❱ 287 │ │ │ self.validate_buildspec_filter() │
│ 288 │ │ │
│ 289 │ │ if self.filter.get("state"): │
│ 290 │ │ │ self.validate_state_filter() │
│ │
│ ╭─────────────────────────── locals ────────────────────────────╮ │
│ │ self = <buildtest.cli.report.Report object at 0x7f905b0c8bb0> │ │
│ ╰───────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/report.py:259 in validate_buildspec_filter │
│ │
│ 256 │ │ logger.debug(f"Filter records by buildspec: {resolved_buildspe │
│ 257 │ │ │
│ 258 │ │ if not resolved_buildspecs: │
│ ❱ 259 │ │ │ raise BuildTestError( │
│ 260 │ │ │ │ f"Invalid File Path for filter field 'buildspec': {sel │
│ 261 │ │ │ ) │
│ 262 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ resolved_buildspecs = None │ │
│ │ self = <buildtest.cli.report.Report object at │ │
│ │ 0x7f905b0c8bb0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
BuildTestError: "Invalid File Path for filter field 'buildspec':
/path/to/invalid.yml"
You may have a valid filepath for buildspec filter field such as
$BUILDTEST_ROOT/tutorials/invalid_executor.yml, but there is no record of a test in the report cache
because this test wasn’t run. In this case you will get the following message.
buildtest rt --filter buildspec=$BUILDTEST_ROOT/tutorials/invalid_executor.yml
$ buildtest rt --filter buildspec=$BUILDTEST_ROOT/tutorials/invalid_executor.yml ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ bin/buildtest:38 in <module> │ │ │ │ 35 │ │ 36 if __name__ == "__main__": │ │ 37 │ │ ❱ 38 sys.exit(main()) │ │ 39 │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │ │ │ main = <function main at 0x7fd24ffc4280> │ │ │ │ os = <module 'os' from │ │ │ │ '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9… │ │ │ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │ │ │ sys = <module 'sys' (built-in)> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ buildtest/main.py:128 in main │ │ │ │ 125 │ │ │ 126 │ # buildtest report │ │ 127 │ elif args.subcommands in ["report", "rt"]: │ │ ❱ 128 │ │ report_cmd(args, configuration=configuration, report_file=repo │ │ 129 │ │ │ 130 │ elif args.subcommands == "path": │ │ 131 │ │ path_cmd( │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = Namespace(subcommands='rt', configfile=None, │ │ │ │ debug=False, loglevel='DEBUG', editor=None, │ │ │ │ view_log=False, logpath=False, print_log=False, │ │ │ │ color=None, no_color=False, helpcolor=False, │ │ │ │ report=None, listopts=False, verbose=False, │ │ │ │ pager=False, row_count=False, terse=False, │ │ │ │ no_header=False, count=None, report_subcommand=None, │ │ │ │ filter={'buildspec': │ │ │ │ '$BUILDTEST_ROOT/tutorials/invalid_executor.yml'}, │ │ │ │ helpfilter=False, filterfields=False, │ │ │ │ helpformat=False, formatfields=False, latest=False, │ │ │ │ oldest=False, start=None, end=None, fail=False, │ │ │ │ passed=False, format=None, detailed=False) │ │ │ │ buildtest_editor = '/usr/bin/vim' │ │ │ │ configuration = <buildtest.config.SiteConfiguration object at │ │ │ │ 0x7fd24e0abd60> │ │ │ │ parser = <buildtest.cli.BuildTestParser object at │ │ │ │ 0x7fd250110a30> │ │ │ │ report_file = None │ │ │ │ system = <buildtest.system.BuildTestSystem object at │ │ │ │ 0x7fd24e0abd00> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ buildtest/cli/report.py:778 in report_cmd │ │ │ │ 775 │ │ list_report() │ │ 776 │ │ return │ │ 777 │ │ │ ❱ 778 │ results = Report( │ │ 779 │ │ configuration=configuration, │ │ 780 │ │ filter=args.filter, │ │ 781 │ │ format=args.format, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = Namespace(subcommands='rt', configfile=None, │ │ │ │ debug=False, loglevel='DEBUG', editor=None, │ │ │ │ view_log=False, logpath=False, print_log=False, │ │ │ │ color=None, no_color=False, helpcolor=False, │ │ │ │ report=None, listopts=False, verbose=False, pager=False, │ │ │ │ row_count=False, terse=False, no_header=False, │ │ │ │ count=None, report_subcommand=None, filter={'buildspec': │ │ │ │ '$BUILDTEST_ROOT/tutorials/invalid_executor.yml'}, │ │ │ │ helpfilter=False, filterfields=False, helpformat=False, │ │ │ │ formatfields=False, latest=False, oldest=False, │ │ │ │ start=None, end=None, fail=False, passed=False, │ │ │ │ format=None, detailed=False) │ │ │ │ configuration = <buildtest.config.SiteConfiguration object at │ │ │ │ 0x7fd24e0abd60> │ │ │ │ consoleColor = 'default' │ │ │ │ pager = False │ │ │ │ report_file = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ buildtest/cli/report.py:99 in __init__ │ │ │ │ 96 │ │ │ │ 97 │ │ self.report = self.load_report() │ │ 98 │ │ self.validate_filter_and_format_fields() │ │ ❱ 99 │ │ self.filter_buildspecs_from_report() │ │ 100 │ │ │ │ 101 │ │ self.process_report() │ │ 102 │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ configuration = <buildtest.config.SiteConfiguration object at │ │ │ │ 0x7fd24e0abd60> │ │ │ │ kwargs = { │ │ │ │ │ 'filter': { │ │ │ │ │ │ 'buildspec': │ │ │ │ '$BUILDTEST_ROOT/tutorials/invalid_executor.yml' │ │ │ │ │ }, │ │ │ │ │ 'format': None, │ │ │ │ │ 'start': None, │ │ │ │ │ 'end': None, │ │ │ │ │ 'failure': False, │ │ │ │ │ 'passed': False, │ │ │ │ │ 'latest': False, │ │ │ │ │ 'oldest': False, │ │ │ │ │ 'report_file': None, │ │ │ │ │ 'count': None, │ │ │ │ │ ... +1 │ │ │ │ } │ │ │ │ self = <buildtest.cli.report.Report object at 0x7fd24e0a0dc0> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ buildtest/cli/report.py:287 in filter_buildspecs_from_report │ │ │ │ 284 │ │ │ return │ │ 285 │ │ │ │ 286 │ │ if self.filter.get("buildspec"): │ │ ❱ 287 │ │ │ self.validate_buildspec_filter() │ │ 288 │ │ │ │ 289 │ │ if self.filter.get("state"): │ │ 290 │ │ │ self.validate_state_filter() │ │ │ │ ╭─────────────────────────── locals ────────────────────────────╮ │ │ │ self = <buildtest.cli.report.Report object at 0x7fd24e0a0dc0> │ │ │ ╰───────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │ │ buildtest/cli/report.py:264 in validate_buildspec_filter │ │ │ │ 261 │ │ │ ) │ │ 262 │ │ │ │ 263 │ │ if not resolved_buildspecs in self.report.keys(): │ │ ❱ 264 │ │ │ raise BuildTestError( │ │ 265 │ │ │ │ f"buildspec file: {resolved_buildspecs} not found in c │ │ 266 │ │ │ ) │ │ 267 │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ resolved_buildspecs = '/home/docs/checkouts/readthedocs.org/user_builds… │ │ │ │ self = <buildtest.cli.report.Report object at │ │ │ │ 0x7fd24e0a0dc0> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ BuildTestError: 'buildspec file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tuto rials/invalid_executor.yml not found in cache'
Find Latest or Oldest test
We can search for oldest or latest test for any given test. This can be useful if you
want to see first or last test run for a particular test. If you want to retrieve the oldest
test you can use --oldest option, likewise you can retrieve the latest run via --latest option.
Let’s take a look at this example, we filter by test name exit1_pass which retrieves all
test runs. In subsequent example we filter by latest and oldest run.
Show oldest run
$ buildtest report --filter name=exit1_pass --format name,id,starttime
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃ starttime ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_pass │ 32962fda │ 2024/09/05 15:34:39 │
│ exit1_pass │ dd622c9c │ 2024/09/05 15:34:44 │
│ exit1_pass │ dc74afeb │ 2024/09/05 15:34:45 │
│ exit1_pass │ 995b6eee │ 2024/09/05 15:34:58 │
│ exit1_pass │ 8fb8c597 │ 2024/09/05 15:35:11 │
│ exit1_pass │ 986d96cd │ 2024/09/05 15:35:11 │
│ exit1_pass │ be59c833 │ 2024/09/05 15:35:14 │
│ exit1_pass │ c035afa0 │ 2024/09/05 15:35:15 │
└────────────┴──────────┴─────────────────────┘
$ buildtest report --filter name=exit1_pass --format name,id,starttime --oldest
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃ starttime ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_pass │ 32962fda │ 2024/09/05 15:34:39 │
└────────────┴──────────┴─────────────────────┘
Show latest run
$ buildtest report --filter name=exit1_pass --format name,id,starttime --latest
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃ starttime ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_pass │ c035afa0 │ 2024/09/05 15:35:15 │
└────────────┴──────────┴─────────────────────┘
You may combine –oldest and –latest options in same command, in this case buildtest will retrieve the first and last record of every test.
Show oldest and latest run
$ buildtest report --filter name=exit1_pass --format name,id,starttime --oldest --latest
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃ starttime ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_pass │ 32962fda │ 2024/09/05 15:34:39 │
│ exit1_pass │ c035afa0 │ 2024/09/05 15:35:15 │
└────────────┴──────────┴─────────────────────┘
Find all Failed Tests (buildtest report --fail)
The buildtest report --fail command can be used to retrieve all failed tests which is equivalent to filtering tests
by state=FAIL since test state is determined by state property. This command can be useful to pin-point failures.
Let’s take a look at next two example, the first command will query report and limit output to 6 rows which can retrieve both pass
and failed test. The second command will run same example with --fail and only retrieve failed tests.
Take note of the state property in table, in second example buildtest will filter test and report all FAIL tests.
Show fail tests
$ buildtest report --format name,id,state --count=6
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ name ┃ id ┃ state ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ variables_bash │ 136a82f6 │ PASS │
│ variables_bash │ 016ed495 │ PASS │
│ variables_bash │ 9048490a │ PASS │
│ variables_bash │ 89eac98f │ PASS │
│ variables_bash │ 1cfe762d │ PASS │
│ variables_bash │ bf470cea │ PASS │
└────────────────┴──────────┴───────┘
$ buildtest report --format name,id,state --fail --count=6
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ name ┃ id ┃ state ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ ulimit_stacksize_unlimited │ ab3ccb43 │ FAIL │
│ ulimit_stacksize_unlimited │ 6d759512 │ FAIL │
│ ulimit_stacksize_unlimited │ e7971401 │ FAIL │
│ ulimit_filedescriptor_4096 │ 04e97c83 │ FAIL │
│ ulimit_filedescriptor_4096 │ a82daa81 │ FAIL │
│ ulimit_filedescriptor_4096 │ 80114be3 │ FAIL │
└────────────────────────────┴──────────┴───────┘
Find all Passed Tests (buildtest report --pass)
The buildtest report --pass, works similar to buildtest report --fail where it will filter test by state=PASS which can be used to find all passed tests.
We can see in example below, that buildtest will retrieve only PASS tests which can be determined by the state property
Show pass tests
$ buildtest report --format name,id,state --pass --count=6
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ name ┃ id ┃ state ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ variables_bash │ 136a82f6 │ PASS │
│ variables_bash │ 016ed495 │ PASS │
│ variables_bash │ 9048490a │ PASS │
│ variables_bash │ 89eac98f │ PASS │
│ variables_bash │ 1cfe762d │ PASS │
│ variables_bash │ bf470cea │ PASS │
└────────────────┴──────────┴───────┘
Note
The --pass and --fail are mutually exclusive option which will query all PASS or FAIL test from report file, if you try to
specify both options on command line you will get an error
$ buildtest report --pass --fail
usage: buildtest [options] [COMMANDS] report [-h] [--pager] [--row-count]
[--terse] [-n] [-c COUNT]
[--filter FILTER] [--helpfilter]
[--filterfields] [--helpformat]
[--formatfields] [--latest]
[--oldest] [-s START] [-e END]
[-f | -p] [--format FORMAT | -d]
...
buildtest [options] [COMMANDS] report: error: argument -f/--fail: not allowed with argument -p/--pass
Find Tests by Start and End Date (buildtest report --start --end)
The buildtest report --start and buildtest report --end command can be used to retrieve test records based on start and end date.
Let’s take a look at these two example, the first test queries report by filtering by state and name. The second command
will run same example with --start --end option. Take note of the starttime and endtime properties in table, in second example buildtest will
filter test and only report tests in the range of [start, end] dates.
Filter by start and end date
$ buildtest report --filter state=FAIL,name=exit1_fail --format name,state,starttime,endtime
Report File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkout
s/1082/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ state ┃ starttime ┃ endtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ FAIL │ 2022/06/09 17:51:50 │ 2022/06/09 17:51:50 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/09 17:51:51 │ 2022/06/09 17:51:51 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/09 17:51:53 │ 2022/06/09 17:51:53 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/09 17:52:01 │ 2022/06/09 17:52:01 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:51:50 │ 2022/06/10 17:51:50 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:51:52 │ 2022/06/10 17:51:52 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:51:52 │ 2022/06/10 17:51:52 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:52:01 │ 2022/06/10 17:52:01 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:57 │ 2022/06/11 17:51:57 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:59 │ 2022/06/11 17:51:59 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:57 │ 2022/06/11 17:51:57 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:59 │ 2022/06/11 17:51:59 │
└────────────────────────────────────────────────────────┴──────────────┴──────────────────────────────────────────┴─────────────────────────────────────────┘
$ buildtest report --filter state=FAIL,name=exit1_fail --format name,state,starttime,endtime --start 2022-06-10 --end 2022-06-11
Report File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkout
s/1082/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ state ┃ starttime ┃ endtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ FAIL │ 2022/06/10 17:51:50 │ 2022/06/10 17:51:50 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:51:52 │ 2022/06/10 17:51:52 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:51:52 │ 2022/06/10 17:51:52 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/10 17:52:01 │ 2022/06/10 17:52:01 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:57 │ 2022/06/11 17:51:57 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:59 │ 2022/06/11 17:51:59 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:57 │ 2022/06/11 17:51:57 │
├────────────────────────────────────────────────────────┼──────────────┼──────────────────────────────────────────┼─────────────────────────────────────────┤
│ exit1_fail │ FAIL │ 2022/06/11 17:51:59 │ 2022/06/11 17:51:59 │
└────────────────────────────────────────────────────────┴──────────────┴──────────────────────────────────────────┴─────────────────────────────────────────┘
Limit number of test records (buildtest report --count)
The buildtest report command will show all test results from the report file and the output can be quite long.
If you want to limit the number of records that get printed then you can use buildtest report --count
where --count is number of rows that that get printed. Shown below is the output of 2 records in table format
buildtest report --count 2
$ buildtest report --count 2
Report File: /tmp/tmpr88132k2/var/report.json
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ name ┃ id ┃ state ┃ runtime ┃ returncode ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│ variables_bash │ 136a82f6 │ PASS │ 0.01711 │ 0 │
│ variables_bash │ 016ed495 │ PASS │ 0.016156 │ 0 │
└────────────────┴──────────┴───────┴──────────┴────────────┘
The --count option also works with terse mode --terse, shown below is the same output
buildtest report --terse --count 2
$ buildtest report --terse --count 2
name|id|state|runtime|returncode
variables_bash|136a82f6|PASS|0.01711|0
variables_bash|016ed495|PASS|0.016156|0
If you set --count=0 no records will be displayed; however, if you set any negative
value for --count then all records will be displayed.
Query tests with --count at 0 and negative value
$ buildtest report --count=0 --no-header --terse | wc -l
0
$ buildtest report --count=-5 --no-header --terse | wc -l
102
Terse Output (buildtest report --terse)
If you would like to parse the result of buildtest report, you can use the --terse or -t option which
will print the report in machine readable format that shows the name of each column followed by each entry. Each entry
is delimited by PIPE symbol (|). The --terse option works with --format and --filter option. In this
next example, we report all FAIL tests in terse output. The first line is the header of tables followed by
output, if you want to disable output of header you can use --no-header option.
buildtest report --filter state=FAIL --format=name,id,state --terse
$ buildtest report --filter state=FAIL --format=name,id,state --terse
name|id|state
ulimit_stacksize_unlimited|ab3ccb43|FAIL
ulimit_stacksize_unlimited|6d759512|FAIL
ulimit_stacksize_unlimited|e7971401|FAIL
ulimit_filedescriptor_4096|04e97c83|FAIL
ulimit_filedescriptor_4096|a82daa81|FAIL
ulimit_filedescriptor_4096|80114be3|FAIL
ulimit_max_user_process_2048|02a05262|FAIL
ulimit_max_user_process_2048|f978e32e|FAIL
ulimit_max_user_process_2048|fce2c6a9|FAIL
systemd_default_target|41284665|FAIL
systemd_default_target|9afcbd19|FAIL
systemd_default_target|7794a73d|FAIL
systemd_default_target|218dcfc5|FAIL
kernel_swapusage|123211f4|FAIL
kernel_swapusage|59bb8969|FAIL
kernel_swapusage|a9d0b510|FAIL
kernel_swapusage|8320c2f7|FAIL
exit1_fail|e403d278|FAIL
exit1_fail|fdde4c1b|FAIL
exit1_fail|92730abf|FAIL
exit1_fail|91078281|FAIL
exit1_fail|50b9874e|FAIL
exit1_fail|e472c1f0|FAIL
exit1_fail|e6af7448|FAIL
exit1_fail|dff1a613|FAIL
Report Summary (buildtest report summary)
The buildtest report summary command can be used to provide a short summary of the test report.
Shown below is an example output from the report summary.
buildtest report summary
$ buildtest report summary
Report File: /tmp/tmpr88132k2/var/report.json
Total Tests: 102
Total Tests by Names: 28
Number of buildspecs in report: 17
The buildtest report summary --detailed command can be used to provide a summary of the test report
with breakdown statistics of tests including all fail tests, number of tests by name, test runs
and buildspecs in report file. To see a short report use
Shown below is an example output from the report summary –detailed.
buildtest report summary --detailed
$ buildtest report summary --detailed
Report File: /tmp/tmpr88132k2/var/report.json
Total Tests: 102
Total Tests by Names: 28
Number of buildspecs in report: 17
Breakdown by test
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Name ┃ Total Pass ┃ Total Fail ┃ Total Runs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ variables_bash │ 6 │ 0 │ 6 │
│ unskipped │ 1 │ 0 │ 1 │
│ hello_world │ 9 │ 0 │ 9 │
│ ulimit_cputime_unlimited │ 3 │ 0 │ 3 │
│ ulimit_stacksize_unlimited │ 0 │ 3 │ 3 │
│ ulimit_filedescriptor_4096 │ 0 │ 3 │ 3 │
│ ulimit_filelock_unlimited │ 3 │ 0 │ 3 │
│ ulimit_max_user_process_2048 │ 0 │ 3 │ 3 │
│ ulimit_vmsize_unlimited │ 3 │ 0 │ 3 │
│ root_disk_usage │ 4 │ 0 │ 4 │
│ systemd_default_target │ 0 │ 4 │ 4 │
│ kernel_swapusage │ 0 │ 4 │ 4 │
│ string_tag │ 3 │ 0 │ 3 │
│ list_of_strings_tags │ 3 │ 0 │ 3 │
│ exit1_pass │ 8 │ 0 │ 8 │
│ returncode_int_match │ 7 │ 0 │ 7 │
│ exit1_fail │ 0 │ 8 │ 8 │
│ returncode_list_mismatch │ 0 │ 7 │ 7 │
│ circle_area │ 6 │ 0 │ 6 │
│ python_hello │ 4 │ 0 │ 4 │
│ add_numbers │ 1 │ 0 │ 1 │
│ summary_example │ 1 │ 0 │ 1 │
│ tcsh_env_declaration │ 1 │ 0 │ 1 │
│ bash_env_variables │ 1 │ 0 │ 1 │
│ csh_env_declaration │ 1 │ 0 │ 1 │
│ csh_shell │ 1 │ 0 │ 1 │
│ sleep │ 1 │ 1 │ 2 │
│ linux_strict_test │ 1 │ 1 │ 2 │
└──────────────────────────────┴────────────┴────────────┴────────────┘
PASS Tests
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ name ┃ id ┃ executor ┃ state ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ variables_bash │ 136a82f6 │ generic.local.b │ PASS │ 0 │ 0.01711 │
│ │ │ ash │ │ │ │
│ variables_bash │ 016ed495 │ generic.local.b │ PASS │ 0 │ 0.016156 │
│ │ │ ash │ │ │ │
│ variables_bash │ 9048490a │ generic.local.b │ PASS │ 0 │ 0.014824 │
│ │ │ ash │ │ │ │
│ variables_bash │ 89eac98f │ generic.local.b │ PASS │ 0 │ 0.01483 │
│ │ │ ash │ │ │ │
│ variables_bash │ 1cfe762d │ generic.local.b │ PASS │ 0 │ 0.019083 │
│ │ │ ash │ │ │ │
│ variables_bash │ bf470cea │ generic.local.b │ PASS │ 0 │ 0.019896 │
│ │ │ ash │ │ │ │
│ unskipped │ b6c8452f │ generic.local.b │ PASS │ 0 │ 0.010637 │
│ │ │ ash │ │ │ │
│ hello_world │ dbddca39 │ generic.local.b │ PASS │ 0 │ 0.008793 │
│ │ │ ash │ │ │ │
│ hello_world │ 7dd62a7a │ generic.local.b │ PASS │ 0 │ 0.009604 │
│ │ │ ash │ │ │ │
│ hello_world │ 50c2fb9c │ generic.local.b │ PASS │ 0 │ 0.008717 │
│ │ │ ash │ │ │ │
│ hello_world │ f3cc9e60 │ generic.local.b │ PASS │ 0 │ 0.009119 │
│ │ │ ash │ │ │ │
│ hello_world │ adc9f8bf │ generic.local.b │ PASS │ 0 │ 0.008566 │
│ │ │ ash │ │ │ │
│ hello_world │ 099274a7 │ generic.local.b │ PASS │ 0 │ 0.00842 │
│ │ │ ash │ │ │ │
│ hello_world │ dc8965c0 │ generic.local.b │ PASS │ 0 │ 0.008419 │
│ │ │ ash │ │ │ │
│ hello_world │ c4fac494 │ generic.local.b │ PASS │ 0 │ 0.008446 │
│ │ │ ash │ │ │ │
│ hello_world │ da034607 │ generic.local.b │ PASS │ 0 │ 0.008779 │
│ │ │ ash │ │ │ │
│ ulimit_cputime_ │ 90019e12 │ generic.local.b │ PASS │ 0 │ 0.008509 │
│ unlimited │ │ ash │ │ │ │
│ ulimit_cputime_ │ 56c91a09 │ generic.local.b │ PASS │ 0 │ 0.008783 │
│ unlimited │ │ ash │ │ │ │
│ ulimit_cputime_ │ cfb440bb │ generic.local.b │ PASS │ 0 │ 0.008534 │
│ unlimited │ │ ash │ │ │ │
│ ulimit_filelock │ 68fd261a │ generic.local.b │ PASS │ 0 │ 0.008763 │
│ _unlimited │ │ ash │ │ │ │
│ ulimit_filelock │ 9df96bd8 │ generic.local.b │ PASS │ 0 │ 0.00902 │
│ _unlimited │ │ ash │ │ │ │
│ ulimit_filelock │ 934ea4c1 │ generic.local.b │ PASS │ 0 │ 0.008547 │
│ _unlimited │ │ ash │ │ │ │
│ ulimit_vmsize_u │ 91b9c045 │ generic.local.b │ PASS │ 0 │ 0.008487 │
│ nlimited │ │ ash │ │ │ │
│ ulimit_vmsize_u │ 6eb786b9 │ generic.local.b │ PASS │ 0 │ 0.009558 │
│ nlimited │ │ ash │ │ │ │
│ ulimit_vmsize_u │ baaf3ac0 │ generic.local.b │ PASS │ 0 │ 0.009182 │
│ nlimited │ │ ash │ │ │ │
└─────────────────┴──────────┴─────────────────┴───────┴────────────┴──────────┘
FAIL Tests
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ name ┃ id ┃ executor ┃ state ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ ulimit_stacksiz │ ab3ccb43 │ generic.local.b │ FAIL │ 0 │ 0.008582 │
│ e_unlimited │ │ ash │ │ │ │
│ ulimit_stacksiz │ 6d759512 │ generic.local.b │ FAIL │ 0 │ 0.009054 │
│ e_unlimited │ │ ash │ │ │ │
│ ulimit_stacksiz │ e7971401 │ generic.local.b │ FAIL │ 0 │ 0.018494 │
│ e_unlimited │ │ ash │ │ │ │
│ ulimit_filedesc │ 04e97c83 │ generic.local.b │ FAIL │ 0 │ 0.008868 │
│ riptor_4096 │ │ ash │ │ │ │
│ ulimit_filedesc │ a82daa81 │ generic.local.b │ FAIL │ 0 │ 0.009192 │
│ riptor_4096 │ │ ash │ │ │ │
│ ulimit_filedesc │ 80114be3 │ generic.local.b │ FAIL │ 0 │ 0.008787 │
│ riptor_4096 │ │ ash │ │ │ │
│ ulimit_max_user │ 02a05262 │ generic.local.b │ FAIL │ 0 │ 0.008537 │
│ _process_2048 │ │ ash │ │ │ │
│ ulimit_max_user │ f978e32e │ generic.local.b │ FAIL │ 0 │ 0.008662 │
│ _process_2048 │ │ ash │ │ │ │
│ ulimit_max_user │ fce2c6a9 │ generic.local.b │ FAIL │ 0 │ 0.008543 │
│ _process_2048 │ │ ash │ │ │ │
│ systemd_default │ 41284665 │ generic.local.b │ FAIL │ 1 │ 0.166876 │
│ _target │ │ ash │ │ │ │
│ systemd_default │ 9afcbd19 │ generic.local.b │ FAIL │ 1 │ 0.029258 │
│ _target │ │ ash │ │ │ │
│ systemd_default │ 7794a73d │ generic.local.b │ FAIL │ 1 │ 0.028825 │
│ _target │ │ ash │ │ │ │
│ systemd_default │ 218dcfc5 │ generic.local.b │ FAIL │ 1 │ 0.029849 │
│ _target │ │ ash │ │ │ │
│ kernel_swapusag │ 123211f4 │ generic.local.b │ FAIL │ 255 │ 0.082325 │
│ e │ │ ash │ │ │ │
│ kernel_swapusag │ 59bb8969 │ generic.local.b │ FAIL │ 255 │ 0.025824 │
│ e │ │ ash │ │ │ │
│ kernel_swapusag │ a9d0b510 │ generic.local.b │ FAIL │ 255 │ 0.025019 │
│ e │ │ ash │ │ │ │
│ kernel_swapusag │ 8320c2f7 │ generic.local.b │ FAIL │ 255 │ 0.023663 │
│ e │ │ ash │ │ │ │
│ exit1_fail │ e403d278 │ generic.local.b │ FAIL │ 1 │ 0.021001 │
│ │ │ ash │ │ │ │
│ exit1_fail │ fdde4c1b │ generic.local.b │ FAIL │ 1 │ 0.020524 │
│ │ │ ash │ │ │ │
│ exit1_fail │ 92730abf │ generic.local.b │ FAIL │ 1 │ 0.020913 │
│ │ │ ash │ │ │ │
│ exit1_fail │ 91078281 │ generic.local.b │ FAIL │ 1 │ 0.020997 │
│ │ │ ash │ │ │ │
│ exit1_fail │ 50b9874e │ generic.local.b │ FAIL │ 1 │ 0.020609 │
│ │ │ ash │ │ │ │
│ exit1_fail │ e472c1f0 │ generic.local.b │ FAIL │ 1 │ 0.020829 │
│ │ │ ash │ │ │ │
│ exit1_fail │ e6af7448 │ generic.local.b │ FAIL │ 1 │ 0.020887 │
│ │ │ ash │ │ │ │
│ exit1_fail │ dff1a613 │ generic.local.b │ FAIL │ 1 │ 0.020911 │
│ │ │ ash │ │ │ │
└─────────────────┴──────────┴─────────────────┴───────┴────────────┴──────────┘
Inspect Tests Records via buildtest inspect
Note
buildtest it is an alias for buildtest inspect command.
In previous examples we saw how we can retrieve test records using buildtest report which
is printed in table format. We have limited the output to a limited fields however, if you want to analyze a particular,
we have a separate command called buildtest inspect that can be used for inspecting a test record
based on name or id. Shown below is the command usage for buildtest inspect command.
buildtest inspect --help
$ buildtest inspect --help
usage: buildtest [options] [COMMANDS] inspect [-h] ...
optional arguments:
-h, --help show this help message and exit
subcommands:
Inspect Test result based on Test ID or Test Name
buildspec (b)
Inspect a test based on buildspec
name (n) Specify name of test
query (q) Query fields from record
list (ls) List all test names, ids, and corresponding buildspecs
You can report all test names and corresponding ids using buildtest inspect list which
will be used for querying tests by name or id.
buildtest inspect list
$ buildtest inspect list
Test Summary by id, name, buildspec
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ id ┃ name ┃ buildspec ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 136a82f6-5aa6-407e-8279 │ variables_bash │ /home/docs/checkouts/re │
│ -0adebd1ea1a0 │ │ adthedocs.org/user_buil │
│ │ │ ds/buildtest/checkouts/ │
│ │ │ stable/tutorials/vars.y │
│ │ │ ml │
│ b6c8452f-78ec-4010-ac33 │ unskipped │ /home/docs/checkouts/re │
│ -921d9c0aed93 │ │ adthedocs.org/user_buil │
│ │ │ ds/buildtest/checkouts/ │
│ │ │ stable/tutorials/skip_t │
│ │ │ ests.yml │
│ dbddca39-f458-4428-a481 │ hello_world │ /home/docs/checkouts/re │
│ -a79f774d2594 │ │ adthedocs.org/user_buil │
│ │ │ ds/buildtest/checkouts/ │
│ │ │ stable/tutorials/hello_ │
│ │ │ world.yml │
│ 7dd62a7a-7eff-497b-bade │ hello_world │ /home/docs/checkouts/re │
...
You can fetch all builder names via buildtest inspect list --builder which is the format used for
querying test records via buildtest inspect name or buildtest inspect query.
buildtest inspect list --builder
$ buildtest inspect list --builder
variables_bash/136a82f6-5aa6-407e-8279-0adebd1ea1a0
variables_bash/016ed495-2b54-42f0-b358-ac3c4c8af295
variables_bash/9048490a-672e-4bcf-a076-1fd9949a9df1
variables_bash/89eac98f-05a2-4ce1-887a-805f952f8b79
variables_bash/1cfe762d-1953-455c-bd08-0db56f968815
...
If you are interested in parsing output of buildtest inspect list, you can may find the --terse option useful. The output will show
headers followed by entries, the headers can be omitted by specifying --no-header option.
buildtest inspect list --terse
$ buildtest inspect list --terse
id|name|buildspec
136a82f6-5aa6-407e-8279-0adebd1ea1a0|variables_bash|/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml
016ed495-2b54-42f0-b358-ac3c4c8af295|variables_bash|/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml
9048490a-672e-4bcf-a076-1fd9949a9df1|variables_bash|/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml
89eac98f-05a2-4ce1-887a-805f952f8b79|variables_bash|/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml
...
Inspecting Test by Name via buildtest inspect name
The buildtest inspect name expects a list of positional argument that correspond to name
of test you want to query and buildtest will fetch the last record for each named test. Let’s see an example to
illustrate the point. We can see that each test is stored as a JSON format and buildtest keeps track of
metadata for each test such as user, hostname, command, path to output and error file, content of test,
state of test, returncode, etc… In this example, we will retrieve record for test name circle_area which
will print the raw content of the test in JSON format.
buildtest inspect name circle_area
$ buildtest inspect name circle_area
We have detected 1 builders with the following names ['circle_area/e1097174-2b0a-4421-8981-28518a48c1c6']
{
'circle_area': [
{
'e1097174-2b0a-4421-8981-28518a48c1c6': {
'id': 'e1097174',
'full_id': 'e1097174-2b0a-4421-8981-28518a48c1c6',
'description': 'Calculate circle of area given a radius',
'summary': '',
'schemafile': 'script.schema.json',
'executor': 'generic.local.bash',
'compiler': None,
'hostname': 'build-25534835-project-280831-buildtest',
'user': 'docs',
'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174',
'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.sh',
'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/stage',
'command': 'bash circle_area_build.sh',
'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.out',
'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.err',
'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/build-env.txt',
'buildspec_content': 'buildspecs:\n circle_area:\n executor: generic.local.bash\n type: script\n shell: python\n description: "Calculate circle of area given a radius"\n tags: [tutorials, python]\n run: |\n import math\n radius = 2\n area = math.pi * radius * radius\n print("Circle Radius ", radius)\n print("Area of circle ", area)\n',
'test_content': '#!/bin/bash\npython /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/stage/circle_area.py',
'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=circle_area\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/stage/circle_area.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log',
'metrics': {},
'check': {
'regex': None,
'returncode': None,
'runtime': None,
'file_regex': None,
'slurm_job_state': None,
'pbs_job_state': None,
'lsf_job_state': None,
'assert_ge': None,
'assert_gt': None,
'assert_le': None,
'assert_lt': None,
'assert_eq': None,
'assert_ne': None,
'contains': None,
'not_contains': None,
'is_symlink': None,
'exists': None,
'is_dir': None,
'is_file': None,
'file_count': None,
'linecount': None,
'file_linecount': None
},
'tags': 'tutorials python',
'starttime': '2024/09/05 15:35:27',
'endtime': '2024/09/05 15:35:27',
'runtime': '0.054836',
'state': 'PASS',
'returncode': '0',
'output': 'Circle Radius 2\nArea of circle 12.566370614359172\n',
'error': '',
'job': {},
'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area_build.sh'
}
}
]
}
You can query multiple tests as positional arguments in the format: buildtest inspect name <test1> <test2>
In this next example, we will retrieve test records for bash_shell and python_hello.
buildtest inspect name bash_shell python_hello
$ buildtest inspect name bash_shell python_hello
We have detected 1 builders with the following names ['python_hello/293e8e6a-451f-43fd-8cda-4ce6e76f7e06']
{
'python_hello': [
{
'293e8e6a-451f-43fd-8cda-4ce6e76f7e06': {
'id': '293e8e6a',
'full_id': '293e8e6a-451f-43fd-8cda-4ce6e76f7e06',
'description': 'Hello World python',
'summary': '',
'schemafile': 'script.schema.json',
'executor': 'generic.local.bash',
'compiler': None,
'hostname': 'build-25534835-project-280831-buildtest',
'user': 'docs',
'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a',
'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.sh',
'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/stage',
'command': 'bash python_hello_build.sh',
'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.out',
'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.err',
'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/build-env.txt',
'buildspec_content': 'buildspecs:\n python_hello:\n type: script\n description: Hello World python\n executor: generic.local.bash\n tags: python\n run: python hello.py\n\n',
'test_content': '#!/usr/bin/bash \n# Content of run section\npython hello.py',
'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=python_hello\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/stage/python_hello.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log',
'metrics': {},
'check': {
'regex': None,
'returncode': None,
'runtime': None,
'file_regex': None,
'slurm_job_state': None,
'pbs_job_state': None,
'lsf_job_state': None,
'assert_ge': None,
'assert_gt': None,
'assert_le': None,
'assert_lt': None,
'assert_eq': None,
'assert_ne': None,
'contains': None,
'not_contains': None,
'is_symlink': None,
'exists': None,
'is_dir': None,
'is_file': None,
'file_count': None,
'linecount': None,
'file_linecount': None
},
'tags': 'python',
'starttime': '2024/09/05 15:35:27',
'endtime': '2024/09/05 15:35:27',
'runtime': '0.056028',
'state': 'PASS',
'returncode': '0',
'output': 'Hello World\n',
'error': '',
'job': {},
'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello_build.sh'
}
}
]
}
If you want to query all test records for a given name you can use the --all option which is applied to all positional
arguments.
Inspect Test by buildspec via buildtest inspect buildspec
buildtest can fetch records based on buildspec via buildtest inspect buildspec which expects
a list of buildspecs. By default, buildtest will fetch the latest record of each test, but if you
want to fetch all records you can pass the --all option.
In example below we will fetch latest record for all tests in tutorials/vars.yml
buildtest it buildspec tutorials/vars.yml
$ buildtest it buildspec tutorials/vars.yml
{
│ '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml': {
│ │ 'variables_bash': {
│ │ │ 'id': 'bf470cea',
│ │ │ 'full_id': 'bf470cea-0241-463e-977e-e73b6c969678',
│ │ │ 'description': 'Declare shell variables in bash',
│ │ │ 'summary': '',
│ │ │ 'schemafile': 'script.schema.json',
│ │ │ 'executor': 'generic.local.bash',
│ │ │ 'compiler': None,
│ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ 'user': 'docs',
│ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea',
│ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.sh',
│ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage',
│ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.out',
│ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.err',
│ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/build-env.txt',
│ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_9am2sjcz.log',
│ │ │ 'metrics': {},
│ │ │ 'check': {
│ │ │ │ 'regex': None,
│ │ │ │ 'returncode': None,
│ │ │ │ 'runtime': None,
│ │ │ │ 'file_regex': None,
│ │ │ │ 'slurm_job_state': None,
│ │ │ │ 'pbs_job_state': None,
│ │ │ │ 'lsf_job_state': None,
│ │ │ │ 'assert_ge': None,
│ │ │ │ 'assert_gt': None,
│ │ │ │ 'assert_le': None,
│ │ │ │ 'assert_lt': None,
│ │ │ │ 'assert_eq': None,
│ │ │ │ 'assert_ne': None,
│ │ │ │ 'contains': None,
│ │ │ │ 'not_contains': None,
│ │ │ │ 'is_symlink': None,
│ │ │ │ 'exists': None,
│ │ │ │ 'is_dir': None,
│ │ │ │ 'is_file': None,
│ │ │ │ 'file_count': None,
│ │ │ │ 'linecount': None,
│ │ │ │ 'file_linecount': None
│ │ │ },
│ │ │ 'tags': 'tutorials',
│ │ │ 'starttime': '2024/09/05 15:35:35',
│ │ │ 'endtime': '2024/09/05 15:35:35',
│ │ │ 'runtime': '0.019896',
│ │ │ 'state': 'PASS',
│ │ │ 'returncode': '0',
│ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ 'error': '',
│ │ │ 'job': {},
│ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash_build.sh'
│ │ }
│ }
}
buildtest will report an error if an input buildspec is invalid filepath such as one below
$ buildtest it buildspec /tmp/buildspec.yml
buildspec: /tmp/buildspec.yml is not valid file
There are no buildspecs in cache based on input buildspecs: ['/tmp/buildspec.yml']
You can also pass multiple buildspes on the command line and fetch all records for a test. In example below we will fetch all records from buildspecs tutorials/vars.yml and tutorials/hello_world.yml
$ buildtest it buildspec --all tutorials/vars.yml tutorials/hello_world.yml
{
│ '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml': {
│ │ 'variables_bash': [
│ │ │ {
│ │ │ │ 'id': '136a82f6',
│ │ │ │ 'full_id': '136a82f6-5aa6-407e-8279-0adebd1ea1a0',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_ctd07imv.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:33:10',
│ │ │ │ 'endtime': '2024/09/05 15:33:10',
│ │ │ │ 'runtime': '0.01711',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/136a82f6/variables_bash_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '016ed495',
│ │ │ │ 'full_id': '016ed495-2b54-42f0-b358-ac3c4c8af295',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_et5zhon3.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:34:32',
│ │ │ │ 'endtime': '2024/09/05 15:34:32',
│ │ │ │ 'runtime': '0.016156',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/016ed495/variables_bash_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '9048490a',
│ │ │ │ 'full_id': '9048490a-672e-4bcf-a076-1fd9949a9df1',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_i_f8o1oy.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:17',
│ │ │ │ 'endtime': '2024/09/05 15:35:17',
│ │ │ │ 'runtime': '0.014824',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/9048490a/variables_bash_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '89eac98f',
│ │ │ │ 'full_id': '89eac98f-05a2-4ce1-887a-805f952f8b79',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_eso_1xrv.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:18',
│ │ │ │ 'endtime': '2024/09/05 15:35:18',
│ │ │ │ 'runtime': '0.01483',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/89eac98f/variables_bash_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '1cfe762d',
│ │ │ │ 'full_id': '1cfe762d-1953-455c-bd08-0db56f968815',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_cwp8_yzu.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:19',
│ │ │ │ 'endtime': '2024/09/05 15:35:19',
│ │ │ │ 'runtime': '0.019083',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/1cfe762d/variables_bash_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'bf470cea',
│ │ │ │ 'full_id': 'bf470cea-0241-463e-977e-e73b6c969678',
│ │ │ │ 'description': 'Declare shell variables in bash',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage',
│ │ │ │ 'command': 'bash variables_bash_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: this is a literal string\n singlequote: \\\'singlequote\\\'\n doublequote: \\"doublequote\\"\n current_user: "$(whoami)"\n num_files: "`find $HOME -type f -maxdepth 1 | wc -l`"\n multiline_string: |\n Hello my name is Bob \\n\n I am 30 years old\n\n\n run: |\n echo "$X+$Y="$(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo "current user:" $current_user\n echo "number of files:" $num_files\n echo -e $multiline_string\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \nX="1"\nY="2"\nliteralstring="this is a literal string"\nsinglequote="\\\'singlequote\\\'"\ndoublequote="\\"doublequote\\""\ncurrent_user="$(whoami)"\nnum_files="`find $HOME -type f -maxdepth 1 | wc -l`"\nmultiline_string="Hello my name is Bob \\n\nI am 30 years old\n"\n# Content of run section\necho "$X+$Y="$(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho "current user:" $current_user\necho "number of files:" $num_files\necho -e $multiline_string\n',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_9am2sjcz.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:35',
│ │ │ │ 'endtime': '2024/09/05 15:35:35',
│ │ │ │ 'runtime': '0.019896',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': '1+2=3\nthis is a literal string\n\\\'singlequote\\\'\n"doublequote"\ncurrent user: docs\nnumber of files: 4\nHello my name is Bob \n I am 30 years old\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/vars/variables_bash/bf470cea/variables_bash_build.sh'
│ │ │ }
│ │ ]
│ },
│ '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/hello_world.yml': {
│ │ 'hello_world': [
│ │ │ {
│ │ │ │ 'id': 'dbddca39',
│ │ │ │ 'full_id': 'dbddca39-f458-4428-a481-a79f774d2594',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_vaacuw6o.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:33:17',
│ │ │ │ 'endtime': '2024/09/05 15:33:17',
│ │ │ │ 'runtime': '0.008793',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dbddca39/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '7dd62a7a',
│ │ │ │ 'full_id': '7dd62a7a-7eff-497b-bade-d6365112c996',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_7jp_8z2i.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:33:18',
│ │ │ │ 'endtime': '2024/09/05 15:33:18',
│ │ │ │ 'runtime': '0.009604',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/7dd62a7a/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '50c2fb9c',
│ │ │ │ 'full_id': '50c2fb9c-a2b9-4c17-ae18-1d1de1e65c5b',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_j1izeks8.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:34:48',
│ │ │ │ 'endtime': '2024/09/05 15:34:48',
│ │ │ │ 'runtime': '0.008717',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/50c2fb9c/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'f3cc9e60',
│ │ │ │ 'full_id': 'f3cc9e60-7e09-473b-9704-69474caa6456',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_qjs3e5tg.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:00',
│ │ │ │ 'endtime': '2024/09/05 15:35:00',
│ │ │ │ 'runtime': '0.009119',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/f3cc9e60/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'adc9f8bf',
│ │ │ │ 'full_id': 'adc9f8bf-c56a-4b2e-9a92-50159a6407fc',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_woh407t2.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:31',
│ │ │ │ 'endtime': '2024/09/05 15:35:31',
│ │ │ │ 'runtime': '0.008566',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/adc9f8bf/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': '099274a7',
│ │ │ │ 'full_id': '099274a7-67e5-43b9-bbe6-10b07eafcced',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_woh407t2.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:30',
│ │ │ │ 'endtime': '2024/09/05 15:35:31',
│ │ │ │ 'runtime': '0.00842',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/099274a7/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'dc8965c0',
│ │ │ │ 'full_id': 'dc8965c0-91cc-4f4b-a928-ba1284959f16',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_woh407t2.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:31',
│ │ │ │ 'endtime': '2024/09/05 15:35:31',
│ │ │ │ 'runtime': '0.008419',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/dc8965c0/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'c4fac494',
│ │ │ │ 'full_id': 'c4fac494-0d05-4a18-928b-0ff4735b5ff6',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_woh407t2.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:30',
│ │ │ │ 'endtime': '2024/09/05 15:35:30',
│ │ │ │ 'runtime': '0.008446',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/c4fac494/hello_world_build.sh'
│ │ │ },
│ │ │ {
│ │ │ │ 'id': 'da034607',
│ │ │ │ 'full_id': 'da034607-2a2a-42ca-ad2e-2f0c2bcd071d',
│ │ │ │ 'description': 'hello world example',
│ │ │ │ 'summary': '',
│ │ │ │ 'schemafile': 'script.schema.json',
│ │ │ │ 'executor': 'generic.local.bash',
│ │ │ │ 'compiler': None,
│ │ │ │ 'hostname': 'build-25534835-project-280831-buildtest',
│ │ │ │ 'user': 'docs',
│ │ │ │ 'testroot': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607',
│ │ │ │ 'testpath': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/hello_world.sh',
│ │ │ │ 'stagedir': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/stage',
│ │ │ │ 'command': 'bash hello_world_build.sh',
│ │ │ │ 'outfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/hello_world.out',
│ │ │ │ 'errfile': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/hello_world.err',
│ │ │ │ 'buildenv': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/build-env.txt',
│ │ │ │ 'buildspec_content': 'buildspecs:\n hello_world:\n executor: generic.local.bash\n type: script\n tags: tutorials\n description: "hello world example"\n run: echo "hello world!"\nmaintainers:\n- "@shahzebsiddiqui"\n',
│ │ │ │ 'test_content': '#!/usr/bin/bash \n# Content of run section\necho "hello world!"',
│ │ │ │ 'buildscript_content': '#!/bin/bash\n\n# Function to handle all signals and perform cleanup\nfunction cleanup() {\n echo "Signal trapped. Performing cleanup before exiting."\n exitcode=$?\n echo "buildtest: command \\`$BASH_COMMAND\' failed (exit code: $exitcode)"\n exit $exitcode\n}\n\n# Trap all signals and call the cleanup function\ntrap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU\n\nexport BUILDTEST_TEST_NAME=hello_world\nexport BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials\nexport BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/stage\n# source executor startup script\nsource /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\nbash /tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/stage/hello_world.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode',
│ │ │ │ 'logpath': '/tmp/tmpr88132k2/var/logs/buildtest_woh407t2.log',
│ │ │ │ 'metrics': {},
│ │ │ │ 'check': {
│ │ │ │ │ 'regex': None,
│ │ │ │ │ 'returncode': None,
│ │ │ │ │ 'runtime': None,
│ │ │ │ │ 'file_regex': None,
│ │ │ │ │ 'slurm_job_state': None,
│ │ │ │ │ 'pbs_job_state': None,
│ │ │ │ │ 'lsf_job_state': None,
│ │ │ │ │ 'assert_ge': None,
│ │ │ │ │ 'assert_gt': None,
│ │ │ │ │ 'assert_le': None,
│ │ │ │ │ 'assert_lt': None,
│ │ │ │ │ 'assert_eq': None,
│ │ │ │ │ 'assert_ne': None,
│ │ │ │ │ 'contains': None,
│ │ │ │ │ 'not_contains': None,
│ │ │ │ │ 'is_symlink': None,
│ │ │ │ │ 'exists': None,
│ │ │ │ │ 'is_dir': None,
│ │ │ │ │ 'is_file': None,
│ │ │ │ │ 'file_count': None,
│ │ │ │ │ 'linecount': None,
│ │ │ │ │ 'file_linecount': None
│ │ │ │ },
│ │ │ │ 'tags': 'tutorials',
│ │ │ │ 'starttime': '2024/09/05 15:35:30',
│ │ │ │ 'endtime': '2024/09/05 15:35:30',
│ │ │ │ 'runtime': '0.008779',
│ │ │ │ 'state': 'PASS',
│ │ │ │ 'returncode': '0',
│ │ │ │ 'output': 'hello world!\n',
│ │ │ │ 'error': '',
│ │ │ │ 'job': {},
│ │ │ │ 'build_script': '/tmp/tmpr88132k2/var/tests/generic.local.bash/hello_world/hello_world/da034607/hello_world_build.sh'
│ │ │ }
│ │ ]
│ }
}
Note
If you pass a valid filepath but file is not in cache you will get an error as follows
$ buildtest it buildspec $BUILDTEST_ROOT/README.rst
Unable to find any buildspecs in cache, please specify one of the following buildspecs:
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/skip_tests.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/hello_world.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/general_tests/configuration/ulimits.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/general_tests/configuration/disk_usage.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/general_tests/configuration/systemd-default-target.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/general_tests/configuration/kernel_state.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/tags_example.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/test_status/pass_returncode.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-shell.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-hello.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/add_numbers.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/summary_example.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/environment.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/csh_shell_examples.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/sleep.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/strict_example.yml
Query Test Records via buildtest inspect query
The buildtest inspect query command can allow you to retrieve query certain fields from
each test records that can be useful when you are inspecting a test. Currently, we can
fetch content of output file, error file, testpath, and build script. Shown below are the list
of available options for buildtest inspect query.
buildtest inspect --help
$ buildtest inspect query --help
usage: buildtest [options] [COMMANDS] inspect query [-h] [--pager]
[--theme Color Themes]
[-b] [-be] [-e] [-o] [-t]
[name ...]
positional arguments:
name Name of builder to query in report file
optional arguments:
-h, --help show this help message and exit
--pager Enable PAGING when viewing result
--theme Color Themes Specify a color theme, Pygments style to use when
displaying output. See
https://pygments.org/docs/styles/#getting-a-list-of-
available-styles for available themese
-b, --buildscript Print build script
-be, --buildenv Print content of build env
-e, --error Print error file
-o, --output Print output file
-t, --testpath Print content of testpath
The buildtest inspect query command expects positional arguments that are name of tests
which you can get by running buildtest inspect list.
For instance, let’s query the test circle_area by running the following:
buildtest inspect query circle_area
$ buildtest inspect query circle_area
─────────────── circle_area/e1097174-2b0a-4421-8981-28518a48c1c6 ───────────────
Executor: generic.local.bash
Description: Calculate circle of area given a radius
State: PASS
Returncode: 0
Runtime: 0.054836 sec
Starttime: 2024/09/05 15:35:27
Endtime: 2024/09/05 15:35:27
Command: bash circle_area_build.sh
Test Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.sh
Build Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area_build.sh
Output File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.out
Error File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.err
Log File: /tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log
buildtest will display metadata for each test. By default, buildtest will report the last run for each test that is specified as a positional argument.
You can retrieve content of output file via --output or short option -o. In this command, we retrieve the last run for circle_area and
print content of output file
buildtest inspect query -o circle_area
$ buildtest inspect query -o circle_area
─────────────── circle_area/e1097174-2b0a-4421-8981-28518a48c1c6 ───────────────
Executor: generic.local.bash
Description: Calculate circle of area given a radius
State: PASS
Returncode: 0
Runtime: 0.054836 sec
Starttime: 2024/09/05 15:35:27
Endtime: 2024/09/05 15:35:27
Command: bash circle_area_build.sh
Test Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.sh
Build Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area_build.sh
Output File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.out
Error File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.err
Log File: /tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log
──────────────────────────────── Output File: ─────────────────────────────────
Circle Radius 2
Area of circle 12.566370614359172
────────────────────────────────────────────────────────────────────────────────
If you want to see content of error file use the -e or --error flag. It would be useful to inspect
content of build script and generated test, which can be retrieved using -t and --buildscript. Let’s
query test circle_area and report all of the content fields
buildtest inspect query -o -e -t -b circle_area
$ buildtest inspect query -o -e -t -b circle_area
─────────────── circle_area/e1097174-2b0a-4421-8981-28518a48c1c6 ───────────────
Executor: generic.local.bash
Description: Calculate circle of area given a radius
State: PASS
Returncode: 0
Runtime: 0.054836 sec
Starttime: 2024/09/05 15:35:27
Endtime: 2024/09/05 15:35:27
Command: bash circle_area_build.sh
Test Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.sh
Build Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area_build.sh
Output File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.out
Error File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/circle_area.err
Log File: /tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log
──────────────────────────────── Output File: ─────────────────────────────────
Circle Radius 2
Area of circle 12.566370614359172
────────────────────────────────────────────────────────────────────────────────
───────────────────────────────── Error File: ─────────────────────────────────
────────────────────────────────────────────────────────────────────────────────
───────────────────────────────── Test File: ──────────────────────────────────
#!/bin/bash
python /tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1
────────────────────────────────────────────────────────────────────────────────
───────────────────────────── Build Script File: ──────────────────────────────
#!/bin/bash
# Function to handle all signals and perform cleanup
function cleanup() {
echo "Signal trapped. Performing cleanup before exiting."
exitcode=$?
echo "buildtest: command \`$BASH_COMMAND' failed (exit code: $exitcode)"
exit $exitcode
}
# Trap all signals and call the cleanup function
trap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTE
export BUILDTEST_TEST_NAME=circle_area
export BUILDTEST_TEST_ROOT=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-
export BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/
export BUILDTEST_STAGE_DIR=/tmp/tmpr88132k2/var/tests/generic.local.bash/python-
# source executor startup script
source /tmp/tmpr88132k2/var/executor/generic.local.bash/before_script.sh
# Run generated script
/tmp/tmpr88132k2/var/tests/generic.local.bash/python-shell/circle_area/e1097174/
# Get return code
returncode=$?
# Exit with return code
exit $returncode
────────────────────────────────────────────────────────────────────────────────
We can query multiple tests using buildtest inspect query since each test is a positional argument. Any
options specified to buildtest inspect query will be applied to all test. For instance, let’s fetch the output the
of test names root_disk_usage and python_hello
buildtest inspect query -o root_disk_usage python_hello
$ buildtest inspect query -o root_disk_usage python_hello
───────────── root_disk_usage/3edc011c-6764-4e8f-a590-a12cc2bb0d43 ─────────────
Executor: generic.local.bash
Description: Check root disk usage and report if it exceeds threshold
State: PASS
Returncode: 0
Runtime: 0.01577 sec
Starttime: 2024/09/05 15:34:34
Endtime: 2024/09/05 15:34:34
Command: bash root_disk_usage_build.sh
Test Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/disk_usage/root_disk_usage/3edc011c/root_disk_usage.sh
Build Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/disk_usage/root_disk_usage/3edc011c/root_disk_usage_build.sh
Output File: /tmp/tmpr88132k2/var/tests/generic.local.bash/disk_usage/root_disk_usage/3edc011c/root_disk_usage.out
Error File: /tmp/tmpr88132k2/var/tests/generic.local.bash/disk_usage/root_disk_usage/3edc011c/root_disk_usage.err
Log File: /tmp/tmpr88132k2/var/logs/buildtest_ou3_sina.log
──────────────────────────────── Output File: ─────────────────────────────────
[OK] Root disk is below threshold of 90%
────────────────────────────────────────────────────────────────────────────────
────────────── python_hello/293e8e6a-451f-43fd-8cda-4ce6e76f7e06 ───────────────
Executor: generic.local.bash
Description: Hello World python
State: PASS
Returncode: 0
Runtime: 0.056028 sec
Starttime: 2024/09/05 15:35:27
Endtime: 2024/09/05 15:35:27
Command: bash python_hello_build.sh
Test Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.sh
Build Script: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello_build.sh
Output File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.out
Error File: /tmp/tmpr88132k2/var/tests/generic.local.bash/python-hello/python_hello/293e8e6a/python_hello.err
Log File: /tmp/tmpr88132k2/var/logs/buildtest_go3qugj2.log
──────────────────────────────── Output File: ─────────────────────────────────
Hello World
────────────────────────────────────────────────────────────────────────────────
If you want to query specific test ID, you can specify name of test followed by / and test ID. You don’t need to specify the full ID however tab completion is available to help fill in the names. For example if you want to query test record for circle_area/8edce927-2ecc-4991-ac40-e376c03394b4 shown in tab completion you can type a first few characters to query the record
$ buildtest inspect query circle_area/
circle_area/08f20b50-d2e2-41ab-a75e-a7df75e5afcc circle_area/8edce927-2ecc-4991-ac40-e376c03394b4 circle_area/d47b6ba8-71b6-4531-b8cd-b6ba9b5f0c6c
circle_area/237c3a96-fad0-4ab7-ab1f-3e7ed1816955 circle_area/baea2e9b-a187-4f9f-bcea-75e768ccb0e0 circle_area/e6652700-4cdb-4f6b-80c5-261e4f448876
circle_area/2c279160-1abf-4c70-957f-d9e4608f521b circle_area/bf8f1762-ebf9-458e-92e2-af3fc6e73eac circle_area/e7cc7138-a650-4cd8-aca8-b904f901a0da
$ buildtest inspect query circle_area/8ed
──────────────────────────────────────────────────────────────────────────────────────────── circle_area/8edce927-2ecc-4991-ac40-e376c03394b4 ─────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Calculate circle of area given a radius
State: PASS
Returncode: 0
Runtime: 0.360774 sec
Starttime: 2021/12/23 12:37:25
Endtime: 2021/12/23 12:37:25
Command: bash --norc --noprofile -eo pipefail circle_area_build.sh
Test Script: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/python-shell/circle_area/8edce927/circle_area.sh
Build Script: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/python-shell/circle_area/8edce927/circle_area_build.sh
Output File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/python-shell/circle_area/8edce927/circle_area.out
Error File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/python-shell/circle_area/8edce927/circle_area.err
Log File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/logs/buildtest_c5dnun2l.log
buildtest will search for test ID using re.match so it is possible to apply a regular expression to seek
out multiple test records. The tests must be enclosed in quotes " in-order to have a valid regular expression. Here are few examples that can be useful
# retrieve all test records for name `circle_area`
buildtest inspect query circle_area/
# retrieve test records starting with ID `8a` and `bc` for test name `exit1`
buildtest inspect query "exit1/(8a|bc)"
Using Alternate Report File
The buildtest report and buildtest inspect command will read from the report file tracked by buildtest which is
stored in $BUILDTEST_ROOT/var/report.json. This single file can became an issue if you are running jobs through CI where you
can potentially overwrite same file or if you want separate report files for each set of builds. Luckily we have an option to handle
this using the buildtest -r <report_path> build -b <buildspec_path> option which can be used to specify an alternate location to report file.
buildtest will write the report file in the desired location, then you can specify the path to report file via
buildtest -r <report_path> report and buildtest -r <report_path> inspect to load the report file when reporting tests.
The report file must be valid JSON file that buildtest understands in order to use buildtest report and
buildtest inspect command. Shown below are some examples using the alternate report file using buildtest report and
buildtest inspect command.
$ buildtest -r $BUILDTEST_ROOT/python.json report --format name,id
Report File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/python.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ id ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ circle_area │ a2814554 │
├────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ python_hello │ dd447e43 │
└────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────┘
You can view path to all report files via buildtest report list which keeps track of any new report files created when using buildtest build -r option.
$ buildtest report list
/tmp/tmpr88132k2/var/report.json
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/python.json