Query Test Report¶
buildtest keeps track of all tests and results in a JSON file. This file is read by buildtest report command to extract certain fields from JSON file and display them in table format. We use python tabulate library for pretty print data in tables. Shown below is command usage to query test reports.
$ buildtest report --help
usage: buildtest [options] [COMMANDS] report [-h] [--filter FILTER] [--format FORMAT] [--helpfilter] [--helpformat]
[--latest] [--oldest] [-n] [-r REPORT] [-t]
...
optional arguments:
-h, --help show this help message and exit
--filter FILTER Filter report by filter fields. The filter fields must be a key=value pair and multiple fields
can be comma separated in the following format: --filter key1=val1,key2=val2 . For list of
filter fields run: --helpfilter.
--format FORMAT format field for printing purposes. For more details see --helpformat for list of available
fields. Fields must be separated by comma (usage: --format <field1>,<field2>,...)
--helpfilter List available filter fields to be used with --filter option
--helpformat List of available format fields
--latest Retrieve latest record of particular test
--oldest Retrieve oldest record of particular test
-n, --no-header Don't print headers column used with terse option (--terse).
-r REPORT, --report REPORT
Specify a report file to read
-t, --terse Print output in machine readable format
subcommands:
Fetch test results from report file and print them in table format
clear delete report file
list List all report files
summary Summarize test report
You may run buildtest report without any option, and buildtest will display all test results
with default format fields. To see a list of all format fields, click here.
$ buildtest report
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| name | id | state | returncode | starttime | endtime | runtime | tags | buildspec |
+================================+==========+=========+==============+=====================+=====================+===========+====================+=====================================================================================================================================+
| variables_bash | 1c4ba849 | PASS | 0 | 2021/08/16 22:11:15 | 2021/08/16 22:11:15 | 0.010175 | tutorials | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| variables_bash | 223864f7 | PASS | 0 | 2021/08/16 22:11:44 | 2021/08/16 22:11:44 | 0.008019 | tutorials | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | bdd8e11b | FAIL | 1 | 2021/08/16 22:11:16 | 2021/08/16 22:11:16 | 0.005038 | tutorials fail | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | d2d34e26 | FAIL | 1 | 2021/08/16 22:11:46 | 2021/08/16 22:11:46 | 0.004837 | tutorials fail | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 91fe0aaf | FAIL | 1 | 2021/08/16 22:11:46 | 2021/08/16 22:11:46 | 0.004752 | tutorials fail | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 5544d8dc | FAIL | 1 | 2021/08/16 22:11:50 | 2021/08/16 22:11:50 | 0.005175 | tutorials fail | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 65bfeef1 | FAIL | 1 | 2021/08/16 22:11:50 | 2021/08/16 22:11:50 | 0.004241 | tutorials fail | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+----------+---------+--------------+---------------------+---------------------+-----------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_pass | ad5aea49 | PASS | 1 | 2021/08/16 22:11:16 | 2021/08/16 22:11:16 | 0.006459 | tutorials pass | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
...
Format Reports (buildtest report --format)¶
Available Format Fields (buildtest report --helpformat)¶
The buildtest report command displays a default format fields that can be changed using the
--format option. The report file (JSON) contains many more fields and we expose some of the fields
with the –format option. To see a list of available format fields you can run buildtest report --helpformat.
This option will list all format fields with their description.
$ buildtest report --helpformat
Fields Description
---------- ----------------------------------------------------------------
buildspec Buildspec file
command Command executed
compiler Retrieve compiler used for test (applicable for compiler schema)
endtime End Time for Test in date format
errfile Error File
executor Executor name
hostname Retrieve hostname of machine where job was submitted from
full_id Full qualified unique build identifier
id Unique Build Identifier (abbreviated)
metrics List all metrics if applicable
name Name of test defined in buildspec
outfile Output file
returncode Return Code from Test Execution
runtime Total runtime in seconds
schemafile Schema file used for validation
starttime Start Time of test in date format
state Test State reported by buildtest (PASS/FAIL)
tags Tag name
testroot Root of test directory
testpath Path to test
user Get user who submitted job
Format Field Usage¶
The --format field are specified in comma separated format (i.e --format <field1>,<field2>).
In this example we format table by fields --format id,executor,state,returncode.
$ buildtest report --format name,id,executor,state,returncode
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+--------------------------------+----------+----------------------+---------+--------------+
| name | id | executor | state | returncode |
+================================+==========+======================+=========+==============+
| variables_bash | 1c4ba849 | generic.local.bash | PASS | 0 |
+--------------------------------+----------+----------------------+---------+--------------+
| variables_bash | 223864f7 | generic.local.bash | PASS | 0 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_fail | bdd8e11b | generic.local.sh | FAIL | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_fail | d2d34e26 | generic.local.sh | FAIL | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_fail | 91fe0aaf | generic.local.sh | FAIL | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_fail | 5544d8dc | generic.local.sh | FAIL | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_fail | 65bfeef1 | generic.local.sh | FAIL | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
| exit1_pass | ad5aea49 | generic.local.sh | PASS | 1 |
+--------------------------------+----------+----------------------+---------+--------------+
...
Filter Reports (buildtest report --filter)¶
The buildtest report command will display all tests results, which can be quite long depending on number of tests
so therefore we need a mechanism to filter the test results. The --filter option can be used
to filter out tests in the output based on filter fields. First, lets see the available filter fields
by run buildtest report --helpfilter which shows a list of filter fields and their description.
$ buildtest report --helpfilter
Filter Fields Description Expected Value
--------------- -------------------------- ----------------
buildspec Filter by buildspec file FILE
name Filter by test name STRING
executor Filter by executor name STRING
state Filter by test state PASS/FAIL
tags Filter tests by tag name STRING
returncode Filter tests by returncode INT
The --filter option expects arguments in key=value format. You can
specify multiple filter delimited by comma. buildtest will treat multiple
filters as logical AND operation. The filter option can be used with
--format field. Let’s see some examples to illustrate the point.
Filter by returncode (--filter returncode)¶
If you want to retrieve all tests with a given returncode, we can use the returncode
property. For instance, let’s retrieve all tests with returncode of 2 by setting --filter returncode=2.
$ buildtest report --filter returncode=2 --format=name,id,returncode
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+--------------------------+----------+--------------+
| name | id | returncode |
+==========================+==========+==============+
| returncode_list_mismatch | dd25afaf | 2 |
+--------------------------+----------+--------------+
| returncode_list_mismatch | 504a0b7b | 2 |
+--------------------------+----------+--------------+
| returncode_list_mismatch | c5e18c4a | 2 |
+--------------------------+----------+--------------+
| returncode_list_mismatch | 0dc69200 | 2 |
+--------------------------+----------+--------------+
| returncode_list_mismatch | bb66d1f5 | 2 |
+--------------------------+----------+--------------+
Note
buildtest automatically converts returncode to integer when matching returncode, so --filter returncode="2" will work too
Filter by test name (--filter name)¶
If you want to filter by test name, use the name attribute in filter option. Let’s assume
we want to filter all tests by name exit1_pass, this can be achieved by setting filter
field as follows: --filter name=exit1_pass. Shown below is an example using name filter field
to filter test results.
$ buildtest report --filter name=exit1_pass --format=name,id,returncode,state
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+------------+----------+--------------+---------+
| name | id | returncode | state |
+============+==========+==============+=========+
| exit1_pass | ad5aea49 | 1 | PASS |
+------------+----------+--------------+---------+
| exit1_pass | f150bc31 | 1 | PASS |
+------------+----------+--------------+---------+
| exit1_pass | e9a8bc57 | 1 | PASS |
+------------+----------+--------------+---------+
| exit1_pass | a0c6f68f | 1 | PASS |
+------------+----------+--------------+---------+
| exit1_pass | 1e4c9d01 | 1 | PASS |
+------------+----------+--------------+---------+
| exit1_pass | ae27ef4a | 1 | PASS |
+------------+----------+--------------+---------+
Filter by buildspec (--filter buildspec)¶
Likewise, we can filter results by buildspec file using buildspec attribute via
--filter buildspec=<file>. The buildspec attribute must resolve to a file path which can be
relative or absolute path. buildtest will resolve path (absolute path) and find the appropriate
tests that belong to the buildspec file. If file doesn’t exist or is not found in cache it will raise an error.
$ buildtest report --filter buildspec=tutorials/python-hello.yml --format=name,id,state,buildspec
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+--------------+----------+---------+---------------------------------------------------------------------------------------------------------+
| name | id | state | buildspec |
+==============+==========+=========+=========================================================================================================+
| python_hello | 75d6ff53 | PASS | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/python-hello.yml |
+--------------+----------+---------+---------------------------------------------------------------------------------------------------------+
| python_hello | 3af45be7 | PASS | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/python-hello.yml |
+--------------+----------+---------+---------------------------------------------------------------------------------------------------------+
Filter by test state (--filter state)¶
If you want to filter results by test state, use the state property. This can be
useful if you want to know all pass or failed tests. The state property expects
value of [PASS|FAIL] since these are the two recorded test states marked by buildtest.
We can also pass multiple filter fields for instance if we want to find all FAIL
tests for executor generic.local.sh we can do the following.
$ buildtest report --filter state=FAIL,executor=generic.local.sh --format=name,id,state,executor
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
+--------------------------------+----------+---------+------------------+
| name | id | state | executor |
+================================+==========+=========+==================+
| exit1_fail | bdd8e11b | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| exit1_fail | d2d34e26 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| exit1_fail | 91fe0aaf | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| exit1_fail | 5544d8dc | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| exit1_fail | 65bfeef1 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| returncode_list_mismatch | dd25afaf | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| returncode_list_mismatch | 504a0b7b | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| returncode_list_mismatch | c5e18c4a | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| returncode_list_mismatch | 0dc69200 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| returncode_list_mismatch | bb66d1f5 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| timelimit_min_fail | 73a48836 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| timelimit_max_fail | 1f869483 | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
| status_returncode_by_executors | 25ae782d | FAIL | generic.local.sh |
+--------------------------------+----------+---------+------------------+
Filter Exception Cases¶
The returncode filter field expects an integer value, so if you try a non-integer
returncode you will get the following message
$ buildtest report --filter returncode=1.5
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/bin/buildtest", line 17, in <module>
buildtest.main.main()
File "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/buildtest/main.py", line 142, in main
report_cmd(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/buildtest/cli/report.py", line 570, in report_cmd
report_file=args.report,
File "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/buildtest/cli/report.py", line 85, in __init__
self._check_filter_fields()
File "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/buildtest/cli/report.py", line 119, in _check_filter_fields
f"Invalid returncode:{self.filter[key]} must be an integer"
buildtest.exceptions.BuildTestError: 'Invalid returncode:1.5 must be an integer'
The state filter field expects value of PASS or FAIL so if you specify an
invalid state you will get an error as follows.
$ buildtest report --filter state=UNKNOWN
filter argument 'state' must be 'PASS' or 'FAIL' got value UNKNOWN
The buildspec field expects a valid file path, it can be an absolute or relative
path, buildtest will resolve absolute path and check if file exist and is in the report
file. If it’s an invalid file we get an error such as
$ buildtest report --filter buildspec=/path/to/invalid.yml
Invalid File Path for filter field 'buildspec': /path/to/invalid.yml
You may have a valid filepath for buildspec filter field such as
$BUILDTEST_ROOT/tutorials/invalid_executor.yml, but there is no record of a test in the report cache
because this test wasn’t run. In this case you will get the following message.
$ buildtest report --filter buildspec=$BUILDTEST_ROOT/tutorials/invalid_executor.yml
buildspec file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/invalid_executor.yml not found in cache
Find Latest or Oldest test¶
We can search for oldest or latest test for any given test. This can be useful if you
want to see first or last test run. If you want to retrieve the oldest
test you can use --oldest option. buildtest will append tests, therefore last
record in dictionary will be latest record, similarly first record is the oldest record.
Let’s take a look at this example, we filter by test name hello_f which retrieves
three entries. Now let’s filter by oldest record by specifying –oldest option
and it will retrieve the first record which is test id 349f3ada.
$ buildtest report --filter name=hello_f --format name,id,starttime
Reading Report File: /Users/siddiq90/.buildtest/report.json
+---------+----------+---------------------+
| name | id | starttime |
+=========+==========+=====================+
| hello_f | 349f3ada | 2021/02/11 18:13:08 |
+---------+----------+---------------------+
| hello_f | ecd4a3f2 | 2021/02/11 18:13:18 |
+---------+----------+---------------------+
| hello_f | 5c87978b | 2021/02/11 18:13:33 |
+---------+----------+---------------------+
$ buildtest report --filter name=hello_f --format name,id,starttime --oldest
Reading Report File: /Users/siddiq90/.buildtest/report.json
+---------+----------+---------------------+
| name | id | starttime |
+=========+==========+=====================+
| hello_f | 349f3ada | 2021/02/11 18:13:08 |
+---------+----------+---------------------+
If you want to retrieve the latest test result you can use --latest option which
will retrieve the last record, in the same example we will retrieve test id 5c87978b.
$ buildtest report --filter name=hello_f --format name,id,starttime --latest
Reading Report File: /Users/siddiq90/.buildtest/report.json
+---------+----------+---------------------+
| name | id | starttime |
+=========+==========+=====================+
| hello_f | 5c87978b | 2021/02/11 18:13:33 |
+---------+----------+---------------------+
You may combine –oldest and –latest options in same command, in this case buildtest will retrieve the first and last record of every test.
$ buildtest report --format name,id,starttime --oldest --latest | more
Reading Report File: /Users/siddiq90/.buildtest/report.json
+------------------------------+----------+---------------------+
| name | id | starttime |
+==============================+==========+=====================+
| variables_bash | 750f48bc | 2021/02/11 18:13:03 |
+------------------------------+----------+---------------------+
| variables_bash | 1bdfd403 | 2021/02/11 18:13:32 |
+------------------------------+----------+---------------------+
| ulimit_filelock_unlimited | b7b852e4 | 2021/02/11 18:13:03 |
+------------------------------+----------+---------------------+
| ulimit_filelock_unlimited | 56345a43 | 2021/02/11 18:13:18 |
+------------------------------+----------+---------------------+
Terse Output¶
If you would like to parse the result of buildtest report, you can use the --terse or -t option which
will print the report in machine readable format that shows the name of each column followed by each entry. Each entry
is delimited by PIPE symbol (|). The --terse option works with --format and --filter option. In this
next example, we report all FAIL tests in terse output. The first line is the header of tables followed by
output, if you want to disable output of header you can use --no-header option.
$ buildtest report --filter state=FAIL --format=name,id,state -t
name|id|state
exit1_fail|bdd8e11b|FAIL
exit1_fail|d2d34e26|FAIL
exit1_fail|91fe0aaf|FAIL
exit1_fail|5544d8dc|FAIL
exit1_fail|65bfeef1|FAIL
returncode_list_mismatch|dd25afaf|FAIL
returncode_list_mismatch|504a0b7b|FAIL
returncode_list_mismatch|c5e18c4a|FAIL
returncode_list_mismatch|0dc69200|FAIL
returncode_list_mismatch|bb66d1f5|FAIL
status_regex_fail|4a85e442|FAIL
timelimit_min_fail|73a48836|FAIL
timelimit_max_fail|1f869483|FAIL
status_returncode_by_executors|25ae782d|FAIL
ulimit_stacksize_unlimited|0e951b96|FAIL
ulimit_stacksize_unlimited|de1f6873|FAIL
ulimit_filedescriptor_4096|c37071b3|FAIL
ulimit_filedescriptor_4096|6f0b9f41|FAIL
ulimit_max_user_process_2048|28118dbe|FAIL
ulimit_max_user_process_2048|7fc52728|FAIL
systemd_default_target|9061f933|FAIL
systemd_default_target|5ccc431b|FAIL
systemd_default_target|185f833c|FAIL
ssh_localhost_remotecommand|ce0e5732|FAIL
ssh_localhost_remotecommand|ce350fe9|FAIL
ssh_localhost_remotecommand|c5e27d66|FAIL
kernel_swapusage|18c8b2a2|FAIL
kernel_swapusage|168713a5|FAIL
kernel_swapusage|e0458d95|FAIL
list_of_strings_tags|b497ac17|FAIL
Report Summary (buildtest report summary)¶
The buildtest report summary command can be used to provide a summary of the test report
with breakdown statistics of tests including all fail tests, number of tests by name, test runs
and buildspecs in report file.
Shown below is an example output from the report summary.
$ buildtest report summary
Report: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/report.json
Total Tests: 79
Total Tests by Names: 37
Number of buildspecs in report: 22
Breakdown by Test
+--------------------------------+--------+--------+--------+
| name | runs | pass | fail |
+================================+========+========+========+
| variables_bash | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| exit1_fail | 5 | 0 | 5 |
+--------------------------------+--------+--------+--------+
| exit1_pass | 6 | 6 | 0 |
+--------------------------------+--------+--------+--------+
| returncode_list_mismatch | 5 | 0 | 5 |
+--------------------------------+--------+--------+--------+
| returncode_int_match | 6 | 6 | 0 |
+--------------------------------+--------+--------+--------+
| status_regex_pass | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| status_regex_fail | 1 | 0 | 1 |
+--------------------------------+--------+--------+--------+
| timelimit_min_max | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| timelimit_min | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| timelimit_max | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| timelimit_min_fail | 1 | 0 | 1 |
+--------------------------------+--------+--------+--------+
| timelimit_max_fail | 1 | 0 | 1 |
+--------------------------------+--------+--------+--------+
| bash_login_shebang | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| bash_nonlogin_shebang | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| unskipped | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| metric_regex_example | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| metric_variable_assignment | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| executor_regex_script_schema | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| executors_vars_env_declaration | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| executors_sbatch_declaration | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| status_returncode_by_executors | 2 | 1 | 1 |
+--------------------------------+--------+--------+--------+
| root_disk_usage | 3 | 3 | 0 |
+--------------------------------+--------+--------+--------+
| ulimit_filelock_unlimited | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| ulimit_cputime_unlimited | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| ulimit_stacksize_unlimited | 2 | 0 | 2 |
+--------------------------------+--------+--------+--------+
| ulimit_vmsize_unlimited | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| ulimit_filedescriptor_4096 | 2 | 0 | 2 |
+--------------------------------+--------+--------+--------+
| ulimit_max_user_process_2048 | 2 | 0 | 2 |
+--------------------------------+--------+--------+--------+
| systemd_default_target | 3 | 0 | 3 |
+--------------------------------+--------+--------+--------+
| ssh_localhost_remotecommand | 3 | 0 | 3 |
+--------------------------------+--------+--------+--------+
| kernel_swapusage | 3 | 0 | 3 |
+--------------------------------+--------+--------+--------+
| string_tag | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| list_of_strings_tags | 1 | 0 | 1 |
+--------------------------------+--------+--------+--------+
| circle_area | 5 | 5 | 0 |
+--------------------------------+--------+--------+--------+
| python_hello | 2 | 2 | 0 |
+--------------------------------+--------+--------+--------+
| run_only_platform_linux | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
| hello_world | 1 | 1 | 0 |
+--------------------------------+--------+--------+--------+
FAIL test
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| name | id | executor | state | returncode | runtime |
+================================+==========+====================+=========+==============+===========+
| exit1_fail | bdd8e11b | generic.local.sh | FAIL | 1 | 0.005038 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| exit1_fail | d2d34e26 | generic.local.sh | FAIL | 1 | 0.004837 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| exit1_fail | 91fe0aaf | generic.local.sh | FAIL | 1 | 0.004752 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| exit1_fail | 5544d8dc | generic.local.sh | FAIL | 1 | 0.005175 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| exit1_fail | 65bfeef1 | generic.local.sh | FAIL | 1 | 0.004241 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| returncode_list_mismatch | dd25afaf | generic.local.sh | FAIL | 2 | 0.004326 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| returncode_list_mismatch | 504a0b7b | generic.local.sh | FAIL | 2 | 0.004391 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| returncode_list_mismatch | c5e18c4a | generic.local.sh | FAIL | 2 | 0.004253 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| returncode_list_mismatch | 0dc69200 | generic.local.sh | FAIL | 2 | 0.004408 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| returncode_list_mismatch | bb66d1f5 | generic.local.sh | FAIL | 2 | 0.006526 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| status_regex_fail | 4a85e442 | generic.local.bash | FAIL | 0 | 0.004481 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| timelimit_min_fail | 73a48836 | generic.local.sh | FAIL | 0 | 2.00672 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| timelimit_max_fail | 1f869483 | generic.local.sh | FAIL | 0 | 3.0067 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| status_returncode_by_executors | 25ae782d | generic.local.sh | FAIL | 0 | 0.004514 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_stacksize_unlimited | 0e951b96 | generic.local.bash | FAIL | 0 | 0.004415 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_stacksize_unlimited | de1f6873 | generic.local.bash | FAIL | 0 | 0.004314 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_filedescriptor_4096 | c37071b3 | generic.local.bash | FAIL | 0 | 0.00433 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_filedescriptor_4096 | 6f0b9f41 | generic.local.bash | FAIL | 0 | 0.004223 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_max_user_process_2048 | 28118dbe | generic.local.bash | FAIL | 0 | 0.004334 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ulimit_max_user_process_2048 | 7fc52728 | generic.local.bash | FAIL | 0 | 0.004229 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| systemd_default_target | 9061f933 | generic.local.bash | FAIL | 1 | 0.005545 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| systemd_default_target | 5ccc431b | generic.local.bash | FAIL | 1 | 0.005941 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| systemd_default_target | 185f833c | generic.local.bash | FAIL | 1 | 0.005331 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ssh_localhost_remotecommand | ce0e5732 | generic.local.bash | FAIL | 255 | 0.105284 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ssh_localhost_remotecommand | ce350fe9 | generic.local.bash | FAIL | 255 | 0.00958 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| ssh_localhost_remotecommand | c5e27d66 | generic.local.bash | FAIL | 255 | 0.010745 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| kernel_swapusage | 18c8b2a2 | generic.local.bash | FAIL | 127 | 0.005565 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| kernel_swapusage | 168713a5 | generic.local.bash | FAIL | 127 | 0.005197 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| kernel_swapusage | e0458d95 | generic.local.bash | FAIL | 127 | 0.00527 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
| list_of_strings_tags | b497ac17 | generic.local.bash | FAIL | 127 | 0.005107 |
+--------------------------------+----------+--------------------+---------+--------------+-----------+
Inspect Tests Records via buildtest inspect¶
In previous examples we saw how we can retrieve test records using buildtest report which
is printed in table format. We have limited the output to a limited fields however, if you want to analyze a particular,
we have a separate command called buildtest inspect that can be used for inspecting a test record
based on name or id. Shown below is the command usage for buildtest inspect command.
$ buildtest inspect --help
usage: buildtest [options] [COMMANDS] inspect [-h] [-r REPORT] ...
optional arguments:
-h, --help show this help message and exit
-r REPORT, --report REPORT
Specify a report file to load when inspecting test
subcommands:
Inspect Test result based on Test ID or Test Name
buildspec Inspect a test based on buildspec
id Specify a Test ID
name Specify name of test
query Query fields from record
list List all test ids
You can report all test names and corresponding ids using buildtest inspect list which
will be used for querying tests by name or id.
$ buildtest inspect list
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| name | id | buildspec |
+================================+======================================+=====================================================================================================================================+
| variables_bash | 1c4ba849-bc6a-4989-8e43-06d38a332531 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| variables_bash | 223864f7-fc35-4d52-813e-85c053fab8c4 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | bdd8e11b-da11-4c13-899f-be68f43de3d7 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | d2d34e26-960f-4231-99f3-5ad8da61caf3 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 91fe0aaf-3369-4006-853a-4369624c95f0 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 5544d8dc-90a9-4506-a76e-3bbe8fa83b3c | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_fail | 65bfeef1-3735-41ce-8fda-1c954a2086d7 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_pass | ad5aea49-2402-4893-a813-da3718f84547 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
+--------------------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| exit1_pass | f150bc31-d94e-4932-977f-5067ebe28c90 | /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml |
...
Inspecting Test by Name via buildtest inspect name¶
The buildtest inspect name expects a list of positional argument that correspond to name
of test you want to query and buildtest will fetch the last record for each named test. Let’s see an example to
illustrate the point. We can see that each test is stored as a JSON format and buildtest keeps track of
metadata for each test such as user, hostname, command, path to output and error file, content of test,
state of test, returncode, etc… In this example, we will retrieve record for test name circle_area which
will print the raw content of the test in JSON format.
$ buildtest inspect name circle_area
{
"circle_area": {
"id": "9b255d3f",
"full_id": "9b255d3f-09f6-4b31-abbd-0f2cbb523d34",
"description": "Calculate circle of area given a radius",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.python",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/stage",
"command": "sh circle_area_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n circle_area:\n executor: generic.local.python\n type: script\n shell: python\n description: \"Calculate circle of area given a radius\"\n tags: [tutorials, python]\n run: |\n import math\n radius = 2\n area = math.pi * radius * radius\n print(\"Circle Radius \", radius)\n print(\"Area of circle \", area)\n",
"test_content": "#!/bin/bash\npython /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/stage/circle_area.py",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=circle_area\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/stage\nexport BUILDTEST_TEST_ID=9b255d3f-09f6-4b31-abbd-0f2cbb523d34\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.python/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/stage/circle_area.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_dq1msjjf.log",
"metrics": {},
"tags": "tutorials python",
"starttime": "2021/08/16 22:11:50",
"endtime": "2021/08/16 22:11:50",
"runtime": "0.043963",
"state": "PASS",
"returncode": "0",
"output": "Circle Radius 2\nArea of circle 12.566370614359172\n",
"error": "circle_area_build.sh: 14: circle_area_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area_build.sh"
}
}
You can query multiple tests as positional arguments in the format: buildtest inspect name <test1> <test2>
In this next example, we will retrieve test records for bash_shell and python_hello.
$ buildtest inspect name bash_shell python_hello
{
"python_hello": {
"id": "3af45be7",
"full_id": "3af45be7-9173-4bd6-b555-f7d8f8380323",
"description": "Hello World python",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/stage",
"command": "sh python_hello_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n python_hello:\n type: script\n description: Hello World python\n executor: generic.local.bash\n tags: python\n run: python hello.py\n\n",
"test_content": "#!/bin/bash \n# Content of run section\npython hello.py",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=python_hello\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/stage\nexport BUILDTEST_TEST_ID=3af45be7-9173-4bd6-b555-f7d8f8380323\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/stage/python_hello.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_mcl96yu0.log",
"metrics": {},
"tags": "python",
"starttime": "2021/08/16 22:11:46",
"endtime": "2021/08/16 22:11:46",
"runtime": "0.043397",
"state": "PASS",
"returncode": "0",
"output": "Hello World\n",
"error": "python_hello_build.sh: 14: python_hello_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello_build.sh"
}
}
If you want to query all test records for a given name you can use the --all option which is applied to all positional
arguments.
Inspect Test by buildspec via buildtest inspect buildspec¶
buildtest can fetch records based on buildspec via buildtest inspect buildspec which expects
a list of buildspecs. By default, buildtest will fetch the latest record of each test, but if you
want to fetch all records you can pass the --all option.
In example below we will fetch latest record for all tests in tutorials/vars.yml
$ buildtest inspect buildspec tutorials/vars.yml
{
"/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml": {
"variables_bash": {
"id": "223864f7",
"full_id": "223864f7-fc35-4d52-813e-85c053fab8c4",
"description": "Declare shell variables in bash",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage",
"command": "sh variables_bash_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: |\n \"this is a literal string ':' \"\n singlequote: \"'singlequote'\"\n doublequote: \"\\\"doublequote\\\"\"\n current_user: \"$(whoami)\"\n files_homedir: \"`find $HOME -type f -maxdepth 1`\"\n\n run: |\n echo \"$X+$Y=\" $(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo $current_user\n echo $files_homedir",
"test_content": "#!/bin/bash \n# Declare shell variables\nX=1\nY=2\nliteralstring=\"this is a literal string ':' \"\n\nsinglequote='singlequote'\ndoublequote=\"doublequote\"\ncurrent_user=$(whoami)\nfiles_homedir=`find $HOME -type f -maxdepth 1`\n\n\n# Content of run section\necho \"$X+$Y=\" $(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho $current_user\necho $files_homedir",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage\nexport BUILDTEST_TEST_ID=223864f7-fc35-4d52-813e-85c053fab8c4\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_c80lh443.log",
"metrics": {},
"tags": "tutorials",
"starttime": "2021/08/16 22:11:44",
"endtime": "2021/08/16 22:11:44",
"runtime": "0.008019",
"state": "PASS",
"returncode": "0",
"output": "1+2= 3\nthis is a literal string ':'\nsinglequote\ndoublequote\ndocs\n/home/docs/.bash_logout /home/docs/.bashrc /home/docs/.profile /home/docs/.wget-hsts\n",
"error": "variables_bash_build.sh: 14: variables_bash_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash_build.sh"
}
}
}
buildtest will report an error if an input buildspec is invalid filepath such as one below
$ buildtest inspect buildspec /tmp/buildspec.yml
buildspec: /tmp/buildspec.yml is not valid file
There are no buildspecs in cache based on input buildspecs: ['/tmp/buildspec.yml']
You can also pass multiple buildspes on the command line and fetch all records for a test. In example below we will fetch all records from tests tutorials/hello_world/yml and tutorials/regex_status.yml
$ buildtest inspect buildspec --all tutorials/vars.yml tutorials/status_regex.yml
{
"/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml": {
"variables_bash": [
{
"id": "1c4ba849",
"full_id": "1c4ba849-bc6a-4989-8e43-06d38a332531",
"description": "Declare shell variables in bash",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/variables_bash.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/stage",
"command": "sh variables_bash_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/variables_bash.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/variables_bash.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: |\n \"this is a literal string ':' \"\n singlequote: \"'singlequote'\"\n doublequote: \"\\\"doublequote\\\"\"\n current_user: \"$(whoami)\"\n files_homedir: \"`find $HOME -type f -maxdepth 1`\"\n\n run: |\n echo \"$X+$Y=\" $(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo $current_user\n echo $files_homedir",
"test_content": "#!/bin/bash \n# Declare shell variables\nX=1\nY=2\nliteralstring=\"this is a literal string ':' \"\n\nsinglequote='singlequote'\ndoublequote=\"doublequote\"\ncurrent_user=$(whoami)\nfiles_homedir=`find $HOME -type f -maxdepth 1`\n\n\n# Content of run section\necho \"$X+$Y=\" $(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho $current_user\necho $files_homedir",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/stage\nexport BUILDTEST_TEST_ID=1c4ba849-bc6a-4989-8e43-06d38a332531\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_hr_5xctx.log",
"metrics": {},
"tags": "tutorials",
"starttime": "2021/08/16 22:11:15",
"endtime": "2021/08/16 22:11:15",
"runtime": "0.010175",
"state": "PASS",
"returncode": "0",
"output": "1+2= 3\nthis is a literal string ':'\nsinglequote\ndoublequote\ndocs\n/home/docs/.bash_logout /home/docs/.bashrc /home/docs/.profile /home/docs/.wget-hsts\n",
"error": "variables_bash_build.sh: 14: variables_bash_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/1c4ba849/variables_bash_build.sh"
},
{
"id": "223864f7",
"full_id": "223864f7-fc35-4d52-813e-85c053fab8c4",
"description": "Declare shell variables in bash",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage",
"command": "sh variables_bash_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: |\n \"this is a literal string ':' \"\n singlequote: \"'singlequote'\"\n doublequote: \"\\\"doublequote\\\"\"\n current_user: \"$(whoami)\"\n files_homedir: \"`find $HOME -type f -maxdepth 1`\"\n\n run: |\n echo \"$X+$Y=\" $(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n echo $current_user\n echo $files_homedir",
"test_content": "#!/bin/bash \n# Declare shell variables\nX=1\nY=2\nliteralstring=\"this is a literal string ':' \"\n\nsinglequote='singlequote'\ndoublequote=\"doublequote\"\ncurrent_user=$(whoami)\nfiles_homedir=`find $HOME -type f -maxdepth 1`\n\n\n# Content of run section\necho \"$X+$Y=\" $(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\necho $current_user\necho $files_homedir",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=variables_bash\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage\nexport BUILDTEST_TEST_ID=223864f7-fc35-4d52-813e-85c053fab8c4\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/stage/variables_bash.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_c80lh443.log",
"metrics": {},
"tags": "tutorials",
"starttime": "2021/08/16 22:11:44",
"endtime": "2021/08/16 22:11:44",
"runtime": "0.008019",
"state": "PASS",
"returncode": "0",
"output": "1+2= 3\nthis is a literal string ':'\nsinglequote\ndoublequote\ndocs\n/home/docs/.bash_logout /home/docs/.bashrc /home/docs/.profile /home/docs/.wget-hsts\n",
"error": "variables_bash_build.sh: 14: variables_bash_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/vars/variables_bash/223864f7/variables_bash_build.sh"
}
]
},
"/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/status_regex.yml": {
"status_regex_pass": [
{
"id": "9694871d",
"full_id": "9694871d-54e1-4ac2-acac-73962899573d",
"description": "Pass test based on regular expression",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/status_regex_pass.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/stage",
"command": "sh status_regex_pass_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/status_regex_pass.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/status_regex_pass.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n status_regex_pass:\n executor: generic.local.bash\n type: script\n tags: [system]\n description: Pass test based on regular expression\n run: echo \"PASS\"\n status:\n regex:\n stream: stdout\n exp: \"^(PASS)$\"\n\n status_regex_fail:\n executor: generic.local.bash\n type: script\n tags: [system]\n description: Pass test based on regular expression\n run: echo \"FAIL\"\n status:\n regex:\n stream: stdout\n exp: \"^(123FAIL)$\"",
"test_content": "#!/bin/bash \n# Content of run section\necho \"PASS\"",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=status_regex_pass\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/stage\nexport BUILDTEST_TEST_ID=9694871d-54e1-4ac2-acac-73962899573d\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/stage/status_regex_pass.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_lywao4up.log",
"metrics": {},
"tags": "system",
"starttime": "2021/08/16 22:11:16",
"endtime": "2021/08/16 22:11:16",
"runtime": "0.00478",
"state": "PASS",
"returncode": "0",
"output": "PASS\n",
"error": "status_regex_pass_build.sh: 14: status_regex_pass_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_pass/9694871d/status_regex_pass_build.sh"
}
],
"status_regex_fail": [
{
"id": "4a85e442",
"full_id": "4a85e442-70e7-4125-9c41-35954b101cb5",
"description": "Pass test based on regular expression",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "build-14488818-project-280831-buildtest",
"user": "docs",
"testroot": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442",
"testpath": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/status_regex_fail.sh",
"stagedir": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/stage",
"command": "sh status_regex_fail_build.sh",
"outfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/status_regex_fail.out",
"errfile": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/status_regex_fail.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n status_regex_pass:\n executor: generic.local.bash\n type: script\n tags: [system]\n description: Pass test based on regular expression\n run: echo \"PASS\"\n status:\n regex:\n stream: stdout\n exp: \"^(PASS)$\"\n\n status_regex_fail:\n executor: generic.local.bash\n type: script\n tags: [system]\n description: Pass test based on regular expression\n run: echo \"FAIL\"\n status:\n regex:\n stream: stdout\n exp: \"^(123FAIL)$\"",
"test_content": "#!/bin/bash \n# Content of run section\necho \"FAIL\"",
"buildscript_content": "#!/bin/bash\n\n\n############# START VARIABLE DECLARATION ########################\nexport BUILDTEST_TEST_NAME=status_regex_fail\nexport BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442\nexport BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials\nexport BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/stage\nexport BUILDTEST_TEST_ID=4a85e442-70e7-4125-9c41-35954b101cb5\n############# END VARIABLE DECLARATION ########################\n\n\n# source executor startup script\nsource /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.bash/before_script.sh\n# Run generated script\n/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/stage/status_regex_fail.sh\n# Get return code\nreturncode=$?\n# Exit with return code\nexit $returncode",
"logpath": "/tmp/buildtest_lywao4up.log",
"metrics": {},
"tags": "system",
"starttime": "2021/08/16 22:11:16",
"endtime": "2021/08/16 22:11:16",
"runtime": "0.004481",
"state": "FAIL",
"returncode": "0",
"output": "FAIL\n",
"error": "status_regex_fail_build.sh: 14: status_regex_fail_build.sh: source: not found\n",
"job": {},
"build_script": "/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/status_regex/status_regex_fail/4a85e442/status_regex_fail_build.sh"
}
]
}
}
If you pass a valid filepath but file is not in cache you will get an error as follows
$ buildtest inspect buildspec $BUILDTEST_ROOT/README.rst
Unable to find any buildspecs in cache, please specify one of the following buildspecs:
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/vars.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/pass_returncode.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/status_regex.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/runtime_status_test.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/shebang.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/skip_tests.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/metrics_regex.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/metrics_variable.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/executor_regex_script.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/script/multiple_executors.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/script/executor_scheduler.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/script/status_by_executors.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/general_tests/configuration/disk_usage.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/general_tests/configuration/ulimits.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/general_tests/configuration/systemd-default-target.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/general_tests/configuration/ssh_localhost.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/general_tests/configuration/kernel_state.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/tags_example.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/python-shell.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/python-hello.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/run_only_platform.yml
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials/hello_world.yml
Inspecting Test by ID via buildtest inspect id¶
The buildtest inspect id works similar to buildtest inspect name except that it
operates on test id. This can be useful if you want to extract a particular test record and not
see all test records at once.
You only need to specify a few characters and buildtest will resolve full test id if there is a match.
The buildtest inspect id can operate on single or multiple ids if you want to specify multiple
ids in single command you can do buildtest inspect id <identifier1> <identifier2>.
Let’s see an example where we query a single test record. Notice, that we only specify a few characters fee and buildtest found a matching record fee66c67-db4e-4d35-8c6d-28ac5cbbaba0
$ buildtest inspect id fee
Reading Report File: /Users/siddiq90/.buildtest/report.json
{
"fee66c67-db4e-4d35-8c6d-28ac5cbbaba0": {
"id": "fee66c67",
"full_id": "fee66c67-db4e-4d35-8c6d-28ac5cbbaba0",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "DOE-7086392.local",
"user": "siddiq90",
"testroot": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2",
"testpath": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/stage/generate.sh",
"stagedir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/stage",
"rundir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/run",
"command": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/stage/generate.sh",
"outfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/run/python_hello.out",
"errfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/python-hello/python_hello/2/run/python_hello.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n python_hello:\n type: script\n description: Hello World python\n executor: generic.local.bash\n tags: python\n run: python hello.py\n\n",
"test_content": "#!/bin/bash \nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.bash/before_script.sh\npython hello.py\nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.bash/after_script.sh",
"tags": "python",
"starttime": "2021/03/31 11:18:21",
"endtime": "2021/03/31 11:18:21",
"runtime": 0.104714,
"state": "PASS",
"returncode": 0,
"output": "Hello World\n",
"error": "",
"job": null
}
}
We can pass multiple IDs to buildtest inspect id and buildtest will retrieve test
record if there is a match. You only need to specify a few characters to ensure we have a unique test
ID and buildtest will retrieve the record.
$ buildtest inspect id 944 a76
Reading Report File: /Users/siddiq90/.buildtest/report.json
{
"a76799db-f11e-4050-8dcb-8b147092c536": {
"id": "a76799db",
"full_id": "a76799db-f11e-4050-8dcb-8b147092c536",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "DOE-7086392.local",
"user": "siddiq90",
"testroot": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0",
"testpath": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/stage/generate.sh",
"stagedir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/stage",
"rundir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/run",
"command": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/stage/generate.sh",
"outfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/run/root_disk_usage.out",
"errfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/disk_usage/root_disk_usage/0/run/root_disk_usage.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n root_disk_usage:\n executor: generic.local.bash\n type: script\n tags: [filesystem, storage]\n description: Check root disk usage and report if it exceeds threshold\n env:\n threshold: 90\n run: |\n root_disk_usage=`df -a / | tail -n 1 | awk '{print $5'} | sed 's/[^0-9]*//g'`\n # if root exceeds threshold\n if [ \"$root_disk_usage\" -gt \"$threshold\" ]; then\n echo \"[WARNING] Root Disk Usage: $root_disk_usage% exceeded threshold of $threshold%\"\n exit 1\n fi\n echo \"[OK] Root disk is below threshold of $threshold%\"\n",
"test_content": "#!/bin/bash \nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.bash/before_script.sh\nexport threshold=90\nroot_disk_usage=`df -a / | tail -n 1 | awk '{print $5'} | sed 's/[^0-9]*//g'`\n# if root exceeds threshold\nif [ \"$root_disk_usage\" -gt \"$threshold\" ]; then\n echo \"[WARNING] Root Disk Usage: $root_disk_usage% exceeded threshold of $threshold%\"\n exit 1\nfi\necho \"[OK] Root disk is below threshold of $threshold%\"\n\nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.bash/after_script.sh",
"tags": "filesystem storage",
"starttime": "2021/03/31 11:17:50",
"endtime": "2021/03/31 11:17:50",
"runtime": 0.114321,
"state": "PASS",
"returncode": 0,
"output": "[OK] Root disk is below threshold of 90%\n",
"error": "",
"job": null
},
"944f6399-b82b-47f9-bb15-8f529dedd4e6": {
"id": "944f6399",
"full_id": "944f6399-b82b-47f9-bb15-8f529dedd4e6",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.python",
"compiler": null,
"hostname": "DOE-7086392.local",
"user": "siddiq90",
"testroot": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0",
"testpath": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/stage/generate.sh",
"stagedir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/stage",
"rundir": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/run",
"command": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/stage/generate.sh",
"outfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/run/circle_area.out",
"errfile": "/Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/run/circle_area.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n circle_area:\n executor: generic.local.python\n type: script\n shell: python\n description: \"Calculate circle of area given a radius\"\n tags: [tutorials, python]\n run: |\n import math\n radius = 2\n area = math.pi * radius * radius\n print(\"Circle Radius \", radius)\n print(\"Area of circle \", area)\n",
"test_content": "#!/bin/bash\nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.python/before_script.sh\npython /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.python/python-shell/circle_area/0/stage/circle_area.py\nsource /Users/siddiq90/Documents/github/buildtest/var/executors/generic.local.python/after_script.sh",
"tags": "tutorials python",
"starttime": "2021/03/31 11:18:00",
"endtime": "2021/03/31 11:18:00",
"runtime": 0.144171,
"state": "PASS",
"returncode": 0,
"output": "Circle Radius 2\nArea of circle 12.566370614359172\n",
"error": "",
"job": null
}
}
If you specify an invalid test id using buildtest inspect id you will get an error
message as follows.
$ buildtest inspect id lad
Unable to find any test records based on id: ['lad'], please run 'buildtest inspect list' to see list of ids.
You will see similar message if you specify an invalid test name using buildtest inspect name command.
Query Test Records via buildtest inspect query¶
The buildtest inspect query command can allow you to retrieve query certain fields from
each test records that can be useful when you are inspecting a test. Currently, we can
fetch content of output file, error file, testpath, and build script. Shown below are the list
of available options for buildtest inspect query.
$ buildtest inspect query --help
usage: buildtest [options] [COMMANDS] inspect query [-h] [-b] [-d {first,last,all}] [-e] [-o] [-t] [name [name ...]]
positional arguments:
name Name of test
optional arguments:
-h, --help show this help message and exit
-b, --buildscript Print build script
-d {first,last,all}, --display {first,last,all}
Determine how records are fetched, by default it will report the last record of the test.
-e, --error Print error file
-o, --output Print output file
-t, --testpath Print content of testpath
The buildtest inspect query command expects positional arguments that are name of tests
which you can get by running buildtest inspect list.
For instance, let’s query the test circle_area by running the following:
$ buildtest inspect query circle_area
______________________________ circle_area (ID: 9b255d3f-09f6-4b31-abbd-0f2cbb523d34) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.043963
starttime: 2021/08/16 22:11:50
endtime: 2021/08/16 22:11:50
buildtest will display metadata for each test. By default, buildtest will report the latest record
for each test that is specified as a positional argument. If you want to see all runs for a particular test
you can use -d all or --display all which will report all records. By default, it will use -d last which
reports the last record. You can retrieve the first record by running -d first which is the oldest record.
Now as you run test, you want to inspect the output file, this can be done by passing -o or --output. Let’s take
what we learned and see the following. In this command, we retrieve all records for circle_area and
print content of output file
$ buildtest inspect query -d all -o circle_area
______________________________ circle_area (ID: 0b6ab2ac-b532-4e7f-9a89-56cbc41ee998) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.04527
starttime: 2021/08/16 22:11:46
endtime: 2021/08/16 22:11:46
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.out *************************
______________________________ circle_area (ID: c8374cbd-5f19-4d2f-9c6f-c80eaa9cd7b1) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.043456
starttime: 2021/08/16 22:11:46
endtime: 2021/08/16 22:11:47
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/c8374cbd/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/c8374cbd/circle_area.out *************************
______________________________ circle_area (ID: a1ef3290-3ab6-47af-8e04-8e98f76788f8) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.044541
starttime: 2021/08/16 22:11:50
endtime: 2021/08/16 22:11:50
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/a1ef3290/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/a1ef3290/circle_area.out *************************
______________________________ circle_area (ID: 45c0e965-0b9e-427a-b684-0a2b7567fb8d) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.043797
starttime: 2021/08/16 22:11:50
endtime: 2021/08/16 22:11:50
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/45c0e965/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/45c0e965/circle_area.out *************************
______________________________ circle_area (ID: 9b255d3f-09f6-4b31-abbd-0f2cbb523d34) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.043963
starttime: 2021/08/16 22:11:50
endtime: 2021/08/16 22:11:50
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/9b255d3f/circle_area.out *************************
If you want to see content of error file use the -e or --error flag. It would be useful to inspect
content of build script and generated test, which can be retrieved using --testpath and --buildscript. Let’s
see query the first record of circle_area and report all of the content fields
$ buildtest inspect query -d first -o -e -t -b circle_area
______________________________ circle_area (ID: 0b6ab2ac-b532-4e7f-9a89-56cbc41ee998) ______________________________
executor: generic.local.python
description: Calculate circle of area given a radius
state: PASS
returncode: 0
runtime: 0.04527
starttime: 2021/08/16 22:11:46
endtime: 2021/08/16 22:11:46
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.out *************************
Circle Radius 2
Area of circle 12.566370614359172
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.out *************************
************************* Start of Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.err *************************
circle_area_build.sh: 14: circle_area_build.sh: source: not found
************************* End of Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.err *************************
************************* Start of Test Path: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.sh *************************
#!/bin/bash
python /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/stage/circle_area.py
************************* End of Test Path: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area.sh *************************
************************* Start of Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area_build.sh *************************
#!/bin/bash
############# START VARIABLE DECLARATION ########################
export BUILDTEST_TEST_NAME=circle_area
export BUILDTEST_TEST_ROOT=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac
export BUILDTEST_BUILDSPEC_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/tutorials
export BUILDTEST_STAGE_DIR=/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/stage
export BUILDTEST_TEST_ID=0b6ab2ac-b532-4e7f-9a89-56cbc41ee998
############# END VARIABLE DECLARATION ########################
# source executor startup script
source /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/executor/generic.local.python/before_script.sh
# Run generated script
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/stage/circle_area.sh
# Get return code
returncode=$?
# Exit with return code
exit $returncode
************************* End of Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.python/python-shell/circle_area/0b6ab2ac/circle_area_build.sh *************************
We can query multiple tests using buildtest inspect query since each test is a positional argument. Any
options specified to buildtest inspect query will be applied to all test. For instance, let’s fetch the output the
of test names root_disk_usage and python_hello
$ buildtest inspect query -o root_disk_usage python_hello
______________________________ root_disk_usage (ID: 0b5ef78e-e4ea-443f-8174-4069d5182889) ______________________________
executor: generic.local.bash
description: Check root disk usage and report if it exceeds threshold
state: PASS
returncode: 0
runtime: 0.011221
starttime: 2021/08/16 22:11:45
endtime: 2021/08/16 22:11:45
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/disk_usage/root_disk_usage/0b5ef78e/root_disk_usage.out *************************
[OK] Root disk is below threshold of 90%
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/disk_usage/root_disk_usage/0b5ef78e/root_disk_usage.out *************************
______________________________ python_hello (ID: 3af45be7-9173-4bd6-b555-f7d8f8380323) ______________________________
executor: generic.local.bash
description: Hello World python
state: PASS
returncode: 0
runtime: 0.043397
starttime: 2021/08/16 22:11:46
endtime: 2021/08/16 22:11:46
************************* Start of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello.out *************************
Hello World
************************* End of Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v0.10.2/var/tests/generic.local.bash/python-hello/python_hello/3af45be7/python_hello.out *************************
Using Alternate Report File¶
The buildtest report and buildtest inspect command will read from the report file tracked by buildtest which is
stored in $BUILDTEST_ROOT/var/report.json. This single file can became an issue if you are running jobs through CI where you
can potentially overwrite same file or if you want separate report files for each set of builds. Luckily we have an option to handle
this using the buildtest build -r /path/to/report option which can be used to specify an alternate location to report file.
buildtest will write the report file in the desired location, then you can specify the path to report file via
buildtest report -r /path/to/report and buildtest inspect -r /path/to/report to load the report file when reporting tests.
The report file must be valid JSON file that buildtest understands in order to use buildtest report and
buildtest inspect command. Shown below are some examples using the alternate report file using buildtest report and
buildtest inspect command.
$ buildtest report -r python.json --format name,id
Reading report file: /Users/siddiq90/Documents/GitHubDesktop/buildtest/docs/python.json
+--------------+----------+
| name | id |
+==============+==========+
| circle_area | 6be6c404 |
+--------------+----------+
| python_hello | f21ba744 |
+--------------+----------+
$ buildtest inspect -r test.json name variables_bash
Reading Report File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/test.json
{
"variables_bash": [
{
"id": "cd0511ce",
"full_id": "cd0511ce-377e-4ed2-95f4-f244e5518732",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "DOE-7086392.local",
"user": "siddiq90",
"testroot": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1",
"testpath": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/stage/generate.sh",
"stagedir": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/stage",
"rundir": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/run",
"command": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/stage/generate.sh",
"outfile": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/run/variables_bash.out",
"errfile": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/1/run/variables_bash.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: |\n \"this is a literal string ':' \"\n singlequote: \"'singlequote'\"\n doublequote: \"\\\"doublequote\\\"\"\n current_user: \"$(whoami)\"\n files_homedir: \"`find $HOME -type f -maxdepth 1`\"\n\n run: |\n echo \"$X+$Y=\" $(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n\n echo $current_user\n echo $files_homedir",
"test_content": "#!/bin/bash \nsource /Users/siddiq90/.buildtest/executor/generic.local.bash/before_script.sh\nX=1\nY=2\nliteralstring=\"this is a literal string ':' \"\n\nsinglequote='singlequote'\ndoublequote=\"doublequote\"\ncurrent_user=$(whoami)\nfiles_homedir=`find $HOME -type f -maxdepth 1`\necho \"$X+$Y=\" $(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\n\necho $current_user\necho $files_homedir\nsource /Users/siddiq90/.buildtest/executor/generic.local.bash/after_script.sh",
"tags": "tutorials",
"starttime": "2021/04/16 14:29:25",
"endtime": "2021/04/16 14:29:25",
"runtime": 0.213196,
"state": "PASS",
"returncode": 0,
"output": "1+2= 3\nthis is a literal string ':'\nsinglequote\ndoublequote\nsiddiq90\n/Users/siddiq90/buildtest_e7yxgttm.log /Users/siddiq90/.anyconnect /Users/siddiq90/buildtest_utwigb8w.log /Users/siddiq90/.DS_Store /Users/siddiq90/.serverauth.555 /Users/siddiq90/.CFUserTextEncoding /Users/siddiq90/.wget-hsts /Users/siddiq90/.bashrc /Users/siddiq90/.zshrc /Users/siddiq90/.coverage /Users/siddiq90/.serverauth.87055 /Users/siddiq90/buildtest_r7bck5zh.log /Users/siddiq90/.zsh_history /Users/siddiq90/.lesshst /Users/siddiq90/calltracker.py /Users/siddiq90/.git-completion.bash /Users/siddiq90/buildtest_wvjaaztp.log /Users/siddiq90/buildtest.log /Users/siddiq90/darhan.log /Users/siddiq90/ascent.yml /Users/siddiq90/.cshrc /Users/siddiq90/buildtest_nyq22whj.log /Users/siddiq90/github-tokens /Users/siddiq90/buildtest_ozb8b52z.log /Users/siddiq90/.zcompdump /Users/siddiq90/buildtest_nab_ckph.log /Users/siddiq90/.serverauth.543 /Users/siddiq90/.s.PGSQL.15007.lock /Users/siddiq90/.bash_profile /Users/siddiq90/.Xauthority /Users/siddiq90/.python_history /Users/siddiq90/.gitconfig /Users/siddiq90/output.txt /Users/siddiq90/.bash_history /Users/siddiq90/.viminfo\n",
"error": "",
"job": null
},
{
"id": "e0901505",
"full_id": "e0901505-a66b-4c91-9b29-d027cb6fabb6",
"schemafile": "script-v1.0.schema.json",
"executor": "generic.local.bash",
"compiler": null,
"hostname": "DOE-7086392.local",
"user": "siddiq90",
"testroot": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2",
"testpath": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/stage/generate.sh",
"stagedir": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/stage",
"rundir": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/run",
"command": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/stage/generate.sh",
"outfile": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/run/variables_bash.out",
"errfile": "/Users/siddiq90/.buildtest/var/tests/generic.local.bash/vars/variables_bash/2/run/variables_bash.err",
"buildspec_content": "version: \"1.0\"\nbuildspecs:\n variables_bash:\n type: script\n executor: generic.local.bash\n description: Declare shell variables in bash\n tags: [tutorials]\n vars:\n X: 1\n Y: 2\n literalstring: |\n \"this is a literal string ':' \"\n singlequote: \"'singlequote'\"\n doublequote: \"\\\"doublequote\\\"\"\n current_user: \"$(whoami)\"\n files_homedir: \"`find $HOME -type f -maxdepth 1`\"\n\n run: |\n echo \"$X+$Y=\" $(($X+$Y))\n echo $literalstring\n echo $singlequote\n echo $doublequote\n\n echo $current_user\n echo $files_homedir",
"test_content": "#!/bin/bash \nsource /Users/siddiq90/.buildtest/executor/generic.local.bash/before_script.sh\nX=1\nY=2\nliteralstring=\"this is a literal string ':' \"\n\nsinglequote='singlequote'\ndoublequote=\"doublequote\"\ncurrent_user=$(whoami)\nfiles_homedir=`find $HOME -type f -maxdepth 1`\necho \"$X+$Y=\" $(($X+$Y))\necho $literalstring\necho $singlequote\necho $doublequote\n\necho $current_user\necho $files_homedir\nsource /Users/siddiq90/.buildtest/executor/generic.local.bash/after_script.sh",
"tags": "tutorials",
"starttime": "2021/04/16 14:29:58",
"endtime": "2021/04/16 14:29:58",
"runtime": 0.075224,
"state": "PASS",
"returncode": 0,
"output": "1+2= 3\nthis is a literal string ':'\nsinglequote\ndoublequote\nsiddiq90\n/Users/siddiq90/buildtest_e7yxgttm.log /Users/siddiq90/.anyconnect /Users/siddiq90/buildtest_utwigb8w.log /Users/siddiq90/.DS_Store /Users/siddiq90/.serverauth.555 /Users/siddiq90/.CFUserTextEncoding /Users/siddiq90/.wget-hsts /Users/siddiq90/.bashrc /Users/siddiq90/.zshrc /Users/siddiq90/.coverage /Users/siddiq90/.serverauth.87055 /Users/siddiq90/buildtest_r7bck5zh.log /Users/siddiq90/.zsh_history /Users/siddiq90/.lesshst /Users/siddiq90/calltracker.py /Users/siddiq90/.git-completion.bash /Users/siddiq90/buildtest_wvjaaztp.log /Users/siddiq90/buildtest.log /Users/siddiq90/darhan.log /Users/siddiq90/ascent.yml /Users/siddiq90/.cshrc /Users/siddiq90/buildtest_nyq22whj.log /Users/siddiq90/github-tokens /Users/siddiq90/buildtest_ozb8b52z.log /Users/siddiq90/.zcompdump /Users/siddiq90/buildtest_nab_ckph.log /Users/siddiq90/.serverauth.543 /Users/siddiq90/.s.PGSQL.15007.lock /Users/siddiq90/.bash_profile /Users/siddiq90/.Xauthority /Users/siddiq90/.python_history /Users/siddiq90/.gitconfig /Users/siddiq90/output.txt /Users/siddiq90/.bash_history /Users/siddiq90/.viminfo\n",
"error": "",
"job": null
}
]
}