Additional Features
Accessing build history (buildtest history)
Note
buildtest hy is an alias for buildtest history command.
buildtest keeps track of all builds (buildtest build) that can be retrieved using buildtest history command
which can be useful when you want to analyze or troubleshoot past builds. The buildtest history command comes with two
subcommands buildtest history list and buildtest history query.
If you want to list all builds you should run buildtest history list which will report a table style format of all builds with corresponding build ID to differentiate each build. Shown below is an example output. The build IDs start at 0 and increment as you run buildtest build command.
buildtest history list
$ buildtest history list
┏━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ ┃ hostnam ┃ ┃ ┃ ┃ pass ┃ fail ┃ total ┃ comman ┃
┃ id ┃ e ┃ user ┃ system ┃ date ┃ tests ┃ tests ┃ tests ┃ d ┃
┡━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ 0 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 0 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 08 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ /home/ │
│ │ │ │ │ │ │ │ │ docs/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/rea │
│ │ │ │ │ │ │ │ │ dthedo │
│ │ │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/tu │
│ │ │ │ │ │ │ │ │ torial │
│ │ │ │ │ │ │ │ │ s/vars │
│ │ │ │ │ │ │ │ │ .yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 1 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 2 │ 4 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 10 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/pa │
│ │ │ │ │ │ │ │ │ ss_ret │
│ │ │ │ │ │ │ │ │ urncod │
│ │ │ │ │ │ │ │ │ e.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 2 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 2 │ 4 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 11 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/st │
│ │ │ │ │ │ │ │ │ atus_r │
│ │ │ │ │ │ │ │ │ egex.y │
│ │ │ │ │ │ │ │ │ ml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 3 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 1 │ 3 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 13 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/re │
│ │ │ │ │ │ │ │ │ gex_on │
│ │ │ │ │ │ │ │ │ _filen │
│ │ │ │ │ │ │ │ │ ame.ym │
│ │ │ │ │ │ │ │ │ l │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 4 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 3 │ 5 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 14 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/ru │
│ │ │ │ │ │ │ │ │ ntime_ │
│ │ │ │ │ │ │ │ │ status │
│ │ │ │ │ │ │ │ │ _test. │
│ │ │ │ │ │ │ │ │ yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 5 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 2 │ 4 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 16 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/ex │
│ │ │ │ │ │ │ │ │ plicit │
│ │ │ │ │ │ │ │ │ _state │
│ │ │ │ │ │ │ │ │ .yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 6 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 1 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 17 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/ex │
│ │ │ │ │ │ │ │ │ ists.y │
│ │ │ │ │ │ │ │ │ ml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 7 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 0 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 20 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_exi │
│ │ │ │ │ │ │ │ │ sts_wi │
│ │ │ │ │ │ │ │ │ th_num │
│ │ │ │ │ │ │ │ │ ber.ym │
│ │ │ │ │ │ │ │ │ l │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 8 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 1 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 21 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_and │
│ │ │ │ │ │ │ │ │ _dir_c │
│ │ │ │ │ │ │ │ │ heck.y │
│ │ │ │ │ │ │ │ │ ml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 9 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 0 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 22 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/is │
│ │ │ │ │ │ │ │ │ _symli │
│ │ │ │ │ │ │ │ │ nk.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 10 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 0 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 23 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_cou │
│ │ │ │ │ │ │ │ │ nt.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 11 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 0 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 24 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_cou │
│ │ │ │ │ │ │ │ │ nt_pat │
│ │ │ │ │ │ │ │ │ tern.y │
│ │ │ │ │ │ │ │ │ ml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 12 │ build-2 │ docs │ generic │ 2023/1 │ 0 │ 1 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 25 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_cou │
│ │ │ │ │ │ │ │ │ nt_fil │
│ │ │ │ │ │ │ │ │ etype. │
│ │ │ │ │ │ │ │ │ yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 13 │ build-2 │ docs │ generic │ 2023/1 │ 0 │ 1 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 26 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/fi │
│ │ │ │ │ │ │ │ │ le_cou │
│ │ │ │ │ │ │ │ │ nt_fil │
│ │ │ │ │ │ │ │ │ e_trav │
│ │ │ │ │ │ │ │ │ erse_l │
│ │ │ │ │ │ │ │ │ imit.y │
│ │ │ │ │ │ │ │ │ ml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 14 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 0 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 27 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/te │
│ │ │ │ │ │ │ │ │ st_sta │
│ │ │ │ │ │ │ │ │ tus/mo │
│ │ │ │ │ │ │ │ │ de.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 15 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 0 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 28 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ tutori │
│ │ │ │ │ │ │ │ │ als/sk │
│ │ │ │ │ │ │ │ │ ip_tes │
│ │ │ │ │ │ │ │ │ ts.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 16 │ build-2 │ docs │ generic │ 2023/1 │ 1 │ 0 │ 1 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 32 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ /home/ │
│ │ │ │ │ │ │ │ │ docs/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/rea │
│ │ │ │ │ │ │ │ │ dthedo │
│ │ │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/tu │
│ │ │ │ │ │ │ │ │ torial │
│ │ │ │ │ │ │ │ │ s/hell │
│ │ │ │ │ │ │ │ │ o_worl │
│ │ │ │ │ │ │ │ │ d.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 17 │ build-2 │ docs │ generic │ 2023/1 │ 5 │ 5 │ 10 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 34 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ /home/ │
│ │ │ │ │ │ │ │ │ docs/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/rea │
│ │ │ │ │ │ │ │ │ dthedo │
│ │ │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/tu │
│ │ │ │ │ │ │ │ │ torial │
│ │ │ │ │ │ │ │ │ s/hell │
│ │ │ │ │ │ │ │ │ o_worl │
│ │ │ │ │ │ │ │ │ d.yml │
│ │ │ │ │ │ │ │ │ -b │
│ │ │ │ │ │ │ │ │ /home/ │
│ │ │ │ │ │ │ │ │ docs/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/rea │
│ │ │ │ │ │ │ │ │ dthedo │
│ │ │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/ge │
│ │ │ │ │ │ │ │ │ neral_ │
│ │ │ │ │ │ │ │ │ tests/ │
│ │ │ │ │ │ │ │ │ config │
│ │ │ │ │ │ │ │ │ uratio │
│ │ │ │ │ │ │ │ │ n │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 18 │ build-2 │ docs │ generic │ 2023/1 │ 2 │ 0 │ 2 │ /home/ │
│ │ 2529914 │ │ │ 1/13 │ │ │ │ docs/c │
│ │ -projec │ │ │ 01:39: │ │ │ │ heckou │
│ │ t-28083 │ │ │ 38 │ │ │ │ ts/rea │
│ │ 1-build │ │ │ │ │ │ │ dthedo │
│ │ test │ │ │ │ │ │ │ cs.org │
│ │ │ │ │ │ │ │ │ /user_ │
│ │ │ │ │ │ │ │ │ builds │
│ │ │ │ │ │ │ │ │ /build │
│ │ │ │ │ │ │ │ │ test/c │
│ │ │ │ │ │ │ │ │ heckou │
│ │ │ │ │ │ │ │ │ ts/sta │
│ │ │ │ │ │ │ │ │ ble/bi │
│ │ │ │ │ │ │ │ │ n/buil │
│ │ │ │ │ │ │ │ │ dtest │
│ │ │ │ │ │ │ │ │ build │
│ │ │ │ │ │ │ │ │ -t │
│ │ │ │ │ │ │ │ │ networ │
│ │ │ │ │ │ │ │ │ k │
└────┴─────────┴──────┴─────────┴────────┴─────────┴────────┴─────────┴────────┘
The buildtest history query command is particularly useful when you want to inspect a particular build. This command
expects a Build Identifier which can be found by inspecting output column id in buildtest history list.
Shown below is an output of build ID 0 which reports relevant detail for the build such as input command, username, hostname, platform, date, etc…
buildtest history query 0
$ buildtest history query 0
{
│ 'command': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest build -b /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml',
│ 'user': 'docs',
│ 'hostname': 'build-22529914-project-280831-buildtest',
│ 'platform': 'Linux',
│ 'date': '2023/11/13 01:39:08',
│ 'buildtest': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest',
│ 'python': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3',
│ 'python_version': '3.8.18',
│ 'testdir': '/tmp/tmpxadaezwi/var/tests',
│ 'configuration': '/tmp/tmpxadaezwi/config.yml',
│ 'system': 'generic',
│ 'logpath': '/tmp/tmpxadaezwi/var/.history/0/buildtest_rz_06odv.log',
│ 'invalid_buildspecs': [],
│ 'buildspecs': {
│ │ 'detected': [
│ │ │ '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml'
│ │ ],
│ │ 'included': [
│ │ │ '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml'
│ │ ],
│ │ 'excluded': []
│ },
│ 'test_summary': {
│ │ 'pass': '1',
│ │ 'fail': '0',
│ │ 'total': '1',
│ │ 'pass_rate': '100.000',
│ │ 'fail_rate': '0.000'
│ },
│ 'builders': {
│ │ 'b98070fe-6020-4103-ac60-a6197193ed6d': {
│ │ │ 'name': 'variables_bash',
│ │ │ 'buildspec': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml',
│ │ │ 'tags': ['tutorials'],
│ │ │ 'executors': 'generic.local.bash',
│ │ │ 'state': 'PASS',
│ │ │ 'returncode': 0,
│ │ │ 'runtime': 0.015961,
│ │ │ 'testpath': '/tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.sh',
│ │ │ 'errfile': '/tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.err',
│ │ │ 'outfile': '/tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.out'
│ │ }
│ }
}
If you want to see all available build IDs, you can use the following command. The -t is terse format and --no-header will
omit the headers for each column and pipe the output to cut to extract the first column which corresponds to build IDs.
$ buildtest hy list --terse --no-header | cut -f 1 -d '|'
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
buildtest has tab complete on buildtest history query which reports a list of build IDs which is another way to
see available IDs to query.
$ buildtest history query
0 1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 23 3 4 5 6 7 8 9
If you want to see logfile for build ID 0 you can use --log option to see logfile in an editor as follows:
buildtest history query 0 --log
buildtest will store output of buildtest build command to file if command ran to completion. You can retrieve output of previous
builds using the --output option. In example below we show output of build file
buildtest history query --output 0
$ buildtest history query --output 0
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-22529914-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2023/11/13 01:39:08 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 1.7 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.8.18 │
│ Configuration File: /tmp/tmpxadaezwi/config.yml │
│ Test Directory: /tmp/tmpxadaezwi/var/tests │
│ Report File: /tmp/tmpxadaezwi/var/report.json │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/vars.yml ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ variable │ script │ generic. │ None │ None │ None │ Declare │ /home/d │
│ s_bash/b │ │ local.ba │ │ │ │ shell │ ocs/che │
│ 98070fe │ │ sh │ │ │ │ variable │ ckouts/ │
│ │ │ │ │ │ │ s in │ readthe │
│ │ │ │ │ │ │ bash │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/s │
│ │ │ │ │ │ │ │ table/t │
│ │ │ │ │ │ │ │ utorial │
│ │ │ │ │ │ │ │ s/vars. │
│ │ │ │ │ │ │ │ yml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
variables_bash/b98070fe: Creating test directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe
variables_bash/b98070fe: Creating the stage directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/stage
variables_bash/b98070fe: Writing build script: /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
variables_bash/b98070fe does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ variables_bash/b98070fe │
└─────────────────────────┘
variables_bash/b98070fe: Current Working Directory : /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/stage
variables_bash/b98070fe: Running Test via command: bash --norc --noprofile -eo pipefail variables_bash_build.sh
variables_bash/b98070fe: Test completed in 0.015961 seconds
variables_bash/b98070fe: Test completed with returncode: 0
variables_bash/b98070fe: Writing output file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.out
variables_bash/b98070fe: Writing error file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.err
Test Summary
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ ┃ ┃ ┃ checks ┃ ┃ ┃
┃ ┃ ┃ ┃ (ReturnCode, ┃ ┃ ┃
┃ ┃ ┃ ┃ Regex, ┃ ┃ ┃
┃ builder ┃ executor ┃ status ┃ Runtime) ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ variables_ba │ generic.loca │ PASS │ None None │ 0 │ 0.015961 │
│ sh/b98070fe │ l.bash │ │ None │ │ │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to /tmp/tmpxadaezwi/var/report.json
Writing Logfile to: /tmp/tmpxadaezwi/var/logs/buildtest_rz_06odv.log
Buildtest Info (buildtest info)
The buildtest info command will provide user with general information pertaining to buildtest and its configuration, system details
such as python wrapper, python version, machine name, operating system. It will also show the version of black, isort and pyflakes which are used when using
buildtest stylecheck command
buildtest info
$ buildtest info
╭───────────────────────────── buildtest details ──────────────────────────────╮
│ Buildtest Version: 1.7 │
│ Buildtest Path: /home/docs/checkouts/readthedocs.org/user_builds/b │
│ Configuration File: /tmp/tmpxadaezwi/config.yml │
│ Available Systems: ['generic'] │
│ Active System: generic │
│ Available Executors: ['generic.local.bash', 'generic.local.sh', 'generi │
│ Build History Directory: /tmp/tmpxadaezwi/var/.history │
│ Number of builds: 19 │
│ Buildspec Cache File: /tmp/tmpxadaezwi/var/buildspecs/cache.json │
│ Default Report File: /tmp/tmpxadaezwi/var/report.json │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────── system details ───────────────────────────────╮
│ Python Path: /home/docs/checkouts/readthedocs.org/user_builds/buildt │
│ Python Version: 3.8.18 │
│ Processor: x86_64 │
│ Host: build-22529914-project-280831-buildtest │
│ Machine: x86_64 │
│ Operating System: ubuntu │
│ Module System: None │
╰──────────────────────────────────────────────────────────────────────────────╯
black: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/black
black version: black, 23.11.0 (compiled: yes)
Python (CPython) 3.8.18
pyflakes: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/pyflakes
pyflakes version: 3.1.0 Python 3.8.18 on Linux
isort: None
isort version:
Accessing buildtest documentation
We provide two command line options to access main documentation and schema docs. This will open a browser on your machine.
To access buildtest docs you can run:
buildtest docs
To access schema docs you can run:
buildtest schemadocs
Enabling colored output for table entries
The buildtest --color <COLOR> <COMMAND> command can be used to select a color while printing output in a
tabular format for several buildtest commands. The list of available colors can be found by running buildtest --helpcolor.
Buildtest has tab-completion setup for --color option which will list all available colors.
Listing available color options (buildtest --helpcolor)
The buildtest --helpcolor command can be used to list the available color options in a tabular format which can be used with the –color option to select a color when printing table entries from several buildtest commands. This option will list all the colors printed in the background for the specified color
buildtest --helpcolor
$ buildtest --helpcolor
Available Colors
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Number ┃ Color ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ 0 │ black │
│ 1 │ red │
│ 2 │ green │
│ 3 │ yellow │
│ 4 │ blue │
│ 5 │ magenta │
│ 6 │ cyan │
│ 7 │ white │
│ 8 │ bright_black │
│ 9 │ bright_red │
│ 10 │ bright_green │
│ 11 │ bright_yellow │
│ 12 │ bright_blue │
│ 13 │ bright_magenta │
│ 14 │ bright_cyan │
│ 15 │ bright_white │
│ 16 │ grey0 │
│ 16 │ gray0 │
│ 17 │ navy_blue │
│ 18 │ dark_blue │
│ 20 │ blue3 │
│ 21 │ blue1 │
│ 22 │ dark_green │
│ 25 │ deep_sky_blue4 │
│ 26 │ dodger_blue3 │
│ 27 │ dodger_blue2 │
│ 28 │ green4 │
│ 29 │ spring_green4 │
│ 30 │ turquoise4 │
│ 32 │ deep_sky_blue3 │
│ 33 │ dodger_blue1 │
│ 40 │ green3 │
│ 41 │ spring_green3 │
│ 36 │ dark_cyan │
│ 37 │ light_sea_green │
│ 38 │ deep_sky_blue2 │
│ 39 │ deep_sky_blue1 │
│ 47 │ spring_green2 │
│ 43 │ cyan3 │
│ 44 │ dark_turquoise │
│ 45 │ turquoise2 │
│ 46 │ green1 │
│ 48 │ spring_green1 │
│ 49 │ medium_spring_green │
│ 50 │ cyan2 │
│ 51 │ cyan1 │
│ 88 │ dark_red │
│ 125 │ deep_pink4 │
│ 55 │ purple4 │
│ 56 │ purple3 │
│ 57 │ blue_violet │
│ 94 │ orange4 │
│ 59 │ grey37 │
│ 59 │ gray37 │
│ 60 │ medium_purple4 │
│ 62 │ slate_blue3 │
│ 63 │ royal_blue1 │
│ 64 │ chartreuse4 │
│ 71 │ dark_sea_green4 │
│ 66 │ pale_turquoise4 │
│ 67 │ steel_blue │
│ 68 │ steel_blue3 │
│ 69 │ cornflower_blue │
│ 76 │ chartreuse3 │
│ 73 │ cadet_blue │
│ 74 │ sky_blue3 │
│ 81 │ steel_blue1 │
│ 114 │ pale_green3 │
│ 78 │ sea_green3 │
│ 79 │ aquamarine3 │
│ 80 │ medium_turquoise │
│ 112 │ chartreuse2 │
│ 83 │ sea_green2 │
│ 85 │ sea_green1 │
│ 122 │ aquamarine1 │
│ 87 │ dark_slate_gray2 │
│ 91 │ dark_magenta │
│ 128 │ dark_violet │
│ 129 │ purple │
│ 95 │ light_pink4 │
│ 96 │ plum4 │
│ 98 │ medium_purple3 │
│ 99 │ slate_blue1 │
│ 106 │ yellow4 │
│ 101 │ wheat4 │
│ 102 │ grey53 │
│ 102 │ gray53 │
│ 103 │ light_slate_grey │
│ 103 │ light_slate_gray │
│ 104 │ medium_purple │
│ 105 │ light_slate_blue │
│ 149 │ dark_olive_green3 │
│ 108 │ dark_sea_green │
│ 110 │ light_sky_blue3 │
│ 111 │ sky_blue2 │
│ 150 │ dark_sea_green3 │
│ 116 │ dark_slate_gray3 │
│ 117 │ sky_blue1 │
│ 118 │ chartreuse1 │
│ 120 │ light_green │
│ 156 │ pale_green1 │
│ 123 │ dark_slate_gray1 │
│ 160 │ red3 │
│ 126 │ medium_violet_red │
│ 164 │ magenta3 │
│ 166 │ dark_orange3 │
│ 167 │ indian_red │
│ 168 │ hot_pink3 │
│ 133 │ medium_orchid3 │
│ 134 │ medium_orchid │
│ 140 │ medium_purple2 │
│ 136 │ dark_goldenrod │
│ 173 │ light_salmon3 │
│ 138 │ rosy_brown │
│ 139 │ grey63 │
│ 139 │ gray63 │
│ 141 │ medium_purple1 │
│ 178 │ gold3 │
│ 143 │ dark_khaki │
│ 144 │ navajo_white3 │
│ 145 │ grey69 │
│ 145 │ gray69 │
│ 146 │ light_steel_blue3 │
│ 147 │ light_steel_blue │
│ 184 │ yellow3 │
│ 157 │ dark_sea_green2 │
│ 152 │ light_cyan3 │
│ 153 │ light_sky_blue1 │
│ 154 │ green_yellow │
│ 155 │ dark_olive_green2 │
│ 193 │ dark_sea_green1 │
│ 159 │ pale_turquoise1 │
│ 162 │ deep_pink3 │
│ 200 │ magenta2 │
│ 169 │ hot_pink2 │
│ 170 │ orchid │
│ 207 │ medium_orchid1 │
│ 172 │ orange3 │
│ 174 │ light_pink3 │
│ 175 │ pink3 │
│ 176 │ plum3 │
│ 177 │ violet │
│ 179 │ light_goldenrod3 │
│ 180 │ tan │
│ 181 │ misty_rose3 │
│ 182 │ thistle3 │
│ 183 │ plum2 │
│ 185 │ khaki3 │
│ 222 │ light_goldenrod2 │
│ 187 │ light_yellow3 │
│ 188 │ grey84 │
│ 188 │ gray84 │
│ 189 │ light_steel_blue1 │
│ 190 │ yellow2 │
│ 192 │ dark_olive_green1 │
│ 194 │ honeydew2 │
│ 195 │ light_cyan1 │
│ 196 │ red1 │
│ 197 │ deep_pink2 │
│ 199 │ deep_pink1 │
│ 201 │ magenta1 │
│ 202 │ orange_red1 │
│ 204 │ indian_red1 │
│ 206 │ hot_pink │
│ 208 │ dark_orange │
│ 209 │ salmon1 │
│ 210 │ light_coral │
│ 211 │ pale_violet_red1 │
│ 212 │ orchid2 │
│ 213 │ orchid1 │
│ 214 │ orange1 │
│ 215 │ sandy_brown │
│ 216 │ light_salmon1 │
│ 217 │ light_pink1 │
│ 218 │ pink1 │
│ 219 │ plum1 │
│ 220 │ gold1 │
│ 223 │ navajo_white1 │
│ 224 │ misty_rose1 │
│ 225 │ thistle1 │
│ 226 │ yellow1 │
│ 227 │ light_goldenrod1 │
│ 228 │ khaki1 │
│ 229 │ wheat1 │
│ 230 │ cornsilk1 │
│ 231 │ grey100 │
│ 231 │ gray100 │
│ 232 │ grey3 │
│ 232 │ gray3 │
│ 233 │ grey7 │
│ 233 │ gray7 │
│ 234 │ grey11 │
│ 234 │ gray11 │
│ 235 │ grey15 │
│ 235 │ gray15 │
│ 236 │ grey19 │
│ 236 │ gray19 │
│ 237 │ grey23 │
│ 237 │ gray23 │
│ 238 │ grey27 │
│ 238 │ gray27 │
│ 239 │ grey30 │
│ 239 │ gray30 │
│ 240 │ grey35 │
│ 240 │ gray35 │
│ 241 │ grey39 │
│ 241 │ gray39 │
│ 242 │ grey42 │
│ 242 │ gray42 │
│ 243 │ grey46 │
│ 243 │ gray46 │
│ 244 │ grey50 │
│ 244 │ gray50 │
│ 245 │ grey54 │
│ 245 │ gray54 │
│ 246 │ grey58 │
│ 246 │ gray58 │
│ 247 │ grey62 │
│ 247 │ gray62 │
│ 248 │ grey66 │
│ 248 │ gray66 │
│ 249 │ grey70 │
│ 249 │ gray70 │
│ 250 │ grey74 │
│ 250 │ gray74 │
│ 251 │ grey78 │
│ 251 │ gray78 │
│ 252 │ grey82 │
│ 252 │ gray82 │
│ 253 │ grey85 │
│ 253 │ gray85 │
│ 254 │ grey89 │
│ 254 │ gray89 │
│ 255 │ grey93 │
│ 255 │ gray93 │
└────────┴─────────────────────┘
Disabling Colored Output
buildtest will display output in color format using the rich.Console
class. You can disable colored output via buildtest --no-color argument or set this
persistent via environment variable BUILDTEST_COLOR=False.
CDASH Integration (buildtest cdash)
The buildtest cdash command is responsible for uploading tests to CDASH server. You will
need to specify CDASH Configuration in your configuration file. Shown below is the command
usage.
buildtest cdash --help
$ buildtest cdash --help
usage: buildtest [options] [COMMANDS] cdash [-h] ...
optional arguments:
-h, --help show this help message and exit
subcommands:
buildtest CDASH integration
view Open CDASH project in web browser
upload Upload test results to CDASH server
The buildtest cdash upload command is responsible for uploading all tests in report.json
into CDASH. You must specify a buildname when using buildtest cdash upload in this example we will
specify a buildname called tutorials:
$ buildtest cdash upload tutorials
Reading report file: /Users/siddiq90/Documents/github/buildtest/var/report.json
Uploading 110 tests
Build Name: tutorials
site: generic
MD5SUM: d7651cb3fbdd19298b0188c441704c3a
You can view the results at: https://my.cdash.org//viewTest.php?buildid=2004360
We can see the output of these tests in CDASH if we go to url https://my.cdash.org//viewTest.php?buildid=2004360
By default buildtest will read the report file in your $HOME/.buildtest/report.json, we can
specify an alternate report file. First let’s see the available help options for
buildtest cdash upload.
buildtest cdash upload --help
$ buildtest cdash upload --help
usage: buildtest [options] [COMMANDS] cdash upload [-h] [--site SITE] [-o]
buildname
positional arguments:
buildname Specify Build Name reported in CDASH
optional arguments:
-h, --help show this help message and exit
--site SITE Specify site name reported in CDASH
-o, --open Open CDASH report in browser
We can pass an alternate report file using -r option when uploading tests
to CDASH. This can be useful if you want to map test results to different buildnames in CDASH
perhaps running a different subset of tests via buildtest build --tags and upload
the test results with different buildname assuming you have different paths to report file.
Let’s say we want to build all python tests using tags and store them in a report file which we
want to push to CDASH with buildgroup name python we can do that as follows
buildtest -r $BUILDTEST_ROOT/python.json build --tags python
$ buildtest -r $BUILDTEST_ROOT/python.json build --tags python
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-22529914-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2023/11/13 01:40:14 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 1.7 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.8.18 │
│ Configuration File: /tmp/tmpxadaezwi/config.yml │
│ Test Directory: /tmp/tmpxadaezwi/var/tests │
│ Report File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-shell.yml ║
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-hello.yml ║
╚══════════════════════════════════════════════════════════════════════════════╝
Buildspecs By Tag=python
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-shell.yml ║
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-hello.yml ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 2
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 2
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 2
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-shell.yml: VALID
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-hello.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ circle_a │ script │ generic. │ None │ None │ None │ Calculat │ /home/d │
│ rea/4c91 │ │ local.ba │ │ │ │ e circle │ ocs/che │
│ c94a │ │ sh │ │ │ │ of area │ ckouts/ │
│ │ │ │ │ │ │ given a │ readthe │
│ │ │ │ │ │ │ radius │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/s │
│ │ │ │ │ │ │ │ table/t │
│ │ │ │ │ │ │ │ utorial │
│ │ │ │ │ │ │ │ s/pytho │
│ │ │ │ │ │ │ │ n-shell │
│ │ │ │ │ │ │ │ .yml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ python_h │ script │ generic. │ None │ None │ None │ Hello │ /home/d │
│ ello/054 │ │ local.ba │ │ │ │ World │ ocs/che │
│ 17fac │ │ sh │ │ │ │ python │ ckouts/ │
│ │ │ │ │ │ │ │ readthe │
│ │ │ │ │ │ │ │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/s │
│ │ │ │ │ │ │ │ table/t │
│ │ │ │ │ │ │ │ utorial │
│ │ │ │ │ │ │ │ s/pytho │
│ │ │ │ │ │ │ │ n-hello │
│ │ │ │ │ │ │ │ .yml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
circle_area/4c91c94a: Creating test directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a
circle_area/4c91c94a: Creating the stage directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a/stage
circle_area/4c91c94a: Writing build script: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a/circle_area_build.sh
python_hello/05417fac: Creating test directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac
python_hello/05417fac: Creating the stage directory: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac/stage
python_hello/05417fac: Writing build script: /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac/python_hello_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
python_hello/05417fac does not have any dependencies adding test to queue
circle_area/4c91c94a does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━┩
│ python_hello/05417fac │
│ circle_area/4c91c94a │
└───────────────────────┘
python_hello/05417fac: Current Working Directory : /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac/stage
python_hello/05417fac: Running Test via command: bash --norc --noprofile -eo pipefail python_hello_build.sh
python_hello/05417fac: Test completed in 0.047914 seconds
python_hello/05417fac: Test completed with returncode: 0
python_hello/05417fac: Writing output file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac/python_hello.out
python_hello/05417fac: Writing error file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-hello/python_hello/05417fac/python_hello.err
circle_area/4c91c94a: Current Working Directory : /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a/stage
circle_area/4c91c94a: Running Test via command: bash --norc --noprofile -eo pipefail circle_area_build.sh
circle_area/4c91c94a: Test completed in 0.04854 seconds
circle_area/4c91c94a: Test completed with returncode: 0
circle_area/4c91c94a: Writing output file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a/circle_area.out
circle_area/4c91c94a: Writing error file - /tmp/tmpxadaezwi/var/tests/generic.local.bash/python-shell/circle_area/4c91c94a/circle_area.err
Test Summary
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ ┃ ┃ ┃ checks ┃ ┃ ┃
┃ ┃ ┃ ┃ (ReturnCode, ┃ ┃ ┃
┃ ┃ ┃ ┃ Regex, ┃ ┃ ┃
┃ builder ┃ executor ┃ status ┃ Runtime) ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ circle_area/ │ generic.loca │ PASS │ None None │ 0 │ 0.04854 │
│ 4c91c94a │ l.bash │ │ None │ │ │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ python_hello │ generic.loca │ PASS │ None None │ 0 │ 0.047914 │
│ /05417fac │ l.bash │ │ None │ │ │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/python.json
Writing Logfile to: /tmp/tmpxadaezwi/var/logs/buildtest_d8_7rdzw.log
Next we upload the tests using the -r option to specify the report file
$ buildtest -r $BUILDTEST_ROOT/python.json cdash upload python
Reading report file: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/python.json
Uploading 2 tests
Build Name: python
site: generic
MD5SUM: 28032172081412079ceaeeb7f1d8b41e
You can view the results at: https://my.cdash.org/viewTest.php?buildid=2439253
The buildtest cdash view command can be used to open CDASH project in a web browser
using the command line. This feature assumes you have set the CDASH setting in your
configuration file.
Cleaning buildtest files (buildtest clean)
The buildtest clean command can be used to remove files generated by buildtest such
as test files, report files, buildspec cache, and history files. You will be prompted for
response to clean up files for confirmation. If you want to avoid user response you can use buildtest clean -y
to accept confirmation for all prompts and buildtest will remove the files.
$ buildtest clean
Remove Test Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests (y/n) [default: y]
Remove Report File /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json (y/n) [default: y]
Remove History Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/.history (y/n) [default: y]
Remove Buildspec Cache /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/buildspecs/cache.json (y/n) [default: y]
======> Remove Test Directory
======> Removing Report File
======> Removing History Directory
======> Removing buildspec cache
Changing Directories (buildtest cd)
The buildtest cd command can be used to change directory to root of test given
a test name. The change will be applied to your shell upon completion of
command. Let’s assume we want to change directory to root of test exit1_pass we can do this as
follows:
$ buildtest cd exit1_pass
Changing directory to root of test: exit1_pass/8c4b6ac9-e94e-40d9-8d96-7aaa3a5d3723
$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9
In this previous example, buildtest will use the latest run for test exit1_pass and switch directory to root of test.
We can confirm this directory is from the latest run by running the following command. The testroot is a property
in the report table that can be fetch via --format field. The --latest option will fetch
the latest run for the test.
$ buildtest report --latest --filter name=exit1_pass --format testroot --terse --no-header
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9
If you switch cd into a particular build you can specify the name followed by backslash and name of test ID. In this example below,
we will specify test name kernel_swapusage/1fa and buildtest will attempt to find first record that starts with the test ID and switch
directory to root of test.
$ buildtest cd kernel_swapusage/1fa
Changing directory to root of test: kernel_swapusage/1fa21875-b099-41b6-8bc7-30e0d2dcc13b
$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/kernel_state/kernel_swapusage/1fa21875
Get Path for Test (buildtest path)
The buildtest path command is used to display path attributes for a test that is available in the test report.
Shown below are available options for buildtest path
buildtest path --help
$ buildtest path --help
usage: buildtest [options] [COMMANDS] path [-h] [-be | -t | -o | -e | -b | -s]
name
positional arguments:
name Name of test
optional arguments:
-h, --help show this help message and exit
-be, --buildenv Show path to build environment
-t, --testpath Show path to test script
-o, --outfile Show path to output file
-e, --errfile Show path to error file
-b, --buildscript Show path to build script
-s, --stagedir Show path to stage directory
If you want to fetch the last run for any given test you can specify the name of the test as follows: buildtest path <name>.
We can specify a test ID for a test by separating the name and test ID with backslash character (/) as follows: buildtest path <name>/<ID>
If you don’t specify any option you will get root of test. In this example, we will retrieve testroot
for test variables_bash which is a property of the test found in the report file.
$ buildtest path variables_bash
/tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe
You can get path to testscript via -t option as show below
$ buildtest path -t variables_bash
/tmp/tmpxadaezwi/var/tests/generic.local.bash/vars/variables_bash/b98070fe/variables_bash.sh
If you want to see content of output file, you can use -o option with cat command as follows:
$ cat $(buildtest path -o variables_bash)
1+2=3
this is a literal string
\'singlequote\'
"doublequote"
current user: docs
number of files: 4
Hello my name is Bob
I am 30 years old
In this next example we will query test circle_area with build ID aaa and buildtest will find the first match record that starts with this record and resolves to aaaa622d which is the short ID of test. In the second example we query the latest path for latest run for test circle_area
$ buildtest path circle_area/aaa
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/aaaa622d
$ buildtest path circle_area
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/fc221b84
We have setup mutual exclusion to avoid user from passing two option at same time. If you do run into this situation you will get the following error.
$ buildtest path -o -e variables_bash
usage: buildtest [options] [COMMANDS] path [-h] [-be | -t | -o | -e | -b | -s]
name
buildtest [options] [COMMANDS] path: error: argument -e/--errfile: not allowed with argument -o/--outfile
If you specify an invalid test name or buildtest can’t find the test id, then buildtest will print list of available test names with IDs.
Test Statistics (buildtest stats)
The buildtest stats command can be used to get statistics for a particular test. The input argument is a positional argument which is
name of test found in the report file. The output will show some useful details such as First and Last Run, show fastest and slowest runtime
including mean and variance. Shown below is the test statistics for exit_fail.
$ buildtest stats python_hello
Total Test Runs: 3
First Run: 2022/06/13 15:29:11
Last Run: 2022/06/13 15:29:21
Fastest Runtime: 0.132854
Slowest Runtime: 0.161916
Mean Runtime 0.144621
Variance Runtime 0.000234
Report File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ state ┃ returncode ┃ starttime ┃ endtime ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python_hello │ PASS │ 0 │ 2022/06/13 15:29:11 │ 2022/06/13 15:29:11 │ 0.132854 │
├──────────────────────────────────────┼───────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤
│ python_hello │ PASS │ 0 │ 2022/06/13 15:29:12 │ 2022/06/13 15:29:12 │ 0.139094 │
├──────────────────────────────────────┼───────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤
│ python_hello │ PASS │ 0 │ 2022/06/13 15:29:21 │ 2022/06/13 15:29:21 │ 0.161916 │
└──────────────────────────────────────┴───────────────────┴─────────────────────────────────┴────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────┴──────────────────────────┘
Buildtest Debug Report (buildtest debugreport)
The buildtest debugreport command is used for debugging especially when you want to raise
an issue to buildtest project. This command will provide system details
along with configuration file and output of log file during the report.
buildtest debugreport
$ buildtest debugreport
Host: build-22529914-project-280831-buildtest
OS: ubuntu
Python: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3
Python Version: 3.8.18
BUILDTEST_VERSION: 1.7
───────────────────────── /tmp/tmpxadaezwi/config.yml ──────────────────────────
1 system:
2 generic:
3 hostnames:
4 - .*
5 description: Generic System
6 moduletool: N/A
7 poolsize: 1
8 pager: false
9 buildspecs:
10 rebuild: false
11 count: 15
12 terse: false
13 report:
14 count: 25
15 terse: false
16 format: name,id,state,runtime,returncode
17 latest: false
18 oldest: false
19 executors:
20 local:
21 bash:
22 description: submit jobs on local machine using bash shell
23 shell: bash
24 sh:
25 description: submit jobs on local machine using sh shell
26 shell: sh
27 csh:
28 description: submit jobs on local machine using csh shell
29 shell: csh
30 zsh:
31 description: submit jobs on local machine using zsh shell
32 shell: zsh
33 compilers:
34 compiler:
35 gcc:
36 builtin_gcc:
37 cc: gcc
38 fc: gfortran
39 cxx: g++
40 cdash:
41 url: https://my.cdash.org/
42 project: buildtest
43 site: generic
44 buildname: tutorials
45 profiles:
46 python:
47 tags:
48 - python
49 testdir: /tmp/tmpxadaezwi/var/tests
50 remove-stagedir: false
51 prof1:
52 tags:
53 - python
54 testdir: /tmp/tmpxadaezwi/var/tests
55 remove-stagedir: false
56 prof2:
57 buildspecs:
58 - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/shell_examples.yml
59 testdir: /tmp/tmpxadaezwi/var/tests
60 remove-stagedir: false
61
Command Line Interface to buildtest configuration (buildtest config)
Once you have implemented your buildtest configuration, you can query the configuration
details using buildtest config command. Shown below is the command usage.
buildtest config --help
$ buildtest config --help
usage: buildtest [options] [COMMANDS] config [-h] ...
optional arguments:
-h, --help show this help message and exit
subcommands:
Query information from buildtest configuration file
edit (e) Open configuration file in editor
path (p) Show path to configuration file
systems List all available systems
validate (val)
Validate buildtest settings file with schema.
view (v) View configuration file
profiles Query profile from buildtest configuration
executors (ex)
Query executors from buildtest configuration
compilers (co)
Search compilers
Note
buildtest cg is an alias for buildtest config command.
Validate buildtest configuration (buildtest config validate)
First thing you should do once you implement your configuration file is to make sure your configuration is valid with the schema.
This can be achieved by running buildtest config validate. When you invoke this
command, buildtest will load the configuration and attempt to validate the file with
schema settings.schema.json. If validation is successful you will get the following message:
$ buildtest config validate
/tmp/tmpxadaezwi/config.yml is valid
Note
If you defined a user setting (~/.buildtest/config.yml) buildtest will validate this file instead of default one.
If there is an error during validation, the output from jsonschema.exceptions.ValidationError
will be displayed in terminal. For example the error below indicates that
moduletool property was expecting one of the values
[environment-modules, lmod, N/A] but it received a value of none:
Invalid buildtest configuration
$ buildtest config validate Traceback (most recent call last): File "/Users/siddiq90/Documents/buildtest/bin/buildtest", line 17, in <module> buildtest.main.main() File "/Users/siddiq90/Documents/buildtest/buildtest/main.py", line 39, in main buildtest_configuration = check_settings(settings_file, retrieve_settings=True) File "/Users/siddiq90/Documents/buildtest/buildtest/config.py", line 41, in check_settings validate(instance=user_schema, schema=config_schema) File "/Users/siddiq90/.local/share/virtualenvs/buildtest-1gHVG2Pd/lib/python3.7/site-packages/jsonschema/validators.py", line 934, in validate raise error jsonschema.exceptions.ValidationError: 'none' is not one of ['environment-modules', 'lmod', 'N/A'] Failed validating 'enum' in schema['properties']['moduletool']: {'description': 'Specify modules tool used for interacting with ' '``module`` command. ', 'enum': ['environment-modules', 'lmod', 'N/A'], 'type': 'string'} On instance['moduletool']: 'none'
View buildtest configuration (buildtest config view)
If you want to view buildtest configuration you can run buildtest config view which will print content of buildtest configuration.
buildtest config view
$ buildtest config view
───────────────────────── /tmp/tmpxadaezwi/config.yml ──────────────────────────
1 system:
2 generic:
3 hostnames:
4 - .*
5 description: Generic System
6 moduletool: N/A
7 poolsize: 1
8 pager: false
9 buildspecs:
10 rebuild: false
11 count: 15
12 terse: false
13 report:
14 count: 25
15 terse: false
16 format: name,id,state,runtime,returncode
17 latest: false
18 oldest: false
19 executors:
20 local:
21 bash:
22 description: submit jobs on local machine using bash shell
23 shell: bash
24 sh:
25 description: submit jobs on local machine using sh shell
26 shell: sh
27 csh:
28 description: submit jobs on local machine using csh shell
29 shell: csh
30 zsh:
31 description: submit jobs on local machine using zsh shell
32 shell: zsh
33 compilers:
34 compiler:
35 gcc:
36 builtin_gcc:
37 cc: gcc
38 fc: gfortran
39 cxx: g++
40 cdash:
41 url: https://my.cdash.org/
42 project: buildtest
43 site: generic
44 buildname: tutorials
45 profiles:
46 python:
47 tags:
48 - python
49 testdir: /tmp/tmpxadaezwi/var/tests
50 remove-stagedir: false
51 prof1:
52 tags:
53 - python
54 testdir: /tmp/tmpxadaezwi/var/tests
55 remove-stagedir: false
56 prof2:
57 buildspecs:
58 - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/shell_examples.yml
59 testdir: /tmp/tmpxadaezwi/var/tests
60 remove-stagedir: false
61
The --theme option can be used to change the color theme of the output. The default theme is monokai and list of
available themes can be retrieved with tab completion for option buildtest config view --theme.
Check path to buildtest configuration file (buildtest config path)
If you want to check path to buildtest configuration file you can run buildtest config path which will print path of buildtest configuration file.
$ buildtest config path
/tmp/tmpxadaezwi/config.yml
Edit buildtest configuration (buildtest config edit)
The buildtest config edit command is used to open buildtest configuration file in your preferred editor. buildtest will
use the environment EDITOR to get the preffered editor; however, one can override the environment variable via command line option
buildtest --editor.
View Executors (buildtest config executors list)
You can use the command buildtest config executors list to view executors from buildtest
configuration file. Shown below is the command usage
buildtest config executors list --help
$ buildtest config executors list --help
usage: buildtest [options] [COMMANDS] config executors list
[-h] [-j | -y | -d | -i | -a]
optional arguments:
-h, --help show this help message and exit
-j, --json View executor in JSON format
-y, --yaml View executors in YAML format
-d, --disabled Show disabled executors
-i, --invalid Show invalid executors
-a, --all Show all executors
You can run buildtest config executors list without any options and it will report a list of named executors that
you would reference in buildspec using the executor property.
$ buildtest config executors list
generic.local.bash
generic.local.sh
generic.local.csh
generic.local.zsh
If you want to see the executor details, you may want to use --json or --yaml option which will show the executor settings in YAML or JSON format.
Shown below is an example output
buildtest config executors list --yaml
$ buildtest config executors list --yaml
executors:
local:
bash:
description: submit jobs on local machine using bash shell
shell: bash
csh:
description: submit jobs on local machine using csh shell
shell: csh
sh:
description: submit jobs on local machine using sh shell
shell: sh
zsh:
description: submit jobs on local machine using zsh shell
shell: zsh
buildtest config executors list --json
$ buildtest config executors list --json
{
"executors": {
"local": {
"bash": {
"description": "submit jobs on local machine using bash shell",
"shell": "bash"
},
"sh": {
"description": "submit jobs on local machine using sh shell",
"shell": "sh"
},
"csh": {
"description": "submit jobs on local machine using csh shell",
"shell": "csh"
},
"zsh": {
"description": "submit jobs on local machine using zsh shell",
"shell": "zsh"
}
}
}
}
Note
The command options for buildtest config executors list are mutually exclusive, so if you
specify multiple options you will get the following error.
$ buildtest config executors list --json --yaml
usage: buildtest [options] [COMMANDS] config executors list
[-h] [-j | -y | -d | -i | -a]
buildtest [options] [COMMANDS] config executors list: error: argument -y/--yaml: not allowed with argument -j/--json
Remove Executors (buildtest config executors remove)
The buildtest config executors remove command will remove an executor from buildtest configuration file.
The positional arguments are the name of the executor you want to remove. Tab completion is available to retrieve
all available executors (buildtest config executors list --all).
Shown below we see tab completion on available executors that can be removed.
buildtest config executors remove
generic.local.bash generic.local.csh generic.local.sh generic.local.zsh
Let’s try listing all executors and remove generic.local.zsh executor, you will notice after deletion,
the configuration file is updated and the executor is no longer present.
Removing executor ‘generic.local.zsh’
$ buildtest --config $BUILDTEST_CI_DIR/config.yml config executors list --all
generic.local.bash
generic.local.sh
generic.local.csh
generic.local.zsh
$ buildtest --config $BUILDTEST_CI_DIR/config.yml config executors remove generic.local.zsh
Removing executor: generic.local.zsh
Updating configuration file: /tmp/tmpxadaezwi/config.yml
Upon deletion, buildtest will validate the configuration before writing the changes back to disk, to ensure the configuration is valid. Shown below we demonstrate an example where we attempt to remove all executors from the configuration file. Buildtest expects there is atleast 1 executor definition for local executor.
buildtest config executors remove generic.local.bash generic.local.sh generic.local.csh generic.local.zsh
$ buildtest config executors remove generic.local.bash generic.local.sh generic.local.csh generic.local.zsh
Removing executor: generic.local.bash
Removing executor: generic.local.sh
Removing executor: generic.local.csh
Unable to remove executor: generic.local.zsh because there are no executors of type: local
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ bin/buildtest:38 in <module> │
│ │
│ 35 │
│ 36 if __name__ == "__main__": │
│ 37 │
│ ❱ 38 sys.exit(main()) │
│ 39 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ buildtest_file = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ main = <function main at 0x7f04abd0bee0> │ │
│ │ os = <module 'os' from │ │
│ │ '/home/docs/.asdf/installs/python/3.8.18/lib/python3.8… │ │
│ │ prefix = '/home/docs/checkouts/readthedocs.org/user_builds/buil… │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/main.py:303 in main │
│ │
│ 300 │ │ if args.config in ["compilers", "co"]: │
│ 301 │ │ │ compiler_cmd(args, configuration) │
│ 302 │ │ else: │
│ ❱ 303 │ │ │ config_cmd(args, configuration, buildtest_editor, system) │
│ 304 │ │
│ 305 │ # buildtest report │
│ 306 │ elif args.subcommands in ["report", "rt"]: │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(color=None, config='executors', │ │
│ │ configfile=None, debug=False, editor=None, │ │
│ │ executor_names=['generic.local.bash', │ │
│ │ 'generic.local.sh', 'generic.local.csh', │ │
│ │ 'generic.local.zsh'], executors='remove', │ │
│ │ helpcolor=False, loglevel='DEBUG', logpath=False, │ │
│ │ no_color=False, print_log=False, report=None, │ │
│ │ subcommands='config', view_log=False) │ │
│ │ buildtest_editor = '/usr/bin/vim' │ │
│ │ config_file = None │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f04a9e7d790> │ │
│ │ logger = <Logger buildtest (DEBUG)> │ │
│ │ no_color = False │ │
│ │ parser = <buildtest.cli.BuildTestParser object at │ │
│ │ 0x7f04abdafa30> │ │
│ │ report_file = None │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f04a9e7d1f0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/config.py:48 in config_cmd │
│ │
│ 45 │ │ │ │ all_executors=args.all, │
│ 46 │ │ │ ) │
│ 47 │ │ if args.executors in ["remove", "rm"]: │
│ ❱ 48 │ │ │ remove_executors(configuration, args.executor_names) │
│ 49 │ │
│ 50 │ elif args.config in ["validate", "val"]: │
│ 51 │ │ validate_config(configuration, system.system["moduletool"]) │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = Namespace(color=None, config='executors', │ │
│ │ configfile=None, debug=False, editor=None, │ │
│ │ executor_names=['generic.local.bash', │ │
│ │ 'generic.local.sh', 'generic.local.csh', │ │
│ │ 'generic.local.zsh'], executors='remove', │ │
│ │ helpcolor=False, loglevel='DEBUG', logpath=False, │ │
│ │ no_color=False, print_log=False, report=None, │ │
│ │ subcommands='config', view_log=False) │ │
│ │ buildexecutor = [buildtest-executor] │ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f04a9e7d790> │ │
│ │ editor = '/usr/bin/vim' │ │
│ │ system = <buildtest.system.BuildTestSystem object at │ │
│ │ 0x7f04a9e7d1f0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/cli/config.py:337 in remove_executors │
│ │
│ 334 │ │ console.print(f"Removing executor: {name}") │
│ 335 │ │ write_back = True │
│ 336 │ │
│ ❱ 337 │ custom_validator( │
│ 338 │ │ configuration.config, schema_table["settings.schema.json"]["re │
│ 339 │ ) │
│ 340 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ configuration = <buildtest.config.SiteConfiguration object at │ │
│ │ 0x7f04a9e7d790> │ │
│ │ exec_name = 'zsh' │ │
│ │ exec_type = 'local' │ │
│ │ executor_names = [ │ │
│ │ │ 'generic.local.bash', │ │
│ │ │ 'generic.local.sh', │ │
│ │ │ 'generic.local.csh', │ │
│ │ │ 'generic.local.zsh' │ │
│ │ ] │ │
│ │ name = 'generic.local.zsh' │ │
│ │ write_back = True │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ │
│ buildtest/schemas/defaults.py:113 in custom_validator │
│ │
│ 110 │ assert isinstance(schema, dict) │
│ 111 │ │
│ 112 │ validator = Draft7Validator(schema, resolver=resolver) │
│ ❱ 113 │ validator.validate(recipe) │
│ 114 │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ recipe = { │ │
│ │ │ 'system': { │ │
│ │ │ │ 'generic': { │ │
│ │ │ │ │ 'hostnames': ['.*'], │ │
│ │ │ │ │ 'description': 'Generic System', │ │
│ │ │ │ │ 'moduletool': 'N/A', │ │
│ │ │ │ │ 'poolsize': 1, │ │
│ │ │ │ │ 'pager': False, │ │
│ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ 'rebuild': False, │ │
│ │ │ │ │ │ 'count': 15, │ │
│ │ │ │ │ │ 'terse': False │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'report': { │ │
│ │ │ │ │ │ 'count': 25, │ │
│ │ │ │ │ │ 'terse': False, │ │
│ │ │ │ │ │ 'format': │ │
│ │ 'name,id,state,runtime,returncode', │ │
│ │ │ │ │ │ 'latest': False, │ │
│ │ │ │ │ │ 'oldest': False │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'executors': {}, │ │
│ │ │ │ │ 'compilers': { │ │
│ │ │ │ │ │ 'compiler': { │ │
│ │ │ │ │ │ │ 'gcc': { │ │
│ │ │ │ │ │ │ │ 'builtin_gcc': { │ │
│ │ │ │ │ │ │ │ │ 'cc': 'gcc', │ │
│ │ │ │ │ │ │ │ │ 'fc': 'gfortran', │ │
│ │ │ │ │ │ │ │ │ 'cxx': 'g++' │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ } │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ 'cdash': { │ │
│ │ │ │ │ │ 'url': 'https://my.cdash.org/', │ │
│ │ │ │ │ │ 'project': 'buildtest', │ │
│ │ │ │ │ │ 'site': 'generic', │ │
│ │ │ │ │ │ 'buildname': 'tutorials' │ │
│ │ │ │ │ }, │ │
│ │ │ │ │ ... +1 │ │
│ │ │ │ } │ │
│ │ │ } │ │
│ │ } │ │
│ │ schema = { │ │
│ │ │ '$id': 'settings.schema.json', │ │
│ │ │ '$schema': 'http://json-schema.org/draft-07/schema#', │ │
│ │ │ 'title': 'buildtest configuration schema', │ │
│ │ │ 'type': 'object', │ │
│ │ │ 'required': ['system'], │ │
│ │ │ 'additionalProperties': False, │ │
│ │ │ 'properties': { │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'patternProperties': { │ │
│ │ │ │ │ │ '^.*$': {'$ref': '#/definitions/system'} │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ }, │ │
│ │ │ 'definitions': { │ │
│ │ │ │ 'terse': { │ │
│ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ 'description': 'A boolean to determine whether │ │
│ │ to enable terse mode' │ │
│ │ │ │ }, │ │
│ │ │ │ 'format': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Determine the format fields to │ │
│ │ display when viewing table results' │ │
│ │ │ │ }, │ │
│ │ │ │ 'count': { │ │
│ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ 'description': 'Determine number of records to │ │
│ │ display in a table' │ │
│ │ │ │ }, │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'required': [ │ │
│ │ │ │ │ │ 'executors', │ │
│ │ │ │ │ │ 'moduletool', │ │
│ │ │ │ │ │ 'hostnames', │ │
│ │ │ │ │ │ 'compilers', │ │
│ │ │ │ │ │ 'buildspecs', │ │
│ │ │ │ │ │ 'report' │ │
│ │ │ │ │ ], │ │
│ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ 'hostnames': { │ │
│ │ │ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ │ │ 'description': 'Specify a list of │ │
│ │ hostnames to check where buildtest can run for the given │ │
│ │ syste'+8, │ │
│ │ │ │ │ │ │ 'items': {'type': 'string'} │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'description': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'system description │ │
│ │ field' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'poolsize': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Specify size of Process │ │
│ │ Pool for parallel processing using ``multiprocessing.Poo'+3 │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'max_jobs': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Maximum number of jobs │ │
│ │ that can be run at a given time for a particular executor' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'testdir': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify full path to │ │
│ │ test directory where buildtest will write tests.' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'logdir': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify location where │ │
│ │ buildtest will write log files' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'moduletool': { │ │
│ │ │ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ │ │ 'description': 'Specify modules tool │ │
│ │ used for interacting with ``module`` command. ', │ │
│ │ │ │ │ │ │ 'enum': [ │ │
│ │ │ │ │ │ │ │ 'environment-modules', │ │
│ │ │ │ │ │ │ │ 'lmod', │ │
│ │ │ │ │ │ │ │ 'N/A' │ │
│ │ │ │ │ │ │ ] │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'timeout': { │ │
│ │ │ │ │ │ │ 'type': 'integer', │ │
│ │ │ │ │ │ │ 'minimum': 1, │ │
│ │ │ │ │ │ │ 'description': 'Specify timeout duration │ │
│ │ in number of seconds' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'pager': { │ │
│ │ │ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ │ │ 'description': 'A boolean to determine │ │
│ │ whether to enable paging when viewing buildspec cache' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ │ │ 'description': 'Specify configuration │ │
│ │ for ``buildtest buildspec`` command', │ │
│ │ │ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ │ │ 'rebuild': { │ │
│ │ │ │ │ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ │ │ │ │ 'description': 'A boolean to │ │
│ │ determine whether to rebuild buildspec cache' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'count': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/count' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'format': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/format' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'terse': { │ │
│ │ │ │ │ │ │ │ │ '$ref': '#/definitions/terse' │ │
│ │ │ │ │ │ │ │ }, │ │
│ │ │ │ │ │ │ │ 'root': { │ │
│ │ │ │ │ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ │ │ │ │ 'items': {'type': 'string'}, │ │
│ │ │ │ │ │ │ │ │ 'description': 'Specify a list │ │
│ │ of directory paths to search buildspecs. This field can be │ │
│ │ used w'+108 │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ ... +6 │ │
│ │ │ │ │ } │ │
│ │ │ │ }, │ │
│ │ │ │ 'cc': { │ │
│ │ │ │ │ 'description': 'Specify path to C compiler │ │
│ │ wrapper. You may specify a compiler wrapper such as `'+101, │ │
│ │ │ │ │ 'type': 'string' │ │
│ │ │ │ }, │ │
│ │ │ │ 'cxx': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Specify path to C++ compiler │ │
│ │ wrapper. You may specify a compiler wrapper such as'+103 │ │
│ │ │ │ }, │ │
│ │ │ │ 'fc': { │ │
│ │ │ │ │ 'type': 'string', │ │
│ │ │ │ │ 'description': 'Specify path to Fortran compiler │ │
│ │ wrapper. You may specify a compiler wrapper suc'+112 │ │
│ │ │ │ }, │ │
│ │ │ │ 'compiler_section': { │ │
│ │ │ │ │ 'description': 'A compiler section is composed │ │
│ │ of ``cc``, ``cxx`` and ``fc`` wrapper these are r'+50, │ │
│ │ │ │ │ 'type': 'object', │ │
│ │ │ │ │ 'additionalProperties': False, │ │
│ │ │ │ │ 'required': ['cc', 'cxx', 'fc'], │ │
│ │ │ │ │ 'properties': { │ │
│ │ │ │ │ │ 'cc': {'$ref': '#/definitions/cc'}, │ │
│ │ │ │ │ │ 'cxx': {'$ref': '#/definitions/cxx'}, │ │
│ │ │ │ │ │ 'fc': {'$ref': '#/definitions/fc'}, │ │
│ │ │ │ │ │ 'module': {'$ref': '#/definitions/module'} │ │
│ │ │ │ │ } │ │
│ │ │ │ }, │ │
│ │ │ │ 'unique_string_array': { │ │
│ │ │ │ │ 'type': 'array', │ │
│ │ │ │ │ 'uniqueItems': True, │ │
│ │ │ │ │ 'items': {'type': 'string'} │ │
│ │ │ │ }, │ │
│ │ │ │ 'disable': { │ │
│ │ │ │ │ 'type': 'boolean', │ │
│ │ │ │ │ 'description': 'Disable executor' │ │
│ │ │ │ }, │ │
│ │ │ │ ... +10 │ │
│ │ │ } │ │
│ │ } │ │
│ │ validator = Draft7Validator( │ │
│ │ │ schema={'$id': 'settings.schema.json', '$schema': │ │
│ │ 'http://json-...ft-07/schema#', 'additionalProperties': │ │
│ │ False, 'definitions': {'account': {'description': 'Specify │ │
│ │ Job ...ing resources', 'type': 'string'}, 'cc': │ │
│ │ {'description': 'Specify path...iler wrapper.', 'type': │ │
│ │ 'string'}, 'cobalt': {'additionalProperties': False, │ │
│ │ 'description': 'An instance ...balt executor', 'properties': │ │
│ │ {'account': {'$ref': '#/definitions/account', 'description': │ │
│ │ 'overrides de...count`` value'}, 'before_script': {'#ref': │ │
│ │ '#/definitions/script', 'description': 'The │ │
│ │ ``before...active shell.'}, 'description': {'description': │ │
│ │ 'description ...your executor', 'type': 'string'}, │ │
│ │ 'disable': {'$ref': '#/definitions/disable'}, ...}, │ │
│ │ 'required': ['queue'], ...}, 'compiler_section': │ │
│ │ {'additionalProperties': False, 'description': 'A compiler │ │
│ │ s...iler wrapper.', 'properties': {'cc': {'$ref': │ │
│ │ '#/definitions/cc'}, 'cxx': {'$ref': '#/definitions/cxx'}, │ │
│ │ 'fc': {'$ref': '#/definitions/fc'}, 'module': {'$ref': │ │
│ │ '#/definitions/module'}}, 'required': ['cc', 'cxx', 'fc'], │ │
│ │ ...}, ...}, ...}, │ │
│ │ │ format_checker=None │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/lib/p │
│ ython3.8/site-packages/jsonschema/validators.py:314 in validate │
│ │
│ 311 │ │ │
│ 312 │ │ def validate(self, *args, **kwargs): │
│ 313 │ │ │ for error in self.iter_errors(*args, **kwargs): │
│ ❱ 314 │ │ │ │ raise error │
│ 315 │ │ │
│ 316 │ │ def is_type(self, instance, type): │
│ 317 │ │ │ try: │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ args = ( │ │
│ │ │ { │ │
│ │ │ │ 'system': { │ │
│ │ │ │ │ 'generic': { │ │
│ │ │ │ │ │ 'hostnames': ['.*'], │ │
│ │ │ │ │ │ 'description': 'Generic System', │ │
│ │ │ │ │ │ 'moduletool': 'N/A', │ │
│ │ │ │ │ │ 'poolsize': 1, │ │
│ │ │ │ │ │ 'pager': False, │ │
│ │ │ │ │ │ 'buildspecs': { │ │
│ │ │ │ │ │ │ 'rebuild': False, │ │
│ │ │ │ │ │ │ 'count': 15, │ │
│ │ │ │ │ │ │ 'terse': False │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'report': { │ │
│ │ │ │ │ │ │ 'count': 25, │ │
│ │ │ │ │ │ │ 'terse': False, │ │
│ │ │ │ │ │ │ 'format': │ │
│ │ 'name,id,state,runtime,returncode', │ │
│ │ │ │ │ │ │ 'latest': False, │ │
│ │ │ │ │ │ │ 'oldest': False │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'executors': {}, │ │
│ │ │ │ │ │ 'compilers': { │ │
│ │ │ │ │ │ │ 'compiler': { │ │
│ │ │ │ │ │ │ │ 'gcc': { │ │
│ │ │ │ │ │ │ │ │ 'builtin_gcc': { │ │
│ │ │ │ │ │ │ │ │ │ 'cc': 'gcc', │ │
│ │ │ │ │ │ │ │ │ │ 'fc': 'gfortran', │ │
│ │ │ │ │ │ │ │ │ │ 'cxx': 'g++' │ │
│ │ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ │ } │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ 'cdash': { │ │
│ │ │ │ │ │ │ 'url': 'https://my.cdash.org/', │ │
│ │ │ │ │ │ │ 'project': 'buildtest', │ │
│ │ │ │ │ │ │ 'site': 'generic', │ │
│ │ │ │ │ │ │ 'buildname': 'tutorials' │ │
│ │ │ │ │ │ }, │ │
│ │ │ │ │ │ ... +1 │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ }, │ │
│ │ ) │ │
│ │ error = <ValidationError: "'local' is a required property"> │ │
│ │ kwargs = {} │ │
│ │ self = Draft7Validator( │ │
│ │ │ schema={'$id': 'settings.schema.json', '$schema': │ │
│ │ 'http://json-...ft-07/schema#', 'additionalProperties': False, │ │
│ │ 'definitions': {'account': {'description': 'Specify Job ...ing │ │
│ │ resources', 'type': 'string'}, 'cc': {'description': 'Specify │ │
│ │ path...iler wrapper.', 'type': 'string'}, 'cobalt': │ │
│ │ {'additionalProperties': False, 'description': 'An instance │ │
│ │ ...balt executor', 'properties': {'account': {'$ref': │ │
│ │ '#/definitions/account', 'description': 'overrides de...count`` │ │
│ │ value'}, 'before_script': {'#ref': '#/definitions/script', │ │
│ │ 'description': 'The ``before...active shell.'}, 'description': │ │
│ │ {'description': 'description ...your executor', 'type': │ │
│ │ 'string'}, 'disable': {'$ref': '#/definitions/disable'}, ...}, │ │
│ │ 'required': ['queue'], ...}, 'compiler_section': │ │
│ │ {'additionalProperties': False, 'description': 'A compiler │ │
│ │ s...iler wrapper.', 'properties': {'cc': {'$ref': │ │
│ │ '#/definitions/cc'}, 'cxx': {'$ref': '#/definitions/cxx'}, │ │
│ │ 'fc': {'$ref': '#/definitions/fc'}, 'module': {'$ref': │ │
│ │ '#/definitions/module'}}, 'required': ['cc', 'cxx', 'fc'], │ │
│ │ ...}, ...}, ...}, │ │
│ │ │ format_checker=None │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
ValidationError: 'local' is a required property
Failed validating 'required' in
schema['properties']['system']['patternProperties']['^.*$']['properties']['execu
tors']:
{'additionalProperties': False,
'description': 'The executor section is used for declaring your '
'executors that are responsible for running jobs. The '
'executor section can be ``local``, ``lsf``, '
'``slurm``, ``cobalt``. The executors are referenced '
'in buildspec using ``executor`` field.',
'properties': {'cobalt': {'additionalProperties': {'$ref':
'#/definitions/cobalt'},
'description': 'The ``cobalt`` section is '
'used for declaring Cobalt '
'executors for running jobs '
'using Cobalt scheduler',
'propertyNames': {'pattern':
'^[A-Za-z0-9_.-]+$'},
'type': 'object'},
'defaults': {'additionalProperties': False,
'description': 'Specify default executor '
'settings for all '
'executors',
'properties': {'account': {'$ref':
'#/definitions/account'},
'maxpendtime': {'$ref':
'#/definitions/maxpendtime'},
'pollinterval': {'default': 30,
'description':
'Specify '
'poll '
'interval '
'in '
'seconds '
'after '
'job '
'submission, '
'where '
'buildtest '
'will '
'sleep '
'and '
'poll '
'all '
'jobs '
'for '
'job '
'states. '
'This '
'field '
'can '
'be '
'configured '
'based '
'on '
'your '
'preference. '
'Excessive '
'polling '
'every '
'few '
'seconds '
'can '
'result '
'in '
'system '
'degradation. ',
'minimum': 1,
'type':
'integer'}},
'type': 'object'},
'local': {'additionalProperties': {'$ref':
'#/definitions/local'},
'description': 'The ``local`` section is '
'used for declaring local '
'executors for running jobs '
'on local machine',
'minProperties': 1,
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'type': 'object'},
'lsf': {'additionalProperties': {'$ref':
'#/definitions/lsf'},
'description': 'The ``lsf`` section is used '
'for declaring LSF executors '
'for running jobs using LSF '
'scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'type': 'object'},
'pbs': {'additionalProperties': {'$ref':
'#/definitions/pbs'},
'description': 'The ``pbs`` section is used '
'for declaring PBS executors '
'for running jobs using PBS '
'scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'type': 'object'},
'slurm': {'additionalProperties': {'$ref':
'#/definitions/slurm'},
'description': 'The ``slurm`` section is '
'used for declaring Slurm '
'executors for running jobs '
'using Slurm scheduler',
'propertyNames': {'pattern': '^[A-Za-z0-9_.-]+$'},
'type': 'object'}},
'required': ['local'],
'type': 'object'}
On instance['system']['generic']['executors']:
{}
View Registered Systems (buildtest config systems)
Your buildtest configuration may compose of one or more systems since you can define multiple systems
in a single configuration file to run buildtest for different HPC clusters. You can use
buildtest config systems to report all system details defined in your configuration file.
In this example below we should the generic system. If you have multiple entries, you will see one
entry per system record.
$ buildtest config systems
System Summary (Configuration=/tmp/tmpxadaezwi/config.yml)
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ system ┃ description ┃ moduletool ┃ hostnames ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ generic │ Generic System │ N/A │ .* │
└───────────────┴─────────────────────────┴───────────────────┴────────────────┘
Example Configurations
buildtest provides a few example configurations for configuring buildtest this
can be retrieved by running buildtest schema -n settings.schema.json --examples
or short option (-e), which will validate each example with schema file
settings.schema.json.
buildtest schema -n settings.schema.json -e
$ buildtest schema -n settings.schema.json -e
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: N/A
executors:
local:
# the line below is invalid due to pattern property
==bash:
description: submit jobs on local machine
shell: bash -v
slurm:
# the line below is invalid due to pattern property
$haswell:
launcher: sbatch
options: ["-p haswell", "-t 00:10"]
lsf:
<>batch:
launcher: bsub
queue: batch
options: ["-q batch", "-t 00:10"]
cobalt:
<normal>:
launcher: qsub
queue: normal
options: ["-n 1", "-t 10"]
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: N/A
poolsize: 1
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
executors:
defaults:
pollinterval: 10
maxpendtime: 30
local:
bash:
description: submit jobs via bash shell
shell: bash
pbs:
workq:
queue: workq
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
#logdir: $BUILDTEST_ROOT/logs
#testdir: $BUILDTEST_ROOT/tests
moduletool: N/A
poolsize: 1
max_jobs: 1
pager: False
cdash:
url: https://my.cdash.org
project: buildtest
site: laptop
processor:
numcpus: 8
cores: 4
threads_per_core: 2
sockets: 1
model: "Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz"
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
executors:
local:
bash:
description: submit jobs on local machine using bash shell
shell: bash
module:
purge: True
restore: "Default"
load: ["gcc"]
sh:
description: submit jobs on local machine using sh shell
shell: sh
before_script: |
date
echo "these commands will be run"
csh:
description: submit jobs on local machine using csh shell
shell: csh -x
tcsh:
description: submit jobs on local machine using tcsh shell
shell: /bin/tcsh
zsh:
description: submit jobs on local machine using zsh shell
shell: /bin/zsh
compilers:
find:
gcc: "^(gcc|GCC|PrgEnv-gnu)"
intel: "^(intel|Intel|PrgEnv-intel)"
cray: "^(cray|PrgEnv-cray)"
clang: "^(clang|Clang)"
cuda: "^(cuda|CUDA)"
pgi: "^(pgi|PGI|PrgEnv-pgi)"
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
gcc@7.2.0:
cc: 'cc'
cxx: 'cxx'
fc: 'fc'
module:
load:
- gcc/7.2.0
intel:
intel@2019:
cc: 'icc'
cxx: 'icpc'
fc: 'ifort'
module:
purge: True
load:
- gcc/7.2.0
- intel/2019
cray:
craype@2.6.2:
cc: 'cc'
cxx: 'CC'
fc: 'fc'
module:
load: [craype/2.6.2]
swap: [PrgEnv-gnu, PrgEnv-cray]
clang:
clang@12.0.0:
cc: 'clang'
cxx: 'clang++'
fc: 'None'
module:
load: [clang/12.0]
cuda:
cuda@11.0:
cc: 'nvcc'
cxx: 'nvcc'
fc: 'None'
module:
load: [cuda/11.0]
pgi:
pgi@18.0:
cc: 'pgcc'
cxx: 'pgc++'
fc: 'pgfortran'
module:
load: [pgi/18.0]
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: N/A
poolsize: 1
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
executors:
local:
bash:
description: submit jobs on local machine
shell: bash -v
slurm:
haswell:
launcher: sbatch
options: ["-p haswell", "-t 00:10"]
lsf:
batch:
launcher: bsub
queue: batch
options: ["-q batch", "-t 00:10"]
cobalt:
normal:
launcher: qsub
queue: normal
options: ["-n 1", "-t 10"]
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: lmod
poolsize: 1
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
executors:
defaults:
pollinterval: 10
maxpendtime: 45
local:
bash:
description: submit jobs via bash shell
shell: bash
lsf:
batch:
description: "LSF Executor name 'batch' that submits jobs to 'batch' q
queue: batch
account: developer
options: ["-W 20"]
before_script: |
time
echo "commands run before job"
test:
description: "LSF Executor name 'test' that submits jobs to 'test' que
launcher: bsub
queue: test
account: qa
options: ["-W 20"]
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: lmod
poolsize: 1
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
testdir: /tmp/buildtest
executors:
defaults:
pollinterval: 20
maxpendtime: 30
account: admin
local:
bash:
description: submit jobs via bash shell
shell: bash
slurm:
normal:
options: ["-C haswell"]
qos: normal
before_script: |
time
echo "commands run before job"
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:
generic:
hostnames: ['.*']
moduletool: lmod
poolsize: 1
executors:
defaults:
maxpendtime: 30
local:
bash:
description: submit jobs via bash shell
shell: bash
cobalt:
knl:
queue: knl
haswell:
queue: haswell
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buil
rebuild: False
# limit number of records to display when running `buildtest buildspec fin
count: 15
# format fields to display when running `buildtest buildspec find`, By def
format: "name,description"
# enable terse mode
terse: False
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False
compilers:
compiler:
gcc:
default:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
If you want to retrieve full json schema file for buildtest configuration you can
run buildtest schema -n settings.schema.json --json or short option -j.
Accessing Log File
Buildtest has several options to retrieve content of logfile, the --print-log or --view-log can be used to
display output of the logfile. The --print-log will display output whereas --view-log will display in paginated mode.
The --logpath can be used to retrieve the path to logfile.
$ buildtest --logpath
/tmp/tmpxadaezwi/var/buildtest.log
Shown below is an example output of log content using the --print-log option
$ buildtest --print-log | head -n 10
2023-11-13 01:40:32,591 - [DEBUG] Starting System Compatibility Check
2023-11-13 01:40:32,593 - [INFO] Machine: x86_64
2023-11-13 01:40:32,593 - [INFO] Host: build-22529914-project-280831-buildtest
2023-11-13 01:40:32,593 - [INFO] User: docs
2023-11-13 01:40:32,593 - [INFO] Operating System: ubuntu
2023-11-13 01:40:32,593 - [INFO] System Kernel: Linux and Kernel Release: 5.19.0-1028-aws
2023-11-13 01:40:32,593 - [INFO] Python Path: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3
2023-11-13 01:40:32,593 - [INFO] Python Version: 3.8.18
2023-11-13 01:40:32,593 - [INFO] BUILDTEST_ROOT: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable
2023-11-13 01:40:32,593 - [INFO] Path to Buildtest: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest
Show All Options and Commands
The buildtest --help-all will display all commands and options available for buildtest. Some options are
hidden by default when using buildtest --help. Please refer to the following output for list of available
commands and options supported by buildtest.
$ buildtest --help-all
usage: buildtest [options] [COMMANDS]
buildtest is a HPC testing framework for building and running tests.
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-c CONFIGFILE, --configfile CONFIGFILE
Specify Path to Configuration File
-d, --debug Stream log messages to stdout
-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Filter log messages based on logging level
--editor {vi,vim,emacs,nano}
Select your preferred editor when opening files.
--view-log Show content of last log
--logpath Print full path to last log file
--print-log Print content of last log without pagination
--color COLOR Print output of table with the selected color.
--no-color Disable colored output
--helpcolor Print available color options in a table format.
-r REPORT, --report REPORT
Specify path to test report file
-H, --help-all List all commands and options
COMMANDS:
build (bd) Build and Run test
buildspec (bc) Buildspec Interface
config (cg) Query buildtest configuration
report (rt) Query test report
inspect (it) Inspect a test
path (p) Show path attributes for a given test
history (hy) Query build history
schema List schema contents and examples
cdash Upload test to CDASH server
cd Change directory to root of test given a test name
clean Remove all generate files from buildtest including
test directory, log files, report file, buildspec
cache, history files
debugreport (debug)
Display system information and additional information
for debugging purposes.
stats Show test statistics for given test
info Show details regarding current buildtest setup
show buildtest command guide
commands (cmds) List all buildtest commands
tutorial-examples Generate documentation examples for Buildtest Tutorial
docs Open buildtest docs in browser
schemadocs Open buildtest schema docs in browser
unittests (test) Run buildtest unit tests
stylecheck (style) Run buildtest style checks
References
GitHub: https://github.com/buildtesters/buildtest
Documentation: https://buildtest.readthedocs.io/en/latest/index.html
Schema Documentation: https://buildtesters.github.io/buildtest/
Slack: http://hpcbuildtest.slack.com/
Please report issues at https://github.com/buildtesters/buildtest/issues
Copyright (c) 2021-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy), Shahzeb Siddiqui, and Vanessa Sochat. All rights reserved.