Additional Features

Accessing build history (buildtest history)

Note

buildtest hy is an alias for buildtest history command.

buildtest keeps track of all builds (buildtest build) that can be retrieved using buildtest history command which can be useful when you want to analyze or troubleshoot past builds. The buildtest history command comes with two subcommands buildtest history list and buildtest history query.

If you want to list all builds you should run buildtest history list which will report a table style format of all builds with corresponding build ID to differentiate each build. Shown below is an example output. The build IDs start at 0 and increment as you run buildtest build command.

buildtest history list
$ buildtest history list
┏━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃    ┃ hostnam ┃      ┃         ┃        ┃ pass    ┃ fail   ┃ total   ┃ comman ┃
┃ id ┃ e       ┃ user ┃ system  ┃ date   ┃ tests   ┃ tests  ┃ tests   ┃ d      ┃
┡━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ 0  │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 0      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 28     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ /home/ │
│    │         │      │         │        │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/tu │
│    │         │      │         │        │         │        │         │ torial │
│    │         │      │         │        │         │        │         │ s/vars │
│    │         │      │         │        │         │        │         │ .yml   │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 1  │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 2      │ 4       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 30     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/pa │
│    │         │      │         │        │         │        │         │ ss_ret │
│    │         │      │         │        │         │        │         │ urncod │
│    │         │      │         │        │         │        │         │ e.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 2  │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 2      │ 4       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 30     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/st │
│    │         │      │         │        │         │        │         │ atus_r │
│    │         │      │         │        │         │        │         │ egex.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 3  │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 1      │ 3       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 32     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/re │
│    │         │      │         │        │         │        │         │ gex_on │
│    │         │      │         │        │         │        │         │ _filen │
│    │         │      │         │        │         │        │         │ ame.ym │
│    │         │      │         │        │         │        │         │ l      │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 4  │ build-2 │ docs │ generic │ 2024/0 │ 3       │ 2      │ 5       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 43     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/ru │
│    │         │      │         │        │         │        │         │ ntime_ │
│    │         │      │         │        │         │        │         │ status │
│    │         │      │         │        │         │        │         │ _test. │
│    │         │      │         │        │         │        │         │ yml    │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 5  │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 2      │ 4       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 44     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/ex │
│    │         │      │         │        │         │        │         │ plicit │
│    │         │      │         │        │         │        │         │ _state │
│    │         │      │         │        │         │        │         │ .yml   │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 6  │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 1      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 45     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/ex │
│    │         │      │         │        │         │        │         │ ists.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 7  │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 0      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 47     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_exi │
│    │         │      │         │        │         │        │         │ sts_wi │
│    │         │      │         │        │         │        │         │ th_num │
│    │         │      │         │        │         │        │         │ ber.ym │
│    │         │      │         │        │         │        │         │ l      │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 8  │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 1      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 47     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_and │
│    │         │      │         │        │         │        │         │ _dir_c │
│    │         │      │         │        │         │        │         │ heck.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 9  │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 0      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 48     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/is │
│    │         │      │         │        │         │        │         │ _symli │
│    │         │      │         │        │         │        │         │ nk.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 10 │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 0      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 49     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_cou │
│    │         │      │         │        │         │        │         │ nt.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 11 │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 0      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 49     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_cou │
│    │         │      │         │        │         │        │         │ nt_pat │
│    │         │      │         │        │         │        │         │ tern.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 12 │ build-2 │ docs │ generic │ 2024/0 │ 0       │ 1      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 50     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_cou │
│    │         │      │         │        │         │        │         │ nt_fil │
│    │         │      │         │        │         │        │         │ etype. │
│    │         │      │         │        │         │        │         │ yml    │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 13 │ build-2 │ docs │ generic │ 2024/0 │ 0       │ 1      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 51     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/fi │
│    │         │      │         │        │         │        │         │ le_cou │
│    │         │      │         │        │         │        │         │ nt_fil │
│    │         │      │         │        │         │        │         │ e_trav │
│    │         │      │         │        │         │        │         │ erse_l │
│    │         │      │         │        │         │        │         │ imit.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 14 │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 0      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 51     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/te │
│    │         │      │         │        │         │        │         │ st_sta │
│    │         │      │         │        │         │        │         │ tus/mo │
│    │         │      │         │        │         │        │         │ de.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 15 │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 0      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 52     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ tutori │
│    │         │      │         │        │         │        │         │ als/sk │
│    │         │      │         │        │         │        │         │ ip_tes │
│    │         │      │         │        │         │        │         │ ts.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 16 │ build-2 │ docs │ generic │ 2024/0 │ 1       │ 0      │ 1       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 55     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ /home/ │
│    │         │      │         │        │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/tu │
│    │         │      │         │        │         │        │         │ torial │
│    │         │      │         │        │         │        │         │ s/hell │
│    │         │      │         │        │         │        │         │ o_worl │
│    │         │      │         │        │         │        │         │ d.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 17 │ build-2 │ docs │ generic │ 2024/0 │ 5       │ 5      │ 10      │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:16: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 57     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ /home/ │
│    │         │      │         │        │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/tu │
│    │         │      │         │        │         │        │         │ torial │
│    │         │      │         │        │         │        │         │ s/hell │
│    │         │      │         │        │         │        │         │ o_worl │
│    │         │      │         │        │         │        │         │ d.yml  │
│    │         │      │         │        │         │        │         │ -b     │
│    │         │      │         │        │         │        │         │ /home/ │
│    │         │      │         │        │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/ge │
│    │         │      │         │        │         │        │         │ neral_ │
│    │         │      │         │        │         │        │         │ tests/ │
│    │         │      │         │        │         │        │         │ config │
│    │         │      │         │        │         │        │         │ uratio │
│    │         │      │         │        │         │        │         │ n      │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 18 │ build-2 │ docs │ generic │ 2024/0 │ 2       │ 0      │ 2       │ /home/ │
│    │ 3444606 │      │         │ 2/14   │         │        │         │ docs/c │
│    │ -projec │      │         │ 17:17: │         │        │         │ heckou │
│    │ t-28083 │      │         │ 00     │         │        │         │ ts/rea │
│    │ 1-build │      │         │        │         │        │         │ dthedo │
│    │ test    │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/sta │
│    │         │      │         │        │         │        │         │ ble/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -t     │
│    │         │      │         │        │         │        │         │ networ │
│    │         │      │         │        │         │        │         │ k      │
└────┴─────────┴──────┴─────────┴────────┴─────────┴────────┴─────────┴────────┘

The buildtest history query command is particularly useful when you want to inspect a particular build. This command expects a Build Identifier which can be found by inspecting output column id in buildtest history list.

Shown below is an output of build ID 0 which reports relevant detail for the build such as input command, username, hostname, platform, date, etc…

buildtest history query 0
$ buildtest history query 0
{
│   'command': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest build -b /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml',
│   'user': 'docs',
│   'hostname': 'build-23444606-project-280831-buildtest',
│   'platform': 'Linux',
│   'date': '2024/02/14 17:16:28',
│   'buildtest': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest',
│   'python': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3',
│   'python_version': '3.8.18',
│   'testdir': '/tmp/tmpjcb9dx2w/var/tests',
│   'configuration': '/tmp/tmpjcb9dx2w/config.yml',
│   'system': 'generic',
│   'logpath': '/tmp/tmpjcb9dx2w/var/.history/0/buildtest_uaulb51y.log',
│   'invalid_buildspecs': [],
│   'buildspecs': {
│   │   'detected': [
│   │   │   '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml'
│   │   ],
│   │   'included': [
│   │   │   '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml'
│   │   ],
│   │   'excluded': []
│   },
│   'test_summary': {
│   │   'pass': '1',
│   │   'fail': '0',
│   │   'total': '1',
│   │   'pass_rate': '100.000',
│   │   'fail_rate': '0.000'
│   },
│   'builders': {
│   │   '17e5a942-9efe-478e-8c89-9bc6e35ea7ec': {
│   │   │   'name': 'variables_bash',
│   │   │   'buildspec': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml',
│   │   │   'tags': ['tutorials'],
│   │   │   'executors': 'generic.local.bash',
│   │   │   'state': 'PASS',
│   │   │   'returncode': 0,
│   │   │   'runtime': 0.009769,
│   │   │   'testpath': '/tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.sh',
│   │   │   'errfile': '/tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.err',
│   │   │   'outfile': '/tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.out'
│   │   }
│   }
}

If you want to see all available build IDs, you can use the following command. The -t is terse format and --no-header will omit the headers for each column and pipe the output to cut to extract the first column which corresponds to build IDs.

$ buildtest hy list --terse --no-header | cut -f 1 -d '|'
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

buildtest has tab complete on buildtest history query which reports a list of build IDs which is another way to see available IDs to query.

$ buildtest history query
0   1   10  11  12  13  14  15  16  17  18  19  2   20  21  22  23  3   4   5   6   7   8   9

If you want to see logfile for build ID 0 you can use --log option to see logfile in an editor as follows:

buildtest history query 0 --log

buildtest will store output of buildtest build command to file if command ran to completion. You can retrieve output of previous builds using the --output option. In example below we show output of build file

buildtest history query --output 0
$ buildtest history query --output 0
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-23444606-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2024/02/14 17:16:28                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.7                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.8.18                                                   │
│ Configuration File: /tmp/tmpjcb9dx2w/config.yml                              │
│ Test Directory:     /tmp/tmpjcb9dx2w/var/tests                               │
│ Report File:        /tmp/tmpjcb9dx2w/var/report.json                         │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/vars.yml                                                           ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/vars.yml: VALID
Total builder objects created: 1
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃          ┃        ┃          ┃          ┃       ┃       ┃ descript ┃ buildsp ┃
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion      ┃ ecs     ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ variable │ script │ generic. │ None     │ None  │ None  │ Declare  │ /home/d │
│ s_bash/1 │        │ local.ba │          │       │       │ shell    │ ocs/che │
│ 7e5a942  │        │ sh       │          │       │       │ variable │ ckouts/ │
│          │        │          │          │       │       │ s in     │ readthe │
│          │        │          │          │       │       │ bash     │ docs.or │
│          │        │          │          │       │       │          │ g/user_ │
│          │        │          │          │       │       │          │ builds/ │
│          │        │          │          │       │       │          │ buildte │
│          │        │          │          │       │       │          │ st/chec │
│          │        │          │          │       │       │          │ kouts/s │
│          │        │          │          │       │       │          │ table/t │
│          │        │          │          │       │       │          │ utorial │
│          │        │          │          │       │       │          │ s/vars. │
│          │        │          │          │       │       │          │ yml     │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
variables_bash/17e5a942: Creating Test Directory: /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
variables_bash/17e5a942 does not have any dependencies adding test to queue
 Builders Eligible to Run  
┏━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                 ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ variables_bash/17e5a942 │
└─────────────────────────┘
variables_bash/17e5a942: Current Working Directory : /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/stage
variables_bash/17e5a942: Running Test via command: bash variables_bash_build.sh
variables_bash/17e5a942: Test completed in 0.009769 seconds with returncode: 0
variables_bash/17e5a942: Writing output file -  /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.out
variables_bash/17e5a942: Writing error file - /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.err
                                  Test Summary                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                 ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ variables_bash/17e5a942 │ generic.local.bash │ PASS   │ 0          │ 0.010   │
└─────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /tmp/tmpjcb9dx2w/var/report.json
Writing Logfile to /tmp/tmpjcb9dx2w/var/logs/buildtest_uaulb51y.log

Buildtest Info (buildtest info)

The buildtest info command will provide user with general information pertaining to buildtest and its configuration, system details such as python wrapper, python version, machine name, operating system. It will also show the version of black, isort and pyflakes which are used when using buildtest stylecheck command

buildtest info
$ buildtest info
╭───────────────────────────── buildtest details ──────────────────────────────╮
│ Buildtest Version:        1.7                                                │
│ Buildtest Path:           /home/docs/checkouts/readthedocs.org/user_builds/b │
│ Configuration File:       /tmp/tmpjcb9dx2w/config.yml                        │
│ Available Systems:        ['generic']                                        │
│ Active System:            generic                                            │
│ Available Executors:      ['generic.local.bash', 'generic.local.sh', 'generi │
│ Build History Directory:  /tmp/tmpjcb9dx2w/var/.history                      │
│ Number of builds:         19                                                 │
│ Buildspec Cache File:     /tmp/tmpjcb9dx2w/var/buildspecs/cache.json         │
│ Default Report File:      /tmp/tmpjcb9dx2w/var/report.json                   │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────── system details ───────────────────────────────╮
│ Python Path:         /home/docs/checkouts/readthedocs.org/user_builds/buildt │
│ Python Version:      3.8.18                                                  │
│ Processor:           x86_64                                                  │
│ Host:                build-23444606-project-280831-buildtest                 │
│ Machine:             x86_64                                                  │
│ Operating System:    ubuntu                                                  │
│ Module System:       None                                                    │
╰──────────────────────────────────────────────────────────────────────────────╯
black: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/black
black version: black, 24.1.1 (compiled: yes)
Python (CPython) 3.8.18

pyflakes: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/pyflakes
pyflakes version: 3.2.0 Python 3.8.18 on Linux

isort: None
isort version:

Accessing buildtest documentation

We provide two command line options to access main documentation and schema docs. This will open a browser on your machine.

To access buildtest docs you can run:

buildtest docs

Enabling colored output for table entries

The buildtest --color <COLOR> <COMMAND> command can be used to select a color while printing output in a tabular format for several buildtest commands. The list of available colors can be found by running buildtest --helpcolor. Buildtest has tab-completion setup for --color option which will list all available colors.

Listing available color options (buildtest --helpcolor)

The buildtest --helpcolor command can be used to list the available color options in a tabular format which can be used with the –color option to select a color when printing table entries from several buildtest commands. This option will list all the colors printed in the background for the specified color

buildtest --helpcolor
$ buildtest --helpcolor
        Available Colors        
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Number ┃ Color               ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ 0      │ black               │
│ 1      │ red                 │
│ 2      │ green               │
│ 3      │ yellow              │
│ 4      │ blue                │
│ 5      │ magenta             │
│ 6      │ cyan                │
│ 7      │ white               │
│ 8      │ bright_black        │
│ 9      │ bright_red          │
│ 10     │ bright_green        │
│ 11     │ bright_yellow       │
│ 12     │ bright_blue         │
│ 13     │ bright_magenta      │
│ 14     │ bright_cyan         │
│ 15     │ bright_white        │
│ 16     │ grey0               │
│ 16     │ gray0               │
│ 17     │ navy_blue           │
│ 18     │ dark_blue           │
│ 20     │ blue3               │
│ 21     │ blue1               │
│ 22     │ dark_green          │
│ 25     │ deep_sky_blue4      │
│ 26     │ dodger_blue3        │
│ 27     │ dodger_blue2        │
│ 28     │ green4              │
│ 29     │ spring_green4       │
│ 30     │ turquoise4          │
│ 32     │ deep_sky_blue3      │
│ 33     │ dodger_blue1        │
│ 40     │ green3              │
│ 41     │ spring_green3       │
│ 36     │ dark_cyan           │
│ 37     │ light_sea_green     │
│ 38     │ deep_sky_blue2      │
│ 39     │ deep_sky_blue1      │
│ 47     │ spring_green2       │
│ 43     │ cyan3               │
│ 44     │ dark_turquoise      │
│ 45     │ turquoise2          │
│ 46     │ green1              │
│ 48     │ spring_green1       │
│ 49     │ medium_spring_green │
│ 50     │ cyan2               │
│ 51     │ cyan1               │
│ 88     │ dark_red            │
│ 125    │ deep_pink4          │
│ 55     │ purple4             │
│ 56     │ purple3             │
│ 57     │ blue_violet         │
│ 94     │ orange4             │
│ 59     │ grey37              │
│ 59     │ gray37              │
│ 60     │ medium_purple4      │
│ 62     │ slate_blue3         │
│ 63     │ royal_blue1         │
│ 64     │ chartreuse4         │
│ 71     │ dark_sea_green4     │
│ 66     │ pale_turquoise4     │
│ 67     │ steel_blue          │
│ 68     │ steel_blue3         │
│ 69     │ cornflower_blue     │
│ 76     │ chartreuse3         │
│ 73     │ cadet_blue          │
│ 74     │ sky_blue3           │
│ 81     │ steel_blue1         │
│ 114    │ pale_green3         │
│ 78     │ sea_green3          │
│ 79     │ aquamarine3         │
│ 80     │ medium_turquoise    │
│ 112    │ chartreuse2         │
│ 83     │ sea_green2          │
│ 85     │ sea_green1          │
│ 122    │ aquamarine1         │
│ 87     │ dark_slate_gray2    │
│ 91     │ dark_magenta        │
│ 128    │ dark_violet         │
│ 129    │ purple              │
│ 95     │ light_pink4         │
│ 96     │ plum4               │
│ 98     │ medium_purple3      │
│ 99     │ slate_blue1         │
│ 106    │ yellow4             │
│ 101    │ wheat4              │
│ 102    │ grey53              │
│ 102    │ gray53              │
│ 103    │ light_slate_grey    │
│ 103    │ light_slate_gray    │
│ 104    │ medium_purple       │
│ 105    │ light_slate_blue    │
│ 149    │ dark_olive_green3   │
│ 108    │ dark_sea_green      │
│ 110    │ light_sky_blue3     │
│ 111    │ sky_blue2           │
│ 150    │ dark_sea_green3     │
│ 116    │ dark_slate_gray3    │
│ 117    │ sky_blue1           │
│ 118    │ chartreuse1         │
│ 120    │ light_green         │
│ 156    │ pale_green1         │
│ 123    │ dark_slate_gray1    │
│ 160    │ red3                │
│ 126    │ medium_violet_red   │
│ 164    │ magenta3            │
│ 166    │ dark_orange3        │
│ 167    │ indian_red          │
│ 168    │ hot_pink3           │
│ 133    │ medium_orchid3      │
│ 134    │ medium_orchid       │
│ 140    │ medium_purple2      │
│ 136    │ dark_goldenrod      │
│ 173    │ light_salmon3       │
│ 138    │ rosy_brown          │
│ 139    │ grey63              │
│ 139    │ gray63              │
│ 141    │ medium_purple1      │
│ 178    │ gold3               │
│ 143    │ dark_khaki          │
│ 144    │ navajo_white3       │
│ 145    │ grey69              │
│ 145    │ gray69              │
│ 146    │ light_steel_blue3   │
│ 147    │ light_steel_blue    │
│ 184    │ yellow3             │
│ 157    │ dark_sea_green2     │
│ 152    │ light_cyan3         │
│ 153    │ light_sky_blue1     │
│ 154    │ green_yellow        │
│ 155    │ dark_olive_green2   │
│ 193    │ dark_sea_green1     │
│ 159    │ pale_turquoise1     │
│ 162    │ deep_pink3          │
│ 200    │ magenta2            │
│ 169    │ hot_pink2           │
│ 170    │ orchid              │
│ 207    │ medium_orchid1      │
│ 172    │ orange3             │
│ 174    │ light_pink3         │
│ 175    │ pink3               │
│ 176    │ plum3               │
│ 177    │ violet              │
│ 179    │ light_goldenrod3    │
│ 180    │ tan                 │
│ 181    │ misty_rose3         │
│ 182    │ thistle3            │
│ 183    │ plum2               │
│ 185    │ khaki3              │
│ 222    │ light_goldenrod2    │
│ 187    │ light_yellow3       │
│ 188    │ grey84              │
│ 188    │ gray84              │
│ 189    │ light_steel_blue1   │
│ 190    │ yellow2             │
│ 192    │ dark_olive_green1   │
│ 194    │ honeydew2           │
│ 195    │ light_cyan1         │
│ 196    │ red1                │
│ 197    │ deep_pink2          │
│ 199    │ deep_pink1          │
│ 201    │ magenta1            │
│ 202    │ orange_red1         │
│ 204    │ indian_red1         │
│ 206    │ hot_pink            │
│ 208    │ dark_orange         │
│ 209    │ salmon1             │
│ 210    │ light_coral         │
│ 211    │ pale_violet_red1    │
│ 212    │ orchid2             │
│ 213    │ orchid1             │
│ 214    │ orange1             │
│ 215    │ sandy_brown         │
│ 216    │ light_salmon1       │
│ 217    │ light_pink1         │
│ 218    │ pink1               │
│ 219    │ plum1               │
│ 220    │ gold1               │
│ 223    │ navajo_white1       │
│ 224    │ misty_rose1         │
│ 225    │ thistle1            │
│ 226    │ yellow1             │
│ 227    │ light_goldenrod1    │
│ 228    │ khaki1              │
│ 229    │ wheat1              │
│ 230    │ cornsilk1           │
│ 231    │ grey100             │
│ 231    │ gray100             │
│ 232    │ grey3               │
│ 232    │ gray3               │
│ 233    │ grey7               │
│ 233    │ gray7               │
│ 234    │ grey11              │
│ 234    │ gray11              │
│ 235    │ grey15              │
│ 235    │ gray15              │
│ 236    │ grey19              │
│ 236    │ gray19              │
│ 237    │ grey23              │
│ 237    │ gray23              │
│ 238    │ grey27              │
│ 238    │ gray27              │
│ 239    │ grey30              │
│ 239    │ gray30              │
│ 240    │ grey35              │
│ 240    │ gray35              │
│ 241    │ grey39              │
│ 241    │ gray39              │
│ 242    │ grey42              │
│ 242    │ gray42              │
│ 243    │ grey46              │
│ 243    │ gray46              │
│ 244    │ grey50              │
│ 244    │ gray50              │
│ 245    │ grey54              │
│ 245    │ gray54              │
│ 246    │ grey58              │
│ 246    │ gray58              │
│ 247    │ grey62              │
│ 247    │ gray62              │
│ 248    │ grey66              │
│ 248    │ gray66              │
│ 249    │ grey70              │
│ 249    │ gray70              │
│ 250    │ grey74              │
│ 250    │ gray74              │
│ 251    │ grey78              │
│ 251    │ gray78              │
│ 252    │ grey82              │
│ 252    │ gray82              │
│ 253    │ grey85              │
│ 253    │ gray85              │
│ 254    │ grey89              │
│ 254    │ gray89              │
│ 255    │ grey93              │
│ 255    │ gray93              │
└────────┴─────────────────────┘

Disabling Colored Output

buildtest will display output in color format using the rich.Console class. You can disable colored output via buildtest --no-color argument or set this persistent via environment variable BUILDTEST_COLOR=False.

CDASH Integration (buildtest cdash)

The buildtest cdash command is responsible for uploading tests to CDASH server. You will need to specify CDASH Configuration in your configuration file. Shown below is the command usage.

buildtest cdash --help
$ buildtest cdash --help
usage: buildtest [options] [COMMANDS] cdash [-h]  ...

optional arguments:
  -h, --help  show this help message and exit

subcommands:
  buildtest CDASH integration

  
    view      Open CDASH project in web browser
    upload    Upload test results to CDASH server

The buildtest cdash upload command is responsible for uploading all tests in report.json into CDASH. You must specify a buildname when using buildtest cdash upload in this example we will specify a buildname called tutorials:

$ buildtest cdash upload tutorials
Reading report file:  /Users/siddiq90/Documents/github/buildtest/var/report.json
Uploading 110 tests
Build Name:  tutorials
site:  generic
MD5SUM: d7651cb3fbdd19298b0188c441704c3a
You can view the results at: https://my.cdash.org//viewTest.php?buildid=2004360

We can see the output of these tests in CDASH if we go to url https://my.cdash.org//viewTest.php?buildid=2004360

_images/CDASH.png

By default buildtest will read the report file in your $HOME/.buildtest/report.json, we can specify an alternate report file. First let’s see the available help options for buildtest cdash upload.

buildtest cdash upload --help
$ buildtest cdash upload --help
usage: buildtest [options] [COMMANDS] cdash upload [-h] [--site SITE] [-o]
                                                   buildname

positional arguments:
  buildname    Specify Build Name reported in CDASH

optional arguments:
  -h, --help   show this help message and exit
  --site SITE  Specify site name reported in CDASH
  -o, --open   Open CDASH report in browser

We can pass an alternate report file using -r option when uploading tests to CDASH. This can be useful if you want to map test results to different buildnames in CDASH perhaps running a different subset of tests via buildtest build --tags and upload the test results with different buildname assuming you have different paths to report file.

Let’s say we want to build all python tests using tags and store them in a report file which we want to push to CDASH with buildgroup name python we can do that as follows

buildtest -r $BUILDTEST_ROOT/python.json build --tags python
$ buildtest -r $BUILDTEST_ROOT/python.json build --tags python
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-23444606-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2024/02/14 17:17:29                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.7                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.8.18                                                   │
│ Configuration File: /tmp/tmpjcb9dx2w/config.yml                              │
│ Test Directory:     /tmp/tmpjcb9dx2w/var/tests                               │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-shell.yml                                                   ║
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-hello.yml                                                   ║
╚══════════════════════════════════════════════════════════════════════════════╝
                            Buildspecs By Tag=python                            
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-shell.yml                                                   ║
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/ ║
║ tutorials/python-hello.yml                                                   ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  2
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  2
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 2
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-shell.yml: VALID
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/python-hello.yml: VALID
Total builder objects created: 2
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃          ┃        ┃          ┃          ┃       ┃       ┃ descript ┃ buildsp ┃
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion      ┃ ecs     ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ circle_a │ script │ generic. │ None     │ None  │ None  │ Calculat │ /home/d │
│ rea/4ca3 │        │ local.ba │          │       │       │ e circle │ ocs/che │
│ 510f     │        │ sh       │          │       │       │ of area  │ ckouts/ │
│          │        │          │          │       │       │ given a  │ readthe │
│          │        │          │          │       │       │ radius   │ docs.or │
│          │        │          │          │       │       │          │ g/user_ │
│          │        │          │          │       │       │          │ builds/ │
│          │        │          │          │       │       │          │ buildte │
│          │        │          │          │       │       │          │ st/chec │
│          │        │          │          │       │       │          │ kouts/s │
│          │        │          │          │       │       │          │ table/t │
│          │        │          │          │       │       │          │ utorial │
│          │        │          │          │       │       │          │ s/pytho │
│          │        │          │          │       │       │          │ n-shell │
│          │        │          │          │       │       │          │ .yml    │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ python_h │ script │ generic. │ None     │ None  │ None  │ Hello    │ /home/d │
│ ello/f89 │        │ local.ba │          │       │       │ World    │ ocs/che │
│ 6384a    │        │ sh       │          │       │       │ python   │ ckouts/ │
│          │        │          │          │       │       │          │ readthe │
│          │        │          │          │       │       │          │ docs.or │
│          │        │          │          │       │       │          │ g/user_ │
│          │        │          │          │       │       │          │ builds/ │
│          │        │          │          │       │       │          │ buildte │
│          │        │          │          │       │       │          │ st/chec │
│          │        │          │          │       │       │          │ kouts/s │
│          │        │          │          │       │       │          │ table/t │
│          │        │          │          │       │       │          │ utorial │
│          │        │          │          │       │       │          │ s/pytho │
│          │        │          │          │       │       │          │ n-hello │
│          │        │          │          │       │       │          │ .yml    │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
circle_area/4ca3510f: Creating Test Directory: /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-shell/circle_area/4ca3510f
python_hello/f896384a: Creating Test Directory: /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-hello/python_hello/f896384a
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
python_hello/f896384a does not have any dependencies adding test to queue
circle_area/4ca3510f does not have any dependencies adding test to queue
Builders Eligible to Run 
┏━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder               ┃
┡━━━━━━━━━━━━━━━━━━━━━━━┩
│ python_hello/f896384a │
│ circle_area/4ca3510f  │
└───────────────────────┘
python_hello/f896384a: Current Working Directory : /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-hello/python_hello/f896384a/stage
python_hello/f896384a: Running Test via command: bash python_hello_build.sh
python_hello/f896384a: Test completed in 0.031114 seconds with returncode: 0
python_hello/f896384a: Writing output file -  /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-hello/python_hello/f896384a/python_hello.out
python_hello/f896384a: Writing error file - /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-hello/python_hello/f896384a/python_hello.err
circle_area/4ca3510f: Current Working Directory : /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-shell/circle_area/4ca3510f/stage
circle_area/4ca3510f: Running Test via command: bash circle_area_build.sh
circle_area/4ca3510f: Test completed in 0.030839 seconds with returncode: 0
circle_area/4ca3510f: Writing output file -  /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-shell/circle_area/4ca3510f/circle_area.out
circle_area/4ca3510f: Writing error file - /tmp/tmpjcb9dx2w/var/tests/generic.local.bash/python-shell/circle_area/4ca3510f/circle_area.err
                                 Test Summary                                 
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder               ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ circle_area/4ca3510f  │ generic.local.bash │ PASS   │ 0          │ 0.031   │
├───────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ python_hello/f896384a │ generic.local.bash │ PASS   │ 0          │ 0.031   │
└───────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/python.json
Writing Logfile to /tmp/tmpjcb9dx2w/var/logs/buildtest_mopakq57.log

Next we upload the tests using the -r option to specify the report file

$ buildtest -r $BUILDTEST_ROOT/python.json cdash upload python
Reading report file:  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/python.json
Uploading 2 tests
Build Name:  python
site:  generic
MD5SUM: e8ef712716c9b68cde8e05c91dba33db
You can view the results at: https://my.cdash.org/viewTest.php?buildid=2489459

The buildtest cdash view command can be used to open CDASH project in a web browser using the command line. This feature assumes you have set the CDASH setting in your configuration file.

Cleaning buildtest files (buildtest clean)

The buildtest clean command can be used to remove files generated by buildtest such as test files, report files, buildspec cache, and history files. You will be prompted for response to clean up files for confirmation. If you want to avoid user response you can use buildtest clean -y to accept confirmation for all prompts and buildtest will remove the files.

$ buildtest clean
Remove Test Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests (y/n) [default: y]
Remove Report File /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json (y/n) [default: y]
Remove History Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/.history (y/n) [default: y]
Remove Buildspec Cache /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/buildspecs/cache.json (y/n) [default: y]
======> Remove Test Directory
======> Removing Report File
======> Removing History Directory
======> Removing buildspec cache

Changing Directories (buildtest cd)

The buildtest cd command can be used to change directory to root of test given a test name. The change will be applied to your shell upon completion of command. Let’s assume we want to change directory to root of test exit1_pass we can do this as follows:

$ buildtest cd exit1_pass
Changing directory to root of test: exit1_pass/8c4b6ac9-e94e-40d9-8d96-7aaa3a5d3723

$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9

In this previous example, buildtest will use the latest run for test exit1_pass and switch directory to root of test.

We can confirm this directory is from the latest run by running the following command. The testroot is a property in the report table that can be fetch via --format field. The --latest option will fetch the latest run for the test.

$ buildtest report --latest --filter name=exit1_pass --format testroot --terse --no-header
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9

If you switch cd into a particular build you can specify the name followed by backslash and name of test ID. In this example below, we will specify test name kernel_swapusage/1fa and buildtest will attempt to find first record that starts with the test ID and switch directory to root of test.

$ buildtest cd kernel_swapusage/1fa
Changing directory to root of test: kernel_swapusage/1fa21875-b099-41b6-8bc7-30e0d2dcc13b

$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/kernel_state/kernel_swapusage/1fa21875

Get Path for Test (buildtest path)

The buildtest path command is used to display path attributes for a test that is available in the test report. Shown below are available options for buildtest path

buildtest path --help
$ buildtest path --help
usage: buildtest [options] [COMMANDS] path [-h] [-be | -t | -o | -e | -b | -s]
                                           name

positional arguments:
  name               Name of test

optional arguments:
  -h, --help         show this help message and exit
  -be, --buildenv    Show path to build environment
  -t, --testpath     Show path to test script
  -o, --outfile      Show path to output file
  -e, --errfile      Show path to error file
  -b, --buildscript  Show path to build script
  -s, --stagedir     Show path to stage directory

If you want to fetch the last run for any given test you can specify the name of the test as follows: buildtest path <name>. We can specify a test ID for a test by separating the name and test ID with backslash character (/) as follows: buildtest path <name>/<ID>

If you don’t specify any option you will get root of test. In this example, we will retrieve testroot for test variables_bash which is a property of the test found in the report file.

$ buildtest path variables_bash
/tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942

You can get path to testscript via -t option as show below

$ buildtest path -t variables_bash
/tmp/tmpjcb9dx2w/var/tests/generic.local.bash/vars/variables_bash/17e5a942/variables_bash.sh

If you want to see content of output file, you can use -o option with cat command as follows:

$ cat $(buildtest path -o variables_bash)
1+2=3
this is a literal string
\'singlequote\'
"doublequote"
current user: docs
number of files: 4
Hello my name is Bob 
 I am 30 years old

In this next example we will query test circle_area with build ID aaa and buildtest will find the first match record that starts with this record and resolves to aaaa622d which is the short ID of test. In the second example we query the latest path for latest run for test circle_area

$ buildtest path circle_area/aaa
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/aaaa622d

$ buildtest path circle_area
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/fc221b84

We have setup mutual exclusion to avoid user from passing two option at same time. If you do run into this situation you will get the following error.

$ buildtest path -o -e variables_bash
usage: buildtest [options] [COMMANDS] path [-h] [-be | -t | -o | -e | -b | -s]
                                           name
buildtest [options] [COMMANDS] path: error: argument -e/--errfile: not allowed with argument -o/--outfile

If you specify an invalid test name or buildtest can’t find the test id, then buildtest will print list of available test names with IDs.

Test Statistics (buildtest stats)

The buildtest stats command can be used to get statistics for a particular test. The input argument is a positional argument which is name of test found in the report file. The output will show some useful details such as First and Last Run, show fastest and slowest runtime including mean and variance. Shown below is the test statistics for exit_fail.

$ buildtest stats python_hello
Total Test Runs:  3
First Run: 2022/06/13 15:29:11
Last Run: 2022/06/13 15:29:21
Fastest Runtime:  0.132854
Slowest Runtime:  0.161916
Mean Runtime 0.144621
Variance Runtime 0.000234
                                                                              Report File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name                                 ┃ state             ┃ returncode                      ┃ starttime                                              ┃ endtime                                                ┃ runtime                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python_hello                         │ PASS              │ 0                               │ 2022/06/13 15:29:11                                    │ 2022/06/13 15:29:11                                    │ 0.132854                 │
├──────────────────────────────────────┼───────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤
│ python_hello                         │ PASS              │ 0                               │ 2022/06/13 15:29:12                                    │ 2022/06/13 15:29:12                                    │ 0.139094                 │
├──────────────────────────────────────┼───────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤
│ python_hello                         │ PASS              │ 0                               │ 2022/06/13 15:29:21                                    │ 2022/06/13 15:29:21                                    │ 0.161916                 │
└──────────────────────────────────────┴───────────────────┴─────────────────────────────────┴────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────┴──────────────────────────┘

Buildtest Debug Report (buildtest debugreport)

The buildtest debugreport command is used for debugging especially when you want to raise an issue to buildtest project. This command will provide system details along with configuration file and output of log file during the report.

buildtest debugreport
$ buildtest debugreport
Host: build-23444606-project-280831-buildtest
OS: ubuntu
Python:  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3
Python Version:  3.8.18
BUILDTEST_VERSION:  1.7
───────────────────────── /tmp/tmpjcb9dx2w/config.yml ──────────────────────────
   1 system:
   2   generic:
   3     hostnames:
   4     - .*
   5     description: Generic System
   6     moduletool: none
   7     poolsize: 1
   8     pager: false
   9     buildspecs:
  10       rebuild: false
  11       count: 15
  12       terse: false
  13     report:
  14       count: 25
  15       terse: false
  16       format: name,id,state,runtime,returncode
  17     executors:
  18       local:
  19         bash:
  20           description: submit jobs on local machine using bash shell
  21           shell: bash
  22         sh:
  23           description: submit jobs on local machine using sh shell
  24           shell: sh
  25         csh:
  26           description: submit jobs on local machine using csh shell
  27           shell: csh
  28     compilers:
  29       compiler:
  30         gcc:
  31           builtin_gcc:
  32             cc: gcc
  33             fc: gfortran
  34             cxx: g++
  35     cdash:
  36       url: https://my.cdash.org/
  37       project: buildtest
  38       site: generic
  39       buildname: tutorials
  40     profiles:
  41       python:
  42         tags:
  43         - python
  44         testdir: /tmp/tmpjcb9dx2w/var/tests
  45         remove-stagedir: false
  46       prof1:
  47         tags:
  48         - python
  49         testdir: /tmp/tmpjcb9dx2w/var/tests
  50         remove-stagedir: false
  51       prof2:
  52         buildspecs:
  53         - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/tutorials/shell_examples.yml
  54         testdir: /tmp/tmpjcb9dx2w/var/tests
  55         remove-stagedir: false
  56

Example Configurations

buildtest provides a few example configurations for configuring buildtest this can be retrieved by running buildtest schema -n settings.schema.json --examples or short option (-e), which will validate each example with schema file settings.schema.json.

buildtest schema -n settings.schema.json -e
$ buildtest schema -n settings.schema.json -e
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: N/A                                                             
    executors:                                                                  
      local:                                                                    
        # the line below is invalid due to pattern property                     
        ==bash:                                                                 
          description: submit jobs on local machine                             
          shell: bash -v                                                        
      slurm:                                                                    
        # the line below is invalid due to pattern property                     
        $haswell:                                                               
          launcher: sbatch                                                      
          options: ["-p haswell", "-t 00:10"]                                   
      lsf:                                                                      
        <>batch:                                                                
          launcher: bsub                                                        
          queue: batch                                                          
          options: ["-q batch", "-t 00:10"]                                     
      cobalt:                                                                   
        <normal>:                                                               
          launcher: qsub                                                        
          queue: normal                                                         
          options: ["-n 1", "-t 10"]                                            
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: none                                                            
    poolsize: 1                                                                 
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
    executors:                                                                  
      defaults:                                                                 
        pollinterval: 10                                                        
        maxpendtime: 30                                                         
      local:                                                                    
        bash:                                                                   
          description: submit jobs via bash shell                               
          shell: bash                                                           
      pbs:                                                                      
        workq:                                                                  
          queue: workq                                                          
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    logdir: $BUILDTEST_ROOT/logs                                                
    testdir: $BUILDTEST_ROOT/tests                                              
    moduletool: none                                                            
    poolsize: 1                                                                 
    max_jobs: 1                                                                 
    pager: False                                                                
    cdash:                                                                      
      url: https://my.cdash.org                                                 
      project: buildtest                                                        
      site: laptop                                                              
    processor:                                                                  
      numcpus: 8                                                                
      cores: 4                                                                  
      threads_per_core: 2                                                       
      sockets: 1                                                                
      model: "Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz"                         
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
    executors:                                                                  
      local:                                                                    
        bash:                                                                   
          description: submit jobs on local machine using bash shell            
          shell: bash                                                           
          module:                                                               
            purge: True                                                         
            restore: "Default"                                                  
            load: ["gcc"]                                                       
        sh:                                                                     
          description: submit jobs on local machine using sh shell              
          shell: sh                                                             
          before_script: |                                                      
            date                                                                
            echo "these commands will be run"                                   
        csh:                                                                    
          description: submit jobs on local machine using csh shell             
          shell: csh -x                                                         
        tcsh:                                                                   
          description: submit jobs on local machine using tcsh shell            
          shell: /bin/tcsh                                                      
        zsh:                                                                    
          description: submit jobs on local machine using zsh shell             
          shell: /bin/zsh                                                       
                                                                                
    compilers:                                                                  
      find:                                                                     
        gcc: "^(gcc|GCC|PrgEnv-gnu)"                                            
        intel: "^(intel|Intel|PrgEnv-intel)"                                    
        cray: "^(cray|PrgEnv-cray)"                                             
        clang: "^(clang|Clang)"                                                 
        cuda: "^(cuda|CUDA)"                                                    
        pgi: "^(pgi|PGI|PrgEnv-pgi)"                                            
                                                                                
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
          gcc@7.2.0:                                                            
            cc: 'cc'                                                            
            cxx: 'cxx'                                                          
            fc: 'fc'                                                            
            module:                                                             
              load:                                                             
              - gcc/7.2.0                                                       
        intel:                                                                  
          intel@2019:                                                           
            cc: 'icc'                                                           
            cxx: 'icpc'                                                         
            fc: 'ifort'                                                         
            module:                                                             
              purge: True                                                       
              load:                                                             
              - gcc/7.2.0                                                       
              - intel/2019                                                      
        cray:                                                                   
          craype@2.6.2:                                                         
            cc: 'cc'                                                            
            cxx: 'CC'                                                           
            fc: 'fc'                                                            
            module:                                                             
              load: [craype/2.6.2]                                              
              swap: [PrgEnv-gnu, PrgEnv-cray]                                   
                                                                                
        clang:                                                                  
          clang@12.0.0:                                                         
            cc: 'clang'                                                         
            cxx: 'clang++'                                                      
            fc: 'None'                                                          
            module:                                                             
              load: [clang/12.0]                                                
        cuda:                                                                   
          cuda@11.0:                                                            
            cc: 'nvcc'                                                          
            cxx: 'nvcc'                                                         
            fc: 'None'                                                          
            module:                                                             
              load: [cuda/11.0]                                                 
        pgi:                                                                    
          pgi@18.0:                                                             
            cc: 'pgcc'                                                          
            cxx: 'pgc++'                                                        
            fc: 'pgfortran'                                                     
            module:                                                             
              load: [pgi/18.0]                                                  
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: none                                                            
    poolsize: 1                                                                 
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
    executors:                                                                  
      local:                                                                    
        bash:                                                                   
          description: submit jobs on local machine                             
          shell: bash -v                                                        
      slurm:                                                                    
        haswell:                                                                
          launcher: sbatch                                                      
          options: ["-p haswell", "-t 00:10"]                                   
      lsf:                                                                      
        batch:                                                                  
          launcher: bsub                                                        
          queue: batch                                                          
          options: ["-q batch", "-t 00:10"]                                     
      cobalt:                                                                   
        normal:                                                                 
          launcher: qsub                                                        
          queue: normal                                                         
          options: ["-n 1", "-t 10"]                                            
      container:                                                                
        ubuntu:                                                                 
          image: ubuntu:20.04                                                   
          platform: docker                                                      
          mounts: "/tmp:/tmp"                                                   
          options: "--rm"                                                       
          description: submit jobs on ubuntu container                          
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    poolsize: 1                                                                 
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
    executors:                                                                  
      defaults:                                                                 
        pollinterval: 10                                                        
        maxpendtime: 45                                                         
      local:                                                                    
        bash:                                                                   
          description: submit jobs via bash shell                               
          shell: bash                                                           
      lsf:                                                                      
        batch:                                                                  
          description: "LSF Executor name 'batch' that submits jobs to 'batch' q
          queue: batch                                                          
          account: developer                                                    
          options: ["-W 20"]                                                    
          before_script: |                                                      
            time                                                                
            echo "commands run before job"                                      
        test:                                                                   
          description: "LSF Executor name 'test' that submits jobs to 'test' que
          launcher: bsub                                                        
          queue: test                                                           
          account: qa                                                           
          options: ["-W 20"]                                                    
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    poolsize: 1                                                                 
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
    testdir: /tmp/buildtest                                                     
    executors:                                                                  
      defaults:                                                                 
        pollinterval: 20                                                        
        maxpendtime: 30                                                         
        account: admin                                                          
      local:                                                                    
        bash:                                                                   
          description: submit jobs via bash shell                               
          shell: bash                                                           
      slurm:                                                                    
        normal:                                                                 
          options: ["-C haswell"]                                               
          qos: normal                                                           
          before_script: |                                                      
            time                                                                
            echo "commands run before job"                                      
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    poolsize: 1                                                                 
    executors:                                                                  
      defaults:                                                                 
        maxpendtime: 30                                                         
      local:                                                                    
        bash:                                                                   
          description: submit jobs via bash shell                               
          shell: bash                                                           
      cobalt:                                                                   
        knl:                                                                    
          queue: knl                                                            
                                                                                
        haswell:                                                                
          queue: haswell                                                        
    buildspecs:                                                                 
      # whether to rebuild cache file automatically when running `buildtest buil
      rebuild: False                                                            
      # limit number of records to display when running `buildtest buildspec fin
      count: 15                                                                 
      # format fields to display when running `buildtest buildspec find`, By def
      format: "name,description"                                                
      # enable terse mode                                                       
      terse: False                                                              
                                                                                
    report:                                                                     
      count: 25                                                                 
      #enable terse mode for report                                             
      terse: False                                                              
      format: "name,id,state,runtime,returncode"                                
                                                                                
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran

If you want to retrieve full json schema file for buildtest configuration you can run buildtest schema -n settings.schema.json --json or short option -j.

Accessing Log File

Buildtest has several options to retrieve content of logfile, the --print-log or --view-log can be used to display output of the logfile. The --print-log will display output whereas --view-log will display in paginated mode.

The --logpath can be used to retrieve the path to logfile.

$ buildtest --logpath
/tmp/tmpjcb9dx2w/var/buildtest.log

Shown below is an example output of log content using the --print-log option

$ buildtest --print-log | head -n 10
2024-02-14 17:17:34,780  - [DEBUG] Starting System Compatibility Check
2024-02-14 17:17:34,781  - [INFO] Machine: x86_64
2024-02-14 17:17:34,781  - [INFO] Host: build-23444606-project-280831-buildtest
2024-02-14 17:17:34,781  - [INFO] User: docs
2024-02-14 17:17:34,781  - [INFO] Operating System: ubuntu
2024-02-14 17:17:34,781  - [INFO] System Kernel: Linux and Kernel Release: 5.19.0-1028-aws
2024-02-14 17:17:34,781  - [INFO] Python Path: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/stable/bin/python3
2024-02-14 17:17:34,781  - [INFO] Python Version: 3.8.18
2024-02-14 17:17:34,781  - [INFO] BUILDTEST_ROOT: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable
2024-02-14 17:17:34,781  - [INFO] Path to Buildtest: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/stable/bin/buildtest

Show All Options and Commands

The buildtest --help-all will display all commands and options available for buildtest. Some options are hidden by default when using buildtest --help. Please refer to the following output for list of available commands and options supported by buildtest.

$ buildtest --help-all
usage: buildtest [options] [COMMANDS]

buildtest is a HPC testing framework for building and running tests.

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
  -c CONFIGFILE, --configfile CONFIGFILE
                        Specify Path to Configuration File
  -d, --debug           Stream log messages to stdout
  -l {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        Filter log messages based on logging level
  --editor {vi,vim,emacs,nano}
                        Select your preferred editor when opening files.
  --view-log            Show content of last log
  --logpath             Print full path to last log file
  --print-log           Print content of last log without pagination
  --color COLOR         Print output of table with the selected color.
  --no-color            Disable colored output
  --helpcolor           Print available color options in a table format.
  -r REPORT, --report REPORT
                        Specify path to test report file
  -H, --help-all        List all commands and options

COMMANDS:
  
    build (bd)          Build and Run test
    buildspec (bc)      Buildspec Interface
    config (cg)         Query buildtest configuration
    report (rt)         Query test report
    inspect (it)        Inspect a test
    path (p)            Show path attributes for a given test
    history (hy)        Query build history
    schema              List schema contents and examples
    cdash               Upload test to CDASH server
    cd                  Change directory to root of test given a test name
    clean               Remove all generate files from buildtest including
                        test directory, log files, report file, buildspec
                        cache, history files
    debugreport (debug)
                        Display system information and additional information
                        for debugging purposes.
    stats               Show test statistics for given test
    info                Show details regarding current buildtest setup
    show                buildtest command guide
    commands (cmds)     List all buildtest commands
    tutorial-examples   Generate documentation examples for Buildtest Tutorial
    docs                Open buildtest docs in browser
    schemadocs          Open buildtest schema docs in browser
    unittests (test)    Run buildtest unit tests
    stylecheck (style)  Run buildtest style checks

    References

    GitHub:                  https://github.com/buildtesters/buildtest
    Documentation:           https://buildtest.readthedocs.io/en/latest/index.html
    Slack:                   http://hpcbuildtest.slack.com/

    Please report issues at https://github.com/buildtesters/buildtest/issues

    Copyright (c) 2021-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy), Shahzeb Siddiqui, and Vanessa Sochat. All rights reserved.