Additional Features

Accessing build history (buildtest history)

Note

buildtest hy is an alias for buildtest history command.

buildtest keeps track of all builds (buildtest build) that can be retrieved using buildtest history command which can be useful when you want to analyze or troubleshoot past builds. The buildtest history command comes with two subcommands buildtest history list and buildtest history query.

If you want to list all builds you should run buildtest history list which will report a table style format of all builds with corresponding build ID to differentiate each build. Shown below is an example output. The build IDs start at 0 and increment as you run buildtest build command.

$ buildtest history list
┏━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃    ┃         ┃      ┃         ┃        ┃ pass    ┃ fail   ┃ total   ┃ comman ┃
┃ id ┃ hostna… ┃ user ┃ system  ┃ date   ┃ tests   ┃ tests  ┃ tests   ┃ d      ┃
┡━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ 0  │ build-… │ docs │ generic │ 2022/… │ 1       │ 0      │ 1       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b /ho │
│    │         │      │         │        │         │        │         │ me/doc │
│    │         │      │         │        │         │        │         │ s/chec │
│    │         │      │         │        │         │        │         │ kouts/ │
│    │         │      │         │        │         │        │         │ readth │
│    │         │      │         │        │         │        │         │ edocs. │
│    │         │      │         │        │         │        │         │ org/us │
│    │         │      │         │        │         │        │         │ er_bui │
│    │         │      │         │        │         │        │         │ lds/bu │
│    │         │      │         │        │         │        │         │ ildtes │
│    │         │      │         │        │         │        │         │ t/chec │
│    │         │      │         │        │         │        │         │ kouts/ │
│    │         │      │         │        │         │        │         │ latest │
│    │         │      │         │        │         │        │         │ /tutor │
│    │         │      │         │        │         │        │         │ ials/v │
│    │         │      │         │        │         │        │         │ ars.ym │
│    │         │      │         │        │         │        │         │ l      │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 1  │ build-… │ docs │ generic │ 2022/… │ 2       │ 2      │ 4       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /pass_ │
│    │         │      │         │        │         │        │         │ return │
│    │         │      │         │        │         │        │         │ code.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 2  │ build-… │ docs │ generic │ 2022/… │ 1       │ 1      │ 2       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /statu │
│    │         │      │         │        │         │        │         │ s_rege │
│    │         │      │         │        │         │        │         │ x.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 3  │ build-… │ docs │ generic │ 2022/… │ 2       │ 3      │ 5       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /runti │
│    │         │      │         │        │         │        │         │ me_sta │
│    │         │      │         │        │         │        │         │ tus_te │
│    │         │      │         │        │         │        │         │ st.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 4  │ build-… │ docs │ generic │ 2022/… │ 2       │ 2      │ 4       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /expli │
│    │         │      │         │        │         │        │         │ cit_st │
│    │         │      │         │        │         │        │         │ ate.ym │
│    │         │      │         │        │         │        │         │ l      │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 5  │ build-… │ docs │ generic │ 2022/… │ 2       │ 0      │ 2       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /sheba │
│    │         │      │         │        │         │        │         │ ng.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 6  │ build-… │ docs │ generic │ 2022/… │ 1       │ 0      │ 1       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /skip_ │
│    │         │      │         │        │         │        │         │ tests. │
│    │         │      │         │        │         │        │         │ yml    │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 7  │ build-… │ docs │ generic │ 2022/… │ 0       │ 1      │ 1       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /metri │
│    │         │      │         │        │         │        │         │ cs_reg │
│    │         │      │         │        │         │        │         │ ex.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 8  │ build-… │ docs │ generic │ 2022/… │ 3       │ 0      │ 3       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /job_d │
│    │         │      │         │        │         │        │         │ epende │
│    │         │      │         │        │         │        │         │ ncy/ex │
│    │         │      │         │        │         │        │         │ 1.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 9  │ build-… │ docs │ generic │ 2022/… │ 3       │ 0      │ 3       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /job_d │
│    │         │      │         │        │         │        │         │ epende │
│    │         │      │         │        │         │        │         │ ncy/ex │
│    │         │      │         │        │         │        │         │ 2.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 10 │ build-… │ docs │ generic │ 2022/… │ 3       │ 1      │ 4       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /job_d │
│    │         │      │         │        │         │        │         │ epende │
│    │         │      │         │        │         │        │         │ ncy/ex │
│    │         │      │         │        │         │        │         │ 3.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 11 │ build-… │ docs │ generic │ 2022/… │ 2       │ 0      │ 2       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /job_d │
│    │         │      │         │        │         │        │         │ epende │
│    │         │      │         │        │         │        │         │ ncy/ex │
│    │         │      │         │        │         │        │         │ 4.yml  │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 12 │ build-… │ docs │ generic │ 2022/… │ 1       │ 1      │ 2       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /execu │
│    │         │      │         │        │         │        │         │ tor_re │
│    │         │      │         │        │         │        │         │ gex_sc │
│    │         │      │         │        │         │        │         │ ript.y │
│    │         │      │         │        │         │        │         │ ml     │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 13 │ build-… │ docs │ generic │ 2022/… │ 1       │ 1      │ 2       │ /home/ │
│    │         │      │         │ 18:20… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /scrip │
│    │         │      │         │        │         │        │         │ t/mult │
│    │         │      │         │        │         │        │         │ iple_e │
│    │         │      │         │        │         │        │         │ xecuto │
│    │         │      │         │        │         │        │         │ rs.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 14 │ build-… │ docs │ generic │ 2022/… │ 1       │ 1      │ 2       │ /home/ │
│    │         │      │         │ 18:21… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /scrip │
│    │         │      │         │        │         │        │         │ t/exec │
│    │         │      │         │        │         │        │         │ utor_s │
│    │         │      │         │        │         │        │         │ chedul │
│    │         │      │         │        │         │        │         │ er.yml │
├────┼─────────┼──────┼─────────┼────────┼─────────┼────────┼─────────┼────────┤
│ 15 │ build-… │ docs │ generic │ 2022/… │ 1       │ 1      │ 2       │ /home/ │
│    │         │      │         │ 18:21… │         │        │         │ docs/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/rea │
│    │         │      │         │        │         │        │         │ dthedo │
│    │         │      │         │        │         │        │         │ cs.org │
│    │         │      │         │        │         │        │         │ /user_ │
│    │         │      │         │        │         │        │         │ builds │
│    │         │      │         │        │         │        │         │ /build │
│    │         │      │         │        │         │        │         │ test/c │
│    │         │      │         │        │         │        │         │ heckou │
│    │         │      │         │        │         │        │         │ ts/lat │
│    │         │      │         │        │         │        │         │ est/bi │
│    │         │      │         │        │         │        │         │ n/buil │
│    │         │      │         │        │         │        │         │ dtest  │
│    │         │      │         │        │         │        │         │ build  │
│    │         │      │         │        │         │        │         │ -b tut │
│    │         │      │         │        │         │        │         │ orials │
│    │         │      │         │        │         │        │         │ /scrip │
│    │         │      │         │        │         │        │         │ t/stat │
│    │         │      │         │        │         │        │         │ us_by_ │
│    │         │      │         │        │         │        │         │ execut │
│    │         │      │         │        │         │        │         │ ors.ym │
│    │         │      │         │        │         │        │         │ l      │
└────┴─────────┴──────┴─────────┴────────┴─────────┴────────┴─────────┴────────┘

The buildtest history query command is particularly useful when you want to inspect a particular build. This command expects a Build Identifier which can be found by inspecting output column id in buildtest history list.

Shown below is an output of build ID 0 which reports relevant detail for the build such as input command, username, hostname, platform, date, etc…

$ buildtest history query 0
{
│   'command': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/bin/buildtest build -b /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tutorials/vars.yml',
│   'user': 'docs',
│   'hostname': 'build-16516694-project-280831-buildtest',
│   'platform': 'Linux',
│   'date': '2022/03/30 18:20:28',
│   'buildtest': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/bin/buildtest',
│   'python': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/latest/bin/python3',
│   'python_version': '3.7.12',
│   'testdir': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests',
│   'configuration': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/buildtest/settings/config.yml',
│   'system': 'generic',
│   'logpath': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/.history/0/buildtest_f8tg0m9q.log',
│   'invalid_buildspecs': [],
│   'buildspecs': {
│   │   'detected': [
│   │   │   '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tutorials/vars.yml'
│   │   ],
│   │   'included': [
│   │   │   '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tutorials/vars.yml'
│   │   ],
│   │   'excluded': []
│   },
│   'test_summary': {
│   │   'pass': '1',
│   │   'fail': '0',
│   │   'total': '1',
│   │   'pass_rate': '100.000',
│   │   'fail_rate': '0.000'
│   },
│   'builders': {
│   │   '59213d24-963d-4957-94ba-30b531befb91': {
│   │   │   'name': 'variables_bash',
│   │   │   'buildspec': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tutorials/vars.yml',
│   │   │   'tags': ['tutorials'],
│   │   │   'executors': 'generic.local.bash',
│   │   │   'state': 'PASS',
│   │   │   'returncode': 0,
│   │   │   'runtime': 0.014886,
│   │   │   'testpath': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/variables_bash/59213d24/variables_bash.sh',
│   │   │   'errfile': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/variables_bash/59213d24/variables_bash.err',
│   │   │   'outfile': '/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/variables_bash/59213d24/variables_bash.out'
│   │   }
│   }
}

If you want to see all available build IDs, you can use the following command. The -t is terse format and --no-header will omit the headers for each column and pipe the output to cut to extract the first column which corresponds to build IDs.

$ buildtest hy list -t --no-header | cut -f 1 -d '|'
0

st/bin/buildtest build -b /home/docs/checkouts/readthedocs.org/user_builds/build
test/checkouts/latest/tutorials/vars.yml
1
50.000
st/bin/buildtest build -b tutorials/pass_returncode.yml
2
50.000
st/bin/buildtest build -b tutorials/status_regex.yml
3
60.000
st/bin/buildtest build -b tutorials/runtime_status_test.yml
4
50.000
st/bin/buildtest build -b tutorials/explicit_state.yml
5

st/bin/buildtest build -b tutorials/shebang.yml
6

st/bin/buildtest build -b tutorials/skip_tests.yml
7
00.000
st/bin/buildtest build -b tutorials/metrics_regex.yml
8

st/bin/buildtest build -b tutorials/job_dependency/ex1.yml
9

st/bin/buildtest build -b tutorials/job_dependency/ex2.yml
10

est/bin/buildtest build -b tutorials/job_dependency/ex3.yml
11
0
est/bin/buildtest build -b tutorials/job_dependency/ex4.yml
12

est/bin/buildtest build -b tutorials/executor_regex_script.yml
13

est/bin/buildtest build -b tutorials/script/multiple_executors.yml
14

est/bin/buildtest build -b tutorials/script/executor_scheduler.yml
15

est/bin/buildtest build -b tutorials/script/status_by_executors.yml

buildtest has tab complete on buildtest history query which reports a list of build IDs which is another way to see available IDs to query.

$ buildtest history query
0   1   10  11  12  13  14  15  16  17  18  19  2   20  21  22  23  3   4   5   6   7   8   9

If you want to see logfile for build ID 0 you can use --log option to see logfile in an editor as follows:

buildtest history query 0 --log

buildtest will store output of buildtest build command to file if command ran to completion. You can retrieve output of previous builds using the --output option. In example below we show output of build file

$ buildtest history query --output 0
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-16516694-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2022/03/30 18:20:27                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/bin/buildtest                                            │
│ buildtest version:  0.14.0                                                   │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/envs/latest/bin/python3                                                   │
│ python version:     3.7.12                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/buildtest/settings/config.yml                            │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/var/tests                                                │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/var/report.json                                          │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/bin/buildtest build -b /home/docs/checkouts/readthedocs. │
│ org/user_builds/buildtest/checkouts/latest/tutorials/vars.yml                │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
Discovered Buildspecs:  1
Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ Buildspecs                                                                   ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ║
╚══════════════════════════════════════════════════════════════════════════════╝
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tuto
rials/vars.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 1
Total spack builder: 0
                             Script Builder Details                             
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Builder        ┃ Executor       ┃ Compiler ┃ description    ┃ buildspecs     ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ variables_bash │ generic.local. │ None     │ Declare shell  │ /home/docs/che │
│ /59213d24      │ bash           │          │ variables in   │ ckouts/readthe │
│                │                │          │ bash           │ docs.org/user_ │
│                │                │          │                │ builds/buildte │
│                │                │          │                │ st/checkouts/l │
│                │                │          │                │ atest/tutorial │
│                │                │          │                │ s/vars.yml     │
└────────────────┴────────────────┴──────────┴────────────────┴────────────────┘
──────────────────────────────── Building Test ─────────────────────────────────
variables_bash/59213d24: Creating test directory: /home/docs/checkouts/readthedo
cs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/
variables_bash/59213d24
variables_bash/59213d24: Creating the stage directory: /home/docs/checkouts/read
thedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/
vars/variables_bash/59213d24/stage
variables_bash/59213d24: Writing build script: /home/docs/checkouts/readthedocs.
org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/var
iables_bash/59213d24/variables_bash_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 2 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
variables_bash/59213d24 does not have any dependencies adding test to queue
variables_bash/59213d24: Running Test via command: bash --norc --noprofile -eo 
pipefail variables_bash_build.sh
variables_bash/59213d24: Test completed in 0.014886 seconds
variables_bash/59213d24: Test completed with returncode: 0
variables_bash/59213d24: Writing output file -  /home/docs/checkouts/readthedocs
.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/va
riables_bash/59213d24/variables_bash.out
variables_bash/59213d24: Writing error file - /home/docs/checkouts/readthedocs.o
rg/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/vari
ables_bash/59213d24/variables_bash.err
In this iteration we are going to run the following tests: [variables_bash/59213d24]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ Checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ Builder      ┃ executor     ┃ status ┃ Runtime)      ┃ ReturnCode ┃ Runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ variables_ba │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.014886 │
│ sh/59213d24  │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildt
est/checkouts/latest/var/report.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/logs/buildtest_f8tg0m9q.log

Accessing buildtest documentation

We provide two command line options to access main documentation and schema docs. This will open a browser on your machine.

To access buildtest docs you can run:

buildtest docs

To access schema docs you can run:

buildtest schemadocs

Disabling Colored Output

buildtest will display output in color format using the rich.Console class. You can disable colored output via buildtest --no-color argument or set this persistent via environment variable BUILDTEST_COLOR=False.

CDASH Integration (buildtest cdash)

The buildtest cdash command is responsible for uploading tests to CDASH server. You will need to specify Configuring test directory in your configuration file. Shown below is the command usage.

$ buildtest cdash --help
usage: buildtest [options] [COMMANDS] cdash [-h]  ...

optional arguments:
  -h, --help  show this help message and exit

subcommands:
  buildtest CDASH integeration

  
    view      Open CDASH project in webbrowser
    upload    Upload Test to CDASH server

The buildtest cdash upload command is responsible for uploading all tests in report.json into CDASH. You must specify a buildname when using buildtest cdash upload in this example we will specify a buildname called tutorials:

$ buildtest cdash upload tutorials
Reading configuration file:  /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest/settings/config.yml
Reading report file:  /Users/siddiq90/.buildtest/report.json
build name:  tutorials
site:  generic
stamp:  20210428-1512-Experimental
MD5SUM: d7651cb3fbdd19298b0188c441704c3a
PUT STATUS: 200
You can view the results at: https://my.cdash.org//viewTest.php?buildid=2004360

We can see the output of these tests in CDASH if we go to url https://my.cdash.org//viewTest.php?buildid=2004360

_images/CDASH.png

By default buildtest will read the report file in your $HOME/.buildtest/report.json, we can specify an alternate report file. First let’s see the available help options for buildtest cdash upload.

$ buildtest cdash upload --help
usage: buildtest [options] [COMMANDS] cdash upload [-h] [--site SITE]
                                                   buildname

positional arguments:
  buildname    Specify Build Name reported in CDASH

optional arguments:
  -h, --help   show this help message and exit
  --site SITE  Specify site name reported in CDASH

We can pass an alternate report file using -r option when uploading tests to CDASH. This can be useful if you want to map test results to different buildnames in CDASH perhaps running a different subset of tests via buildtest build --tags and upload the test results with different buildname assuming you have different paths to report file.

Let’s say we want to build all python tests using tags and store them in a report file which we want to push to CDASH with buildgroup name python we can do that as follows

$ buildtest -r $BUILDTEST_ROOT/python.json build --tags python
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-16516694-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2022/03/30 18:21:10                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/bin/buildtest                                            │
│ buildtest version:  0.14.0                                                   │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/envs/latest/bin/python3                                                   │
│ python version:     3.7.12                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/buildtest/settings/config.yml                            │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/var/tests                                                │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/python.json                                              │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ st/checkouts/latest/bin/buildtest -r /home/docs/checkouts/readthedocs.org/us │
│ er_builds/buildtest/checkouts/latest/python.json build --tags python         │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
Discovered Buildspecs:  2
Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  2
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ Buildspecs                                                                   ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ║
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ║
╚══════════════════════════════════════════════════════════════════════════════╝
                            Buildspecs By Tag=python                            
╔══════════════════════════════════════════════════════════════════════════════╗
║ Buildspecs                                                                   ║
╟──────────────────────────────────────────────────────────────────────────────╢
║  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/lates… ║
║  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/lates… ║
╚══════════════════════════════════════════════════════════════════════════════╝
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 2
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tuto
rials/python-hello.yml: VALID
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/tuto
rials/python-shell.yml: VALID
Total builder objects created: 2
Total compiler builder: 0
Total script builder: 2
Total spack builder: 0
                             Script Builder Details                             
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Builder        ┃ Executor       ┃ Compiler ┃ description    ┃ buildspecs     ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ python_hello/1 │ generic.local. │ None     │ Hello World    │ /home/docs/che │
│ 9d41dcc        │ bash           │          │ python         │ ckouts/readthe │
│                │                │          │                │ docs.org/user_ │
│                │                │          │                │ builds/buildte │
│                │                │          │                │ st/checkouts/l │
│                │                │          │                │ atest/tutorial │
│                │                │          │                │ s/python-hello │
│                │                │          │                │ .yml           │
├────────────────┼────────────────┼──────────┼────────────────┼────────────────┤
│ circle_area/d0 │ generic.local. │ None     │ Calculate      │ /home/docs/che │
│ 465649         │ bash           │          │ circle of area │ ckouts/readthe │
│                │                │          │ given a radius │ docs.org/user_ │
│                │                │          │                │ builds/buildte │
│                │                │          │                │ st/checkouts/l │
│                │                │          │                │ atest/tutorial │
│                │                │          │                │ s/python-shell │
│                │                │          │                │ .yml           │
└────────────────┴────────────────┴──────────┴────────────────┴────────────────┘
──────────────────────────────── Building Test ─────────────────────────────────
python_hello/19d41dcc: Creating test directory: /home/docs/checkouts/readthedocs
.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-
hello/python_hello/19d41dcc
python_hello/19d41dcc: Creating the stage directory: /home/docs/checkouts/readth
edocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/py
thon-hello/python_hello/19d41dcc/stage
python_hello/19d41dcc: Writing build script: /home/docs/checkouts/readthedocs.or
g/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-hel
lo/python_hello/19d41dcc/python_hello_build.sh
circle_area/d0465649: Creating test directory: /home/docs/checkouts/readthedocs.
org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-s
hell/circle_area/d0465649
circle_area/d0465649: Creating the stage directory: /home/docs/checkouts/readthe
docs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/pyt
hon-shell/circle_area/d0465649/stage
circle_area/d0465649: Writing build script: /home/docs/checkouts/readthedocs.org
/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-shel
l/circle_area/d0465649/circle_area_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 2 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
python_hello/19d41dcc does not have any dependencies adding test to queue
circle_area/d0465649 does not have any dependencies adding test to queue
circle_area/d0465649: Running Test via command: bash --norc --noprofile -eo 
pipefail circle_area_build.sh
python_hello/19d41dcc: Running Test via command: bash --norc --noprofile -eo 
pipefail python_hello_build.sh
circle_area/d0465649: Test completed in 0.061391 seconds
circle_area/d0465649: Test completed with returncode: 0
circle_area/d0465649: Writing output file -  /home/docs/checkouts/readthedocs.or
g/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-she
ll/circle_area/d0465649/circle_area.out
circle_area/d0465649: Writing error file - /home/docs/checkouts/readthedocs.org/
user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-shell
/circle_area/d0465649/circle_area.err
python_hello/19d41dcc: Test completed in 0.086398 seconds
python_hello/19d41dcc: Test completed with returncode: 0
python_hello/19d41dcc: Writing output file -  /home/docs/checkouts/readthedocs.o
rg/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-he
llo/python_hello/19d41dcc/python_hello.out
python_hello/19d41dcc: Writing error file - /home/docs/checkouts/readthedocs.org
/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/python-hell
o/python_hello/19d41dcc/python_hello.err
In this iteration we are going to run the following tests: [python_hello/19d41dcc, circle_area/d0465649]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ Checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ Builder      ┃ executor     ┃ status ┃ Runtime)      ┃ ReturnCode ┃ Runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ circle_area/ │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.061391 │
│ d0465649     │              │        │               │            │          │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ python_hello │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.086398 │
│ /19d41dcc    │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildt
est/checkouts/latest/python.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/logs/buildtest_nwznfcnn.log

Next we upload the tests using the -r option to specify the report file

$ buildtest -r $BUILDTEST_ROOT/python.json cdash upload python
Reading report file:  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/python.json
build name:  python
site:  generic
stamp:  20220330-1821-Experimental
MD5SUM: 43489904d99255d998e5266e39b5f4eb
PUT STATUS: 200
You can view the results at: https://my.cdash.org//viewTest.php?buildid=2147991

The buildtest cdash view command can be used to open CDASH project in a web browser using the command line. This feature assumes you have set the CDASH setting in your configuration file.

Cleaning buildtest files (buildtest clean)

The buildtest clean command can be used to remove files generated by buildtest such as test files, report files, buildspec cache, and history files. You will be prompted for response to clean up files for confirmation. If you want to avoid user response you can use buildtest clean -y to accept confirmation for all prompts and buildtest will remove the files.

$ buildtest clean
Remove Test Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests (y/n) [default: y]
Remove Report File /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json (y/n) [default: y]
Remove History Directory /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/.history (y/n) [default: y]
Remove Buildspec Cache /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/buildspecs/cache.json (y/n) [default: y]
======> Remove Test Directory
======> Removing Report File
======> Removing History Directory
======> Removing buildspec cache

Changing Directories (buildtest cd)

The buildtest cd command can be used to change directory to root of test given a test name. The change will be applied to your shell upon completion of command. Let’s assume we want to change directory to root of test exit1_pass we can do this as follows:

$ buildtest cd exit1_pass
Changing directory to root of test: exit1_pass/8c4b6ac9-e94e-40d9-8d96-7aaa3a5d3723

$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9

In this previous example, buildtest will use the latest run for test exit1_pass and switch directory to root of test.

We can confirm this directory is from the latest run by running the following command. The testroot is a property in the report table that can be fetch via --format field. The --latest option will fetch the latest run for the test.

$ buildtest report --latest --filter name=exit1_pass --format testroot --terse --no-header
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.sh/pass_returncode/exit1_pass/8c4b6ac9

If you switch cd into a particular build you can specify the name followed by backslash and name of test ID. In this example below, we will specify test name kernel_swapusage/1fa and buildtest will attempt to find first record that starts with the test ID and switch directory to root of test.

$ buildtest cd kernel_swapusage/1fa
Changing directory to root of test: kernel_swapusage/1fa21875-b099-41b6-8bc7-30e0d2dcc13b

$ pwd
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/kernel_state/kernel_swapusage/1fa21875

Get Path for Test (buildtest path)

The buildtest path command is used to display path attributes for a test that is available in the test report. Shown below are available options for buildtest path

$ buildtest path -h
usage: buildtest [options] [COMMANDS] path [-h] [-t | -o | -e | -b | -s] name

positional arguments:
  name               Name of test

optional arguments:
  -h, --help         show this help message and exit
  -t, --testpath     Show path to test script
  -o, --outfile      Show path to output file
  -e, --errfile      Show path to error file
  -b, --buildscript  Show path to build script
  -s, --stagedir     Show path to stage directory

If you want to fetch the last run for any given test you can specify the name of the test as follows: buildtest path <name>. We can specify a test ID for a test by separating the name and test ID with backslash character (/) as follows: buildtest path <name>/<ID>

If you don’t specify any option you will get root of test. In this example, we will retrieve testroot for test variables_bash which is a property of the test found in the report file.

$ buildtest path variables_bash
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/variables_bash/59213d24

You can get path to testscript via -t option as show below

$ buildtest path -t variables_bash
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/var/tests/generic.local.bash/vars/variables_bash/59213d24/variables_bash.sh

If you want to see content of output file, you can use -o option with cat command as follows:

$ cat $(buildtest path -o variables_bash)
1+2=3
this is a literal string
\'singlequote\'
"doublequote"
current user: docs
number of files: 4
Hello my name is Bob 
 I am 30 years old

In this next example we will query test circle_area with build ID aaa and buildtest will find the first match record that starts with this record and resolves to aaaa622d which is the short ID of test. In the second example we query the latest path for latest run for test circle_area

$ buildtest path circle_area/aaa
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/aaaa622d

$ buildtest path circle_area
/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.python/python-shell/circle_area/fc221b84

We have setup mutual exclusion to avoid user from passing two option at same time. If you do run into this situation you will get the following error.

$ buildtest path -o -e variables_bash
usage: buildtest [options] [COMMANDS] path [-h] [-t | -o | -e | -b | -s] name
buildtest [options] [COMMANDS] path: error: argument -e/--errfile: not allowed with argument -o/--outfile

If you specify an invalid test name or buildtest can’t find the test id, then buildtest will print list of available test names with IDs.

Buildtest Debug Report (buildtest debugreport)

The buildtest debugreport command is used for debugging especially when you want to raise an issue to buildtest project. This command will provide system details along with configuration file and output of log file during the report.

$ buildtest debugreport
Host: build-16516694-project-280831-buildtest
OS: ubuntu
Python:  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/envs/latest/
bin/python3
Python Version:  3.7.12
BUILDTEST_VERSION:  0.14.0
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
   1 system:                                                                    
   2   generic:                                                                 
   3     # specify list of hostnames where buildtest can run for given system re
   4     hostnames: [".*"]                                                      
   5                                                                            
   6     # system description                                                   
   7     description: Generic System                                            
   8     # specify module system used at your site (environment-modules, lmod)  
   9     moduletool: N/A                                                        
  10                                                                            
  11     executors:                                                             
  12       # define local executors for running jobs locally                    
  13       local:                                                               
  14         bash:                                                              
  15           description: submit jobs on local machine using bash shell       
  16           shell: bash                                                      
  17         sh:                                                                
  18           description: submit jobs on local machine using sh shell         
  19           shell: sh                                                        
  20         csh:                                                               
  21           description: submit jobs on local machine using csh shell        
  22           shell: csh                                                       
  23         zsh:                                                               
  24           description: submit jobs on local machine using zsh shell        
  25           shell: zsh                                                       
  26     # compiler block                                                       
  27     compilers:                                                             
  28       # regular expression to search for compilers based on module pattern.
  29       # find:                                                              
  30       #  gcc: "^(gcc)"                                                     
  31       #  intel: "^(intel)"                                                 
  32       #  cray: "^(craype)"                                                 
  33       #  pgi: "^(pgi)"                                                     
  34       #  cuda: ^(cuda)"                                                    
  35       #  clang: "^(clang)"                                                 
  36                                                                            
  37       # declare compiler instance which can be site-specific. You can let '
  38       compiler:                                                            
  39         gcc:                                                               
  40           builtin_gcc:                                                     
  41             cc: gcc                                                        
  42             fc: gfortran                                                   
  43             cxx: g++                                                       
  44                                                                            
  45     # location of log directory                                            
  46     # logdir: /tmp/                                                        
  47                                                                            
  48     # specify location where buildtest will write tests                    
  49     # testdir: /tmp                                                        
  50                                                                            
  51     # specify one or more directory where buildtest should load buildspecs 
  52     # buildspec_roots: []                                                  
  53                                                                            
  54     cdash:                                                                 
  55       url: https://my.cdash.org/                                           
  56       project: buildtest                                                   
  57       site: generic                                                        
  58       buildname: tutorials                                                 
  59                                                                            
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
   1 2022-03-30 18:21:14,335 [system.py:42 - check() ] - [DEBUG] Starting System
   2 2022-03-30 18:21:14,337 [system.py:67 - check() ] - [DEBUG] Machine: x86_64
   3 2022-03-30 18:21:14,337 [system.py:68 - check() ] - [DEBUG] Host: build-165
   4 2022-03-30 18:21:14,337 [system.py:69 - check() ] - [DEBUG] User: docs     
   5 2022-03-30 18:21:14,337 [system.py:70 - check() ] - [DEBUG] Operating Syste
   6 2022-03-30 18:21:14,337 [system.py:72 - check() ] - [DEBUG] System Kernel: 
   7 2022-03-30 18:21:14,337 [system.py:74 - check() ] - [DEBUG] Python Path: /h
   8 2022-03-30 18:21:14,337 [system.py:75 - check() ] - [DEBUG] Python Version:
   9 2022-03-30 18:21:14,337 [system.py:76 - check() ] - [DEBUG] BUILDTEST_ROOT:
  10 2022-03-30 18:21:14,337 [system.py:77 - check() ] - [DEBUG] Path to Buildte
  11 2022-03-30 18:21:14,337 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  12 2022-03-30 18:21:14,337 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  13 2022-03-30 18:21:14,337 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  14 2022-03-30 18:21:14,337 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  15 2022-03-30 18:21:14,337 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  16 2022-03-30 18:21:14,337 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  17 2022-03-30 18:21:14,337 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  18 2022-03-30 18:21:14,337 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  19 2022-03-30 18:21:14,337 [system.py:82 - check() ] - [DEBUG] Finished System
  20 2022-03-30 18:21:14,347 [config.py:129 - validate() ] - [DEBUG] Loading def
  21 2022-03-30 18:21:14,347 [utils.py:40 - load_schema() ] - [DEBUG] Successful
  22 2022-03-30 18:21:14,347 [config.py:133 - validate() ] - [DEBUG] Validating 
  23 2022-03-30 18:21:14,352 [config.py:136 - validate() ] - [DEBUG] Validation 
  24 2022-03-30 18:21:14,353 [system.py:42 - check() ] - [DEBUG] Starting System
  25 2022-03-30 18:21:14,353 [system.py:67 - check() ] - [DEBUG] Machine: x86_64
  26 2022-03-30 18:21:14,353 [system.py:68 - check() ] - [DEBUG] Host: build-165
  27 2022-03-30 18:21:14,353 [system.py:69 - check() ] - [DEBUG] User: docs     
  28 2022-03-30 18:21:14,353 [system.py:70 - check() ] - [DEBUG] Operating Syste
  29 2022-03-30 18:21:14,353 [system.py:72 - check() ] - [DEBUG] System Kernel: 
  30 2022-03-30 18:21:14,353 [system.py:74 - check() ] - [DEBUG] Python Path: /h
  31 2022-03-30 18:21:14,353 [system.py:75 - check() ] - [DEBUG] Python Version:
  32 2022-03-30 18:21:14,353 [system.py:76 - check() ] - [DEBUG] BUILDTEST_ROOT:
  33 2022-03-30 18:21:14,353 [system.py:77 - check() ] - [DEBUG] Path to Buildte
  34 2022-03-30 18:21:14,353 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  35 2022-03-30 18:21:14,353 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  36 2022-03-30 18:21:14,353 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  37 2022-03-30 18:21:14,354 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  38 2022-03-30 18:21:14,354 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  39 2022-03-30 18:21:14,354 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  40 2022-03-30 18:21:14,354 [system.py:152 - check_binaries() ] - [DEBUG] We wi
  41 2022-03-30 18:21:14,354 [system.py:156 - check_binaries() ] - [DEBUG] Canno
  42 2022-03-30 18:21:14,354 [system.py:82 - check() ] - [DEBUG] Finished System
  43 2022-03-30 18:21:14,354 [main.py:101 -  main() ] - [INFO] [red]Processing b
  44

Command Line Interface to buildtest configuration

Once you have implemented your buildtest configuration, you can query the configuration details using buildtest config command. Shown below is the command usage.

$ buildtest config --help
usage: buildtest [options] [COMMANDS] config [-h]  ...

optional arguments:
  -h, --help  show this help message and exit

subcommands:
  Query information from buildtest configuration file

  
    compilers
              Search compilers
    edit      Open configuration file in editor
    executors
              Query executors from buildtest configuration
    systems   List all available systems
    validate  Validate buildtest settings file with schema.
    view      View configuration file

Note

buildtest cg is an alias for buildtest config command.

Validate buildtest configuration (buildtest config validate)

First thing you should do once you implement your configuration file is to make sure your configuration is valid with the schema. This can be achieved by running buildtest config validate. When you invoke this command, buildtest will load the configuration and attempt to validate the file with schema settings.schema.json. If validation is successful you will get the following message:

$ buildtest config validate
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest/buildtest/settings/config.yml is valid

Note

If you defined a user setting (~/.buildtest/config.yml) buildtest will validate this file instead of default one.

If there is an error during validation, the output from jsonschema.exceptions.ValidationError will be displayed in terminal. For example the error below indicates that moduletool property was expecting one of the values [environment-modules, lmod, N/A] but it recieved a value of none:

$ buildtest config validate
Traceback (most recent call last):
  File "/Users/siddiq90/Documents/buildtest/bin/buildtest", line 17, in <module>
    buildtest.main.main()
  File "/Users/siddiq90/Documents/buildtest/buildtest/main.py", line 39, in main
    buildtest_configuration = check_settings(settings_file, retrieve_settings=True)
  File "/Users/siddiq90/Documents/buildtest/buildtest/config.py", line 41, in check_settings
    validate(instance=user_schema, schema=config_schema)
  File "/Users/siddiq90/.local/share/virtualenvs/buildtest-1gHVG2Pd/lib/python3.7/site-packages/jsonschema/validators.py", line 934, in validate
    raise error
jsonschema.exceptions.ValidationError: 'none' is not one of ['environment-modules', 'lmod', 'N/A']

Failed validating 'enum' in schema['properties']['moduletool']:
    {'description': 'Specify modules tool used for interacting with '
                    '``module`` command. ',
     'enum': ['environment-modules', 'lmod', 'N/A'],
     'type': 'string'}

On instance['moduletool']:
    'none'

View buildtest configuration (buildtest config view)

If you want to view buildtest configuration you can run buildtest config view which will print content of buildtest configuration.

$ buildtest config view
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
   1 system:                                                                    
   2   generic:                                                                 
   3     # specify list of hostnames where buildtest can run for given system re
   4     hostnames: [".*"]                                                      
   5                                                                            
   6     # system description                                                   
   7     description: Generic System                                            
   8     # specify module system used at your site (environment-modules, lmod)  
   9     moduletool: N/A                                                        
  10                                                                            
  11     executors:                                                             
  12       # define local executors for running jobs locally                    
  13       local:                                                               
  14         bash:                                                              
  15           description: submit jobs on local machine using bash shell       
  16           shell: bash                                                      
  17         sh:                                                                
  18           description: submit jobs on local machine using sh shell         
  19           shell: sh                                                        
  20         csh:                                                               
  21           description: submit jobs on local machine using csh shell        
  22           shell: csh                                                       
  23         zsh:                                                               
  24           description: submit jobs on local machine using zsh shell        
  25           shell: zsh                                                       
  26     # compiler block                                                       
  27     compilers:                                                             
  28       # regular expression to search for compilers based on module pattern.
  29       # find:                                                              
  30       #  gcc: "^(gcc)"                                                     
  31       #  intel: "^(intel)"                                                 
  32       #  cray: "^(craype)"                                                 
  33       #  pgi: "^(pgi)"                                                     
  34       #  cuda: ^(cuda)"                                                    
  35       #  clang: "^(clang)"                                                 
  36                                                                            
  37       # declare compiler instance which can be site-specific. You can let '
  38       compiler:                                                            
  39         gcc:                                                               
  40           builtin_gcc:                                                     
  41             cc: gcc                                                        
  42             fc: gfortran                                                   
  43             cxx: g++                                                       
  44                                                                            
  45     # location of log directory                                            
  46     # logdir: /tmp/                                                        
  47                                                                            
  48     # specify location where buildtest will write tests                    
  49     # testdir: /tmp                                                        
  50                                                                            
  51     # specify one or more directory where buildtest should load buildspecs 
  52     # buildspec_roots: []                                                  
  53                                                                            
  54     cdash:                                                                 
  55       url: https://my.cdash.org/                                           
  56       project: buildtest                                                   
  57       site: generic                                                        
  58       buildname: tutorials                                                 
  59

Note

buildtest config view will display contents of user buildtest settings ~/.buildtest/config.yml if found, otherwise it will display the default configuration

View Executors (buildtest config executors)

You can use the command buildtest config executors to view executors from buildtest configuration file. Shown below is the command usage

$ buildtest config executors --help
usage: buildtest [options] [COMMANDS] config executors [-h]
                                                       [-j | -y | -d | -i]

optional arguments:
  -h, --help      show this help message and exit
  -j, --json      View executor in JSON format
  -y, --yaml      View executors in YAML format
  -d, --disabled  Show disabled executors
  -i, --invalid   Show invalid executors

You can run buildtest config executors without any options and it will report a list of named executors that you would reference in buildspec using the executor property. If you prefer json or yaml format you can use --json or --yaml option.

$ buildtest config executors
generic.local.bash
generic.local.sh
generic.local.csh
generic.local.zsh

View Registered Systems

Your buildtest configuration may compose of one or more systems since you can define multiple systems in a single configuration file to run buildtest for different HPC clusters. You can use buildtest config systems to report all system details defined in your configuration file. In this example below we should the generic system. If you have multiple entries, you will see one entry per system record.

$ buildtest config systems
System Summary (Configuration=/home/docs/checkouts/readthedocs.org/user_builds/b
            uildtest/checkouts/latest/buildtest/settings/config.yml)            
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ system        ┃ description             ┃ moduletool        ┃ hostnames      ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ generic       │ Generic System          │ N/A               │ .*             │
└───────────────┴─────────────────────────┴───────────────────┴────────────────┘

Example Configurations

buildtest provides a few example configurations for configuring buildtest this can be retrieved by running buildtest schema -n settings.schema.json --examples or short option (-e), which will validate each example with schema file settings.schema.json.

$ buildtest schema -n settings.schema.json -e
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    executors:                                                                  
      defaults:                                                                 
        pollinterval: 10                                                        
        maxpendtime: 45                                                         
      lsf:                                                                      
        batch:                                                                  
          description: "LSF Executor name 'batch' that submits jobs to 'batch' q
          queue: batch                                                          
          account: developer                                                    
          options: ["-W 20"]                                                    
          before_script: |                                                      
            time                                                                
            echo "commands run before job"                                      
        test:                                                                   
          description: "LSF Executor name 'test' that submits jobs to 'test' que
          launcher: bsub                                                        
          queue: test                                                           
          account: qa                                                           
          options: ["-W 20"]                                                    
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
                                                                                
    moduletool: N/A                                                             
    executors:                                                                  
      local:                                                                    
        bash:                                                                   
          description: submit jobs on local machine                             
          shell: bash -v                                                        
                                                                                
      slurm:                                                                    
        haswell:                                                                
          launcher: sbatch                                                      
          options:                                                              
            - "-p haswell"                                                      
            - "-t 00:10"                                                        
                                                                                
      lsf:                                                                      
        batch:                                                                  
          launcher: bsub                                                        
          queue: batch                                                          
          options:                                                              
            - "-q batch"                                                        
            - "-t 00:10"                                                        
      cobalt:                                                                   
        normal:                                                                 
          launcher: qsub                                                        
          queue: normal                                                         
          options:                                                              
            - "-n 1"                                                            
            - "-t 10"                                                           
                                                                                
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    #logdir: $BUILDTEST_ROOT/logs                                               
    #testdir: $BUILDTEST_ROOT/tests                                             
    moduletool: N/A                                                             
    cdash:                                                                      
      url: https://my.cdash.org                                                 
      project: buildtest                                                        
      site: laptop                                                              
    processor:                                                                  
      numcpus: 8                                                                
      cores: 4                                                                  
      threads_per_core: 2                                                       
      sockets: 1                                                                
      model: "Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz"                         
    executors:                                                                  
      local:                                                                    
        bash:                                                                   
          description: submit jobs on local machine using bash shell            
          shell: bash                                                           
          module:                                                               
            purge: True                                                         
            restore: "Default"                                                  
            load: ["gcc"]                                                       
        sh:                                                                     
          description: submit jobs on local machine using sh shell              
          shell: sh                                                             
          before_script: |                                                      
            date                                                                
            echo "these commands will be run"                                   
        csh:                                                                    
          description: submit jobs on local machine using csh shell             
          shell: csh -x                                                         
                                                                                
        tcsh:                                                                   
          description: submit jobs on local machine using tcsh shell            
          shell: /bin/tcsh                                                      
                                                                                
        zsh:                                                                    
          description: submit jobs on local machine using zsh shell             
          shell: /bin/zsh                                                       
                                                                                
    compilers:                                                                  
      find:                                                                     
        gcc: "^(gcc|GCC|PrgEnv-gnu)"                                            
        intel: "^(intel|Intel|PrgEnv-intel)"                                    
        cray: "^(cray|PrgEnv-cray)"                                             
        clang: "^(clang|Clang)"                                                 
        cuda: "^(cuda|CUDA)"                                                    
        pgi: "^(pgi|PGI|PrgEnv-pgi)"                                            
                                                                                
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
          gcc@7.2.0:                                                            
            cc: 'cc'                                                            
            cxx: 'cxx'                                                          
            fc: 'fc'                                                            
            module:                                                             
              load:                                                             
              - gcc/7.2.0                                                       
        intel:                                                                  
          intel@2019:                                                           
            cc: 'icc'                                                           
            cxx: 'icpc'                                                         
            fc: 'ifort'                                                         
            module:                                                             
              purge: True                                                       
              load:                                                             
              - gcc/7.2.0                                                       
              - intel/2019                                                      
        cray:                                                                   
          craype@2.6.2:                                                         
            cc: 'cc'                                                            
            cxx: 'CC'                                                           
            fc: 'fc'                                                            
            module:                                                             
              load: [craype/2.6.2]                                              
              swap: [PrgEnv-gnu, PrgEnv-cray]                                   
                                                                                
        clang:                                                                  
          clang@12.0.0:                                                         
            cc: 'clang'                                                         
            cxx: 'clang++'                                                      
            fc: 'None'                                                          
            module:                                                             
              load: [clang/12.0]                                                
        cuda:                                                                   
          cuda@11.0:                                                            
            cc: 'nvcc'                                                          
            cxx: 'nvcc'                                                         
            fc: 'None'                                                          
            module:                                                             
              load: [cuda/11.0]                                                 
        pgi:                                                                    
          pgi@18.0:                                                             
            cc: 'pgcc'                                                          
            cxx: 'pgc++'                                                        
            fc: 'pgfortran'                                                     
            module:                                                             
              load: [pgi/18.0]                                                  
                                                                                
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    buildspec_roots:                                                            
      - $HOME/buildtest-cori                                                    
    testdir: /tmp/buildtest                                                     
    executors:                                                                  
      defaults:                                                                 
        pollinterval: 20                                                        
        maxpendtime: 30                                                         
        account: admin                                                          
      slurm:                                                                    
        normal:                                                                 
          options: ["-C haswell"]                                               
          qos: normal                                                           
          before_script: |                                                      
            time                                                                
            echo "commands run before job"                                      
                                                                                
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: N/A                                                             
    executors:                                                                  
      defaults:                                                                 
         pollinterval: 10                                                       
         maxpendtime: 30                                                        
      pbs:                                                                      
        workq:                                                                  
          queue: workq                                                          
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran                                               
─ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/latest… ─
system:                                                                         
  generic:                                                                      
    hostnames: ['.*']                                                           
    moduletool: lmod                                                            
    executors:                                                                  
      defaults:                                                                 
         maxpendtime: 30                                                        
                                                                                
      cobalt:                                                                   
        knl:                                                                    
          queue: knl                                                            
                                                                                
        haswell:                                                                
          queue: haswell                                                        
                                                                                
    compilers:                                                                  
      compiler:                                                                 
        gcc:                                                                    
          default:                                                              
            cc: /usr/bin/gcc                                                    
            cxx: /usr/bin/g++                                                   
            fc: /usr/bin/gfortran

If you want to retrieve full json schema file for buildtest configuration you can run buildtest schema -n settings.schema.json --json or short option -j.