Compiler Schema¶
The compiler schema is used for compilation of programs, currently we support
single source file compilation. In order to use the compiler schema you must set type: compiler
in your
sub-schema. See compiler schema docs
Compilation Examples¶
We assume the reader has basic understanding of Global Schema validation. Shown below is the schema header definition for compiler-v1.0.schema.json:
{
"$id": "compiler-v1.0.schema.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "compiler schema version 1.0",
"description": "The compiler schema is of ``type: compiler`` in sub-schema which is used for compiling and running programs",
"type": "object",
"required": [
"type",
"source",
"compilers",
"executor"
],
The required fields for compiler schema are type, compilers, source and executor.
Shown below is a test name hello_f
that compiles Fortran code with GNU compiler.
version: "1.0"
buildspecs:
hello_f:
type: compiler
description: "Hello World Fortran Compilation"
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/hello.f90"
compilers:
name: ["^(builtin_gcc)$"]
default:
gcc:
fflags: -Wall
The source
property is used to specify input program for
compilation, this can be a file relative to buildspec file or an absolute path.
In this example the source file src/hello.f90
is relative to buildspec file.
The compilers
section specifies compiler configuration, the name
field is required property which is used to search compilers based on regular expression.
In this example we use the builtin_gcc compiler as regular expression which is the system
gcc compiler provided by buildtest. The default
section specifies default compiler
configuration applicable to a specific compiler group.
Shown below is an example build for the buildspec example
$ buildtest build -b tutorials/compilers/gnu_hello_fortran.yml
User: siddiq90
Hostname: DOE-7086392.local
Platform: Darwin
Current Time: 2021/07/06 18:54:28
buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
buildtest version: 0.9.6
python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
python version: 3.7.3
Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
Configuration File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest/settings/config.yml
Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/gnu_hello_fortran.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+---------------------------------------------------------------------------------------------+
| Discovered Buildspecs |
+=============================================================================================+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_fortran.yml |
+---------------------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+---------------------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_fortran.yml
name description
------- -------------------------------
hello_f Hello World Fortran Compilation
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
---------+----------+----------+--------------------+--------------------------+-------------+-----------------------------------------------------------------------------------------------------------------------------
hello_f | 5e3d8b5f | compiler | generic.local.bash | ['tutorials', 'compile'] | builtin_gcc | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/gnu_hello_fortran/hello_f/1/hello_f_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
name | id | executor | status | returncode
---------+----------+--------------------+----------+--------------
hello_f | 5e3d8b5f | generic.local.bash | FAIL | 127
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 0/1 Percentage: 0.000%
Failed Tests: 1/1 Percentage: 100.000%
Writing Logfile to: /var/folders/1m/_jjv09h17k37mkktwnmbkmj0002t_q/T/buildtest_ycbz5z6n.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
The generated test for test name hello_f is the following:
#!/bin/bash
# name of executable
_EXEC=hello.f90.exe
# Compilation Line
gfortran -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello.f90
# Run executable
./$_EXEC
buildtest will use compiler wrappers specified in your settings to build the test, however these values can be overridden in buildspec file which will be discussed later.
The builtin_gcc
compiler is defined below this can be retrieved by running
buildtest config compilers
. The -y
will display compilers in YAML format.
$ buildtest config compilers -y
gcc:
builtin_gcc:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
buildtest will compile and run the code depending on the compiler flags. buildtest,
will detect the file extension of source file (source
property) to detect
programming language and finally generate the appropriate C, C++, or Fortran
compilation based on language detected. In this example, buildtest detects a
.f90 file extension and determines this is a Fortran program.
Shown below is the file extension table buildtest uses for determining the programming language.
Language |
File Extension |
---|---|
C |
.c |
C++ |
.cc .cxx .cpp .c++ |
Fortran |
.f90 .F90 .f95 .f .F .FOR .for .FTN .ftn |
Compiler Selection¶
buildtest selects compiler based on name
property which is a list of regular expression
applied for available compilers defined in buildtest configuration. In example below
we select all compilers with regular expression ^(builtin_gcc|gcc)
that is specified in line name: ["^(builtin_gcc|gcc)"]
version: "1.0"
buildspecs:
vecadd_gnu:
type: compiler
description: Vector Addition example with GNU compiler
tags: [tutorials, compile]
executor: generic.local.bash
source: src/vecAdd.c
compilers:
name: ["^(builtin_gcc|gcc)"]
default:
gcc:
cflags: -fopenacc
ldflags: -lm
Currently, we have 3 compilers defined in buildtest settings, shown below is a listing
of all compilers. We used buildtest config compilers find
to detect compilers.
$ buildtest config compilers
builtin_gcc
gcc/9.3.0-n7p74fd
gcc/10.2.0-37fmsw7
Note
This example may vary on your machine depending on compilers available via module
command.
We expect buildtest to select all three compilers based on our regular expression. In the following
build, notice we have three tests for vecadd_gnu
one for each compiler:
$ buildtest build -b tutorials/compilers/vecadd.yml
User: siddiq90
Hostname: DOE-7086392.local
Platform: Darwin
Current Time: 2021/06/10 21:52:32
buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
buildtest version: 0.9.5
python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
python version: 3.7.3
Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
Configuration File: /Users/siddiq90/.buildtest/config.yml
Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/vecadd.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+----------------------------------------------------------------------------------+
| Discovered Buildspecs |
+==================================================================================+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/vecadd.yml |
+----------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+----------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/vecadd.yml
name description
---------- -----------------------------------------
vecadd_gnu Vector Addition example with GNU compiler
vecadd_gnu Vector Addition example with GNU compiler
vecadd_gnu Vector Addition example with GNU compiler
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
------------+----------+----------+--------------------+--------------------------+--------------------+------------------------------------------------------------------------------------------------------------------------
vecadd_gnu | 6f6b16e1 | compiler | generic.local.bash | ['tutorials', 'compile'] | builtin_gcc | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/vecadd/vecadd_gnu/2/vecadd_gnu_build.sh
vecadd_gnu | a76dd163 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/vecadd/vecadd_gnu/3/vecadd_gnu_build.sh
vecadd_gnu | 82360702 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/10.2.0-37fmsw7 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/vecadd/vecadd_gnu/4/vecadd_gnu_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
name | id | executor | status | returncode
------------+----------+--------------------+----------+--------------
vecadd_gnu | 6f6b16e1 | generic.local.bash | PASS | 0
vecadd_gnu | a76dd163 | generic.local.bash | PASS | 0
vecadd_gnu | 82360702 | generic.local.bash | PASS | 0
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 3/3 Percentage: 100.000%
Failed Tests: 0/3 Percentage: 0.000%
Writing Logfile to: /Users/siddiq90/buildtest/buildtest_b0jwyoyv.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
buildtest will use compiler settings including module configuration from buildtest
settings (config.yml
). In example below we show the compiler definitions for the
three gcc compilers. The module
section is the declaration of modules to load, by default
we disable purge (purge: False
) which instructs buildtest to not insert module purge
.
The load
is a list of modules to load via module load
.
Shown below is the compiler configuration.
1compilers:
2 find:
3 gcc: ^(gcc)
4 compiler:
5 gcc:
6 builtin_gcc:
7 cc: gcc
8 fc: gfortran
9 cxx: g++
10 gcc/9.3.0-n7p74fd:
11 cc: gcc
12 cxx: g++
13 fc: gfortran
14 module:
15 load:
16 - gcc/9.3.0-n7p74fd
17 purge: false
18 gcc/10.2.0-37fmsw7:
19 cc: gcc
20 cxx: g++
21 fc: gfortran
22 module:
23 load:
24 - gcc/10.2.0-37fmsw7
25 purge: false
If we take a closer look at the generated test we see the module load command in the test script.
1#!/bin/bash
2
3
4# name of executable
5_EXEC=vecAdd.c.exe
6# Loading modules
7module load gcc/10.2.0-37fmsw7
8# Compilation Line
9gcc -fopenacc -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/vecAdd.c -lm
10
11
12# Run executable
13./$_EXEC
1#!/bin/bash
2
3
4# name of executable
5_EXEC=vecAdd.c.exe
6# Loading modules
7module load gcc/9.3.0-n7p74fd
8# Compilation Line
9gcc -fopenacc -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/vecAdd.c -lm
10
11
12# Run executable
13./$_EXEC
Excluding Compilers¶
The exclude
property is part of compilers section which allows one to exclude compilers
upon discovery by name
field. The exclude property is a list of compiler names that
will be removed from test generation which is done prior to build phase. buildtest will exclude
any compilers specified in exclude
if they were found based on regular
expression in name
field. In this example, we slightly modified previous example
by excluding gcc/10.2.0-37fmsw7
compiler. This is specified by exclude: [gcc/10.2.0-37fmsw7]
.
version: "1.0"
buildspecs:
vecadd_gnu_exclude:
type: compiler
description: Vector Addition example with GNU compilers but exclude gcc@10.2.0
tags: [tutorials, compile]
executor: generic.local.bash
source: src/vecAdd.c
compilers:
name: ["^(gcc)"]
exclude: [gcc/10.2.0-37fmsw7]
default:
gcc:
cflags: -fopenacc
ldflags: -lm
Notice when we build this test, buildtest will exclude gcc/10.2.0-37fmsw7 compiler and test is not created during build phase.
1$ buildtest build -b tutorials/compilers/compiler_exclude.yml
2
3
4User: siddiq90
5Hostname: DOE-7086392.local
6Platform: Darwin
7Current Time: 2021/06/10 21:56:11
8buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
9buildtest version: 0.9.5
10python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
11python version: 3.7.3
12Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
13Configuration File: /Users/siddiq90/.buildtest/config.yml
14Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/compiler_exclude.yml
15
16+-------------------------------+
17| Stage: Discovering Buildspecs |
18+-------------------------------+
19
20+--------------------------------------------------------------------------------------------+
21| Discovered Buildspecs |
22+============================================================================================+
23| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_exclude.yml |
24+--------------------------------------------------------------------------------------------+
25Discovered Buildspecs: 1
26Excluded Buildspecs: 0
27Detected Buildspecs after exclusion: 1
28Excluding compiler: gcc/10.2.0-37fmsw7 from test generation
29
30+---------------------------+
31| Stage: Parsing Buildspecs |
32+---------------------------+
33
34 schemafile | validstate | buildspec
35---------------------------+--------------+--------------------------------------------------------------------------------------------
36 compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_exclude.yml
37
38
39
40name description
41------------------ -----------------------------------------------------------------
42vecadd_gnu_exclude Vector Addition example with GNU compilers but exclude gcc@10.2.0
43
44+----------------------+
45| Stage: Building Test |
46+----------------------+
47
48
49
50 name | id | type | executor | tags | compiler | testpath
51--------------------+----------+----------+--------------------+--------------------------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------
52 vecadd_gnu_exclude | a7373d09 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/compiler_exclude/vecadd_gnu_exclude/0/vecadd_gnu_exclude_build.sh
53
54+---------------------+
55| Stage: Running Test |
56+---------------------+
57
58 name | id | executor | status | returncode
59--------------------+----------+--------------------+----------+--------------
60 vecadd_gnu_exclude | a7373d09 | generic.local.bash | PASS | 0
61
62+----------------------+
63| Stage: Test Summary |
64+----------------------+
65
66Passed Tests: 1/1 Percentage: 100.000%
67Failed Tests: 0/1 Percentage: 0.000%
68
69
70Writing Logfile to: /Users/siddiq90/buildtest/buildtest_4szlay_j.log
71A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
Compiler Defaults and Override Default Settings¶
Sometimes you may want to set default compiler flags (cflags, fflags, cxxflags),
preprocessor (cppflags) or linker flags (ldflags) for compiler group (gcc, intel, pgi, etc…).
This can be achieved using the default
property that is part of compilers section.
The default
field is organized into compiler groups, in example below we set default C compiler flags
(cflags: -O1
). In addition, we can override default settings using the
config
property where one must specify the compiler name to override.
In example below we can override compiler settings for gcc/9.3.0-n7p74fd
to use -O2
and gcc/10.2.0-37fmsw7
to use -O3
for cflags .
version: "1.0"
buildspecs:
hello_c:
type: compiler
description: "Hello World C Compilation"
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/hello.c"
compilers:
name: ["^(builtin_gcc|gcc)"]
default:
gcc:
cflags: -O1
config:
gcc/9.3.0-n7p74fd:
cflags: -O2
gcc/10.2.0-37fmsw7:
cflags: -O3
Next we run this test, and we get three tests for test name hello_c.
$ buildtest build -b tutorials/compilers/gnu_hello_c.yml
User: siddiq90
Hostname: DOE-7086392.local
Platform: Darwin
Current Time: 2021/06/10 22:00:08
buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
buildtest version: 0.9.5
python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
python version: 3.7.3
Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
Configuration File: /Users/siddiq90/.buildtest/config.yml
Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/gnu_hello_c.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+---------------------------------------------------------------------------------------+
| Discovered Buildspecs |
+=======================================================================================+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_c.yml |
+---------------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+---------------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_c.yml
name description
------- -------------------------
hello_c Hello World C Compilation
hello_c Hello World C Compilation
hello_c Hello World C Compilation
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
---------+----------+----------+--------------------+--------------------------+--------------------+-----------------------------------------------------------------------------------------------------------------------
hello_c | afa92b9d | compiler | generic.local.bash | ['tutorials', 'compile'] | builtin_gcc | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/gnu_hello_c/hello_c/2/hello_c_build.sh
hello_c | 498010d3 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/gnu_hello_c/hello_c/3/hello_c_build.sh
hello_c | ee753488 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/10.2.0-37fmsw7 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/gnu_hello_c/hello_c/4/hello_c_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
name | id | executor | status | returncode
---------+----------+--------------------+----------+--------------
hello_c | afa92b9d | generic.local.bash | PASS | 0
hello_c | 498010d3 | generic.local.bash | PASS | 0
hello_c | ee753488 | generic.local.bash | PASS | 0
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 3/3 Percentage: 100.000%
Failed Tests: 0/3 Percentage: 0.000%
Writing Logfile to: /Users/siddiq90/buildtest/buildtest_dtyx0ags.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
Writing Logfile to: /private/tmp/buildtest/buildtest_hh9k7vm6.log
If we inspect the following test, we see the compiler flags are associated with the compiler. The test below
is for builtin_gcc which use the default -O1
compiler flag as shown below.
1#!/bin/bash
2
3
4# name of executable
5_EXEC=hello.c.exe
6# Compilation Line
7gcc -O1 -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello.c
8
9
10# Run executable
11./$_EXEC
The test for gcc/10.2.0-37fmsw7 and gcc/9.3.0-n7p74fd have cflags -O3
and -O2
set in their respective tests.
1#!/bin/bash
2
3
4# name of executable
5_EXEC=hello.c.exe
6# Loading modules
7module load gcc/10.2.0-37fmsw7
8# Compilation Line
9gcc -O3 -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello.c
10
11
12# Run executable
13./$_EXEC
1#!/bin/bash
2
3
4# name of executable
5_EXEC=hello.c.exe
6# Loading modules
7module load gcc/9.3.0-n7p74fd
8# Compilation Line
9gcc -O2 -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello.c
10
11
12# Run executable
13./$_EXEC
Setting environment variables¶
Environment variables can be set using env
property which is a list of
key/value pair to assign environment variables. This property can be used in default
section within a compiler group. In example below we have an OpenMP Hello World example in C
where we define OMP_NUM_THREADS environment variable which controls number of OpenMP
threads to use when running program. In this example we use 2 threads for all gcc
compiler group
version: "1.0"
buildspecs:
openmp_hello_c_example:
type: compiler
description: OpenMP Hello World C example
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/hello_omp.c"
compilers:
name: ["^(gcc)"]
default:
gcc:
cflags: -fopenmp
env:
OMP_NUM_THREADS: 2
Shown below is one of the generated test and notice that buildtest will set environment variable OMP_NUM_THREADS.
1#!/bin/bash
2
3
4# name of executable
5_EXEC=hello_omp.c.exe
6# Declare environment variables
7export OMP_NUM_THREADS=2
8
9
10# Loading modules
11module load gcc/10.2.0-37fmsw7
12# Compilation Line
13gcc -fopenmp -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello_omp.c
14
15
16# Run executable
17./$_EXEC
Similarly, one can define environment variables at the compiler level in config
section.
buildtest will override value defined in default
section. In this example, we
make slight modification to the test, so that gcc/10.2.0-37fmsw7
will use 4 threads
when running program. This will override the default value of 2.
version: "1.0"
buildspecs:
override_environmentvars:
type: compiler
description: override default environment variables
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/hello_omp.c"
compilers:
name: ["^(gcc)"]
default:
gcc:
cflags: -fopenmp
env:
OMP_NUM_THREADS: 2
config:
gcc/10.2.0-37fmsw7:
env:
OMP_NUM_THREADS: 4
Next we build this test as follows:
$ buildtest build -b tutorials/compilers/envvar_override.yml
User: siddiq90
Hostname: DOE-7086392.local
Platform: Darwin
Current Time: 2021/06/10 22:04:19
buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
buildtest version: 0.9.5
python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
python version: 3.7.3
Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
Configuration File: /Users/siddiq90/.buildtest/config.yml
Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/envvar_override.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+-------------------------------------------------------------------------------------------+
| Discovered Buildspecs |
+===========================================================================================+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/envvar_override.yml |
+-------------------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+-------------------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/envvar_override.yml
name description
------------------------ --------------------------------------
override_environmentvars override default environment variables
override_environmentvars override default environment variables
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
--------------------------+----------+----------+--------------------+--------------------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------
override_environmentvars | 72619a4b | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/override_environmentvars_build.sh
override_environmentvars | 31098506 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/10.2.0-37fmsw7 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/override_environmentvars_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
name | id | executor | status | returncode
--------------------------+----------+--------------------+----------+--------------
override_environmentvars | 72619a4b | generic.local.bash | PASS | 0
override_environmentvars | 31098506 | generic.local.bash | PASS | 0
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Writing Logfile to: /Users/siddiq90/buildtest/buildtest_p3wdnl1t.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
Now let’s inspect the test by running buildtest inspect name
and we notice there are two test records for override_environmentvars using
gcc/9.3.0-n7p74fd and gcc/10.2.0-37fmsw7.
1$ buildtest inspect name override_environmentvars
2Reading Report File: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/report.json
3
4{
5 "override_environmentvars": [
6 {
7 "id": "72619a4b",
8 "full_id": "72619a4b-3ed2-489c-aebd-2e0cacbf2d6a",
9 "description": "override default environment variables",
10 "schemafile": "compiler-v1.0.schema.json",
11 "executor": "generic.local.bash",
12 "compiler": "gcc/9.3.0-n7p74fd",
13 "hostname": "DOE-7086392.local",
14 "user": "siddiq90",
15 "testroot": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0",
16 "testpath": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/stage/override_environmentvars.sh",
17 "stagedir": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/stage",
18 "command": "sh override_environmentvars_build.sh",
19 "outfile": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/override_environmentvars.out",
20 "errfile": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/override_environmentvars.err",
21 "buildspec_content": "version: \"1.0\"\nbuildspecs:\n override_environmentvars:\n type: compiler\n description: override default environment variables\n executor: generic.local.bash\n tags: [tutorials, compile]\n source: \"src/hello_omp.c\"\n compilers:\n name: [\"^(gcc)\"]\n default:\n gcc:\n cflags: -fopenmp\n env:\n OMP_NUM_THREADS: 2\n config:\n gcc/10.2.0-37fmsw7:\n env:\n OMP_NUM_THREADS: 4",
22 "test_content": "#!/bin/bash \n_EXEC=hello_omp.c.exe\nexport OMP_NUM_THREADS=2\nmodule load gcc/9.3.0-n7p74fd\ngcc -fopenmp -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello_omp.c\n./$_EXEC",
23 "logpath": "/Users/siddiq90/buildtest/buildtest_p3wdnl1t.log",
24 "tags": "tutorials compile",
25 "starttime": "2021/06/10 22:04:19",
26 "endtime": "2021/06/10 22:04:20",
27 "runtime": 0.727095,
28 "state": "PASS",
29 "returncode": 0,
30 "output": "Hello World from thread = 0\nHello World from thread = 1\n",
31 "error": "The following have been reloaded with a version change:\n 1) gcc/10.2.0-37fmsw7 => gcc/9.3.0-n7p74fd\n",
32 "job": null,
33 "build_script": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/0/override_environmentvars_build.sh"
34 },
35 {
36 "id": "31098506",
37 "full_id": "31098506-2bbf-4a50-8386-2fcd5bcddff5",
38 "description": "override default environment variables",
39 "schemafile": "compiler-v1.0.schema.json",
40 "executor": "generic.local.bash",
41 "compiler": "gcc/10.2.0-37fmsw7",
42 "hostname": "DOE-7086392.local",
43 "user": "siddiq90",
44 "testroot": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1",
45 "testpath": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/stage/override_environmentvars.sh",
46 "stagedir": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/stage",
47 "command": "sh override_environmentvars_build.sh",
48 "outfile": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/override_environmentvars.out",
49 "errfile": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/override_environmentvars.err",
50 "buildspec_content": "version: \"1.0\"\nbuildspecs:\n override_environmentvars:\n type: compiler\n description: override default environment variables\n executor: generic.local.bash\n tags: [tutorials, compile]\n source: \"src/hello_omp.c\"\n compilers:\n name: [\"^(gcc)\"]\n default:\n gcc:\n cflags: -fopenmp\n env:\n OMP_NUM_THREADS: 2\n config:\n gcc/10.2.0-37fmsw7:\n env:\n OMP_NUM_THREADS: 4",
51 "test_content": "#!/bin/bash \n_EXEC=hello_omp.c.exe\nexport OMP_NUM_THREADS=4\nmodule load gcc/10.2.0-37fmsw7\ngcc -fopenmp -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello_omp.c\n./$_EXEC",
52 "logpath": "/Users/siddiq90/buildtest/buildtest_p3wdnl1t.log",
53 "tags": "tutorials compile",
54 "starttime": "2021/06/10 22:04:20",
55 "endtime": "2021/06/10 22:04:20",
56 "runtime": 0.482645,
57 "state": "PASS",
58 "returncode": 0,
59 "output": "Hello World from thread = 1\nHello World from thread = 3\nHello World from thread = 2\nHello World from thread = 0\n",
60 "error": "",
61 "job": null,
62 "build_script": "/Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/envvar_override/override_environmentvars/1/override_environmentvars_build.sh"
63 }
64 ]
65}
Tweak how test are passed¶
The status
property can be used to determine how buildtest will pass the test. By
default, buildtest will use returncode to determine if test PASS
or FAIL
with
exitcode 0 as PASS and anything else is FAIL.
Sometimes, it may be useful check output of test to determine using regular expression. This
can be done via status
property. In this example, we define two tests, the first one defines status
property in the default gcc group. This means all compilers that belong to gcc
group will be matched with the regular expression.
In second example we override the status regex
property for gcc/10.2.0-37fmsw7. We expect
the test to produce an output of final result: 1.000000
so we expect one failure from
gcc/10.2.0-37fmsw7.
version: "1.0"
buildspecs:
default_status_regex:
type: compiler
description: Regular expression check in stdout for gcc group
tags: [tutorials, compile]
executor: generic.local.bash
source: src/vecAdd.c
compilers:
name: ["^(gcc)"]
default:
gcc:
cflags: -fopenacc
ldflags: -lm
status:
regex:
stream: stdout
exp: "^final result: 1.000000$"
override_status_regex:
type: compiler
description: Override regular expression for compiler gcc/10.2.0-37fmsw7
tags: [tutorials, compile]
executor: generic.local.bash
source: src/vecAdd.c
compilers:
name: ["^(gcc)"]
default:
gcc:
cflags: -fopenacc
ldflags: -lm
status:
regex:
stream: stdout
exp: "^final result: 1.000000$"
config:
gcc/10.2.0-37fmsw7:
status:
regex:
stream: stdout
exp: "^final result: 0.99$"
If we build this test, notice that test id 9320ca41 failed which corresponds to
gcc/10.2.0-37fmsw7
compiler test. The test fails because it fails to pass on
regular expression even though we have a returncode of 0.
1$ buildtest build -b tutorials/compilers/compiler_status_regex.yml
2
3
4User: siddiq90
5Hostname: DOE-7086392.local
6Platform: Darwin
7Current Time: 2021/06/10 22:08:03
8buildtest path: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest
9buildtest version: 0.9.5
10python path: /Users/siddiq90/.local/share/virtualenvs/buildtest-KLOcDrW0/bin/python
11python version: 3.7.3
12Test Directory: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests
13Configuration File: /Users/siddiq90/.buildtest/config.yml
14Command: /Users/siddiq90/Documents/GitHubDesktop/buildtest/bin/buildtest build -b tutorials/compilers/compiler_status_regex.yml
15
16+-------------------------------+
17| Stage: Discovering Buildspecs |
18+-------------------------------+
19
20+-------------------------------------------------------------------------------------------------+
21| Discovered Buildspecs |
22+=================================================================================================+
23| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_status_regex.yml |
24+-------------------------------------------------------------------------------------------------+
25Discovered Buildspecs: 1
26Excluded Buildspecs: 0
27Detected Buildspecs after exclusion: 1
28
29+---------------------------+
30| Stage: Parsing Buildspecs |
31+---------------------------+
32
33 schemafile | validstate | buildspec
34---------------------------+--------------+-------------------------------------------------------------------------------------------------
35 compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_status_regex.yml
36
37
38
39name description
40--------------------- -----------------------------------------------------------
41default_status_regex Regular expression check in stdout for gcc group
42default_status_regex Regular expression check in stdout for gcc group
43override_status_regex Override regular expression for compiler gcc/10.2.0-37fmsw7
44override_status_regex Override regular expression for compiler gcc/10.2.0-37fmsw7
45
46+----------------------+
47| Stage: Building Test |
48+----------------------+
49
50
51
52 name | id | type | executor | tags | compiler | testpath
53-----------------------+----------+----------+--------------------+--------------------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------
54 default_status_regex | a023a2c2 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/compiler_status_regex/default_status_regex/0/default_status_regex_build.sh
55 default_status_regex | 155865c3 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/10.2.0-37fmsw7 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/compiler_status_regex/default_status_regex/1/default_status_regex_build.sh
56 override_status_regex | 3411bddf | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/9.3.0-n7p74fd | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/compiler_status_regex/override_status_regex/0/override_status_regex_build.sh
57 override_status_regex | 295310a4 | compiler | generic.local.bash | ['tutorials', 'compile'] | gcc/10.2.0-37fmsw7 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/tests/generic.local.bash/compiler_status_regex/override_status_regex/1/override_status_regex_build.sh
58
59+---------------------+
60| Stage: Running Test |
61+---------------------+
62
63 name | id | executor | status | returncode
64-----------------------+----------+--------------------+----------+--------------
65 default_status_regex | a023a2c2 | generic.local.bash | PASS | 0
66 default_status_regex | 155865c3 | generic.local.bash | PASS | 0
67 override_status_regex | 3411bddf | generic.local.bash | PASS | 0
68 override_status_regex | 295310a4 | generic.local.bash | FAIL | 0
69
70+----------------------+
71| Stage: Test Summary |
72+----------------------+
73
74Passed Tests: 3/4 Percentage: 75.000%
75Failed Tests: 1/4 Percentage: 25.000%
76
77
78Writing Logfile to: /Users/siddiq90/buildtest/buildtest_hp7_gpbn.log
79A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest.log
Single Test Multiple Compilers¶
It’s possible to run single test across multiple compilers (gcc, intel, cray, etc…). In the
next example, we will build an OpenMP reduction test using gcc, intel and cray compilers. In this
test, we use name
field to select compilers that start with gcc, intel and PrgEnv-cray
as compiler names. The default
section is organized by compiler groups which inherits compiler flags
for all compilers. OpenMP flag for gcc, intel and cray differ for instance one must use -fopenmp
for gcc,
--qopenmp
for intel and -h omp
for cray.
1 version: "1.0"
2 buildspecs:
3 reduction:
4 type: compiler
5 executor: local.bash
6 source: src/reduction.c
7 description: OpenMP reduction example using gcc, intel and cray compiler
8 tags: [openmp]
9 compilers:
10 name: ["^(gcc|intel|PrgEnv-cray)"]
11 default:
12 all:
13 env:
14 OMP_NUM_THREADS: 4
15 gcc:
16 cflags: -fopenmp
17 intel:
18 cflags: -qopenmp
19 cray:
20 cflags: -h omp
In this example OMP_NUM_THREADS environment variable under the all
section which
will be used for all compiler groups. This example was built on Cori, we expect this
test to run against every gcc, intel and PrgEnv-cray compiler module:
$ buildtest build -b buildspecs/apps/openmp/reduction.yml
User: siddiq90
Hostname: cori02
Platform: Linux
Current Time: 2021/06/11 08:42:54
buildtest path: /global/homes/s/siddiq90/github/buildtest/bin/buildtest
buildtest version: 0.9.5
python path: /global/homes/s/siddiq90/.conda/envs/buildtest/bin/python
python version: 3.8.8
Test Directory: /global/u1/s/siddiq90/github/buildtest/var/tests
Configuration File: /global/u1/s/siddiq90/.buildtest/config.yml
Command: /global/homes/s/siddiq90/github/buildtest/bin/buildtest build -b buildspecs/apps/openmp/reduction.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+----------------------------------------------------------------------------------+
| Discovered Buildspecs |
+==================================================================================+
| /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/openmp/reduction.yml |
+----------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+----------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/openmp/reduction.yml
name description
--------- ---------------------------------------------------------------
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
reduction OpenMP reduction example using gcc, intel, PrgEnv-cray compiler
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
-----------+----------+----------+-----------------+------------+-----------------------------------------+------------------------------------------------------------------------------------------------------------
reduction | fd93fdcb | compiler | cori.local.bash | ['openmp'] | gcc/6.1.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/25/reduction_build.sh
reduction | 43737191 | compiler | cori.local.bash | ['openmp'] | gcc/7.3.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/26/reduction_build.sh
reduction | 6e2e95cd | compiler | cori.local.bash | ['openmp'] | gcc/8.1.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/27/reduction_build.sh
reduction | c48a8d8d | compiler | cori.local.bash | ['openmp'] | gcc/8.2.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/28/reduction_build.sh
reduction | a6201c48 | compiler | cori.local.bash | ['openmp'] | gcc/8.3.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/29/reduction_build.sh
reduction | aa06b1be | compiler | cori.local.bash | ['openmp'] | gcc/9.3.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/30/reduction_build.sh
reduction | 02b8e7aa | compiler | cori.local.bash | ['openmp'] | gcc/10.1.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/31/reduction_build.sh
reduction | bd9abd7e | compiler | cori.local.bash | ['openmp'] | gcc/6.3.0 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/32/reduction_build.sh
reduction | 9409a86f | compiler | cori.local.bash | ['openmp'] | gcc/8.1.1-openacc-gcc-8-branch-20190215 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/33/reduction_build.sh
reduction | b9700a0f | compiler | cori.local.bash | ['openmp'] | PrgEnv-cray/6.0.5 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/34/reduction_build.sh
reduction | a605c970 | compiler | cori.local.bash | ['openmp'] | PrgEnv-cray/6.0.7 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/35/reduction_build.sh
reduction | 9ef915a9 | compiler | cori.local.bash | ['openmp'] | PrgEnv-cray/6.0.9 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/36/reduction_build.sh
reduction | 4f9e4242 | compiler | cori.local.bash | ['openmp'] | intel/19.0.3.199 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/37/reduction_build.sh
reduction | e37befed | compiler | cori.local.bash | ['openmp'] | intel/19.1.2.254 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/38/reduction_build.sh
reduction | 1e9b0ab5 | compiler | cori.local.bash | ['openmp'] | intel/16.0.3.210 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/39/reduction_build.sh
reduction | 4e6d6f8a | compiler | cori.local.bash | ['openmp'] | intel/17.0.1.132 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/40/reduction_build.sh
reduction | ad1e44af | compiler | cori.local.bash | ['openmp'] | intel/17.0.2.174 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/41/reduction_build.sh
reduction | 49acf44b | compiler | cori.local.bash | ['openmp'] | intel/18.0.1.163 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/42/reduction_build.sh
reduction | 4192750c | compiler | cori.local.bash | ['openmp'] | intel/18.0.3.222 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/43/reduction_build.sh
reduction | 06584529 | compiler | cori.local.bash | ['openmp'] | intel/19.0.0.117 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/44/reduction_build.sh
reduction | 82fd9bab | compiler | cori.local.bash | ['openmp'] | intel/19.0.8.324 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/45/reduction_build.sh
reduction | 6140e8b4 | compiler | cori.local.bash | ['openmp'] | intel/19.1.0.166 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/46/reduction_build.sh
reduction | ac509e2e | compiler | cori.local.bash | ['openmp'] | intel/19.1.1.217 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/47/reduction_build.sh
reduction | 9c39818e | compiler | cori.local.bash | ['openmp'] | intel/19.1.2.275 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/48/reduction_build.sh
reduction | 2cb3acd1 | compiler | cori.local.bash | ['openmp'] | intel/19.1.3.304 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.local.bash/reduction/reduction/49/reduction_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
name | id | executor | status | returncode
-----------+----------+-----------------+----------+--------------
reduction | fd93fdcb | cori.local.bash | PASS | 0
reduction | 43737191 | cori.local.bash | PASS | 0
reduction | 6e2e95cd | cori.local.bash | PASS | 0
reduction | c48a8d8d | cori.local.bash | PASS | 0
reduction | a6201c48 | cori.local.bash | PASS | 0
reduction | aa06b1be | cori.local.bash | PASS | 0
reduction | 02b8e7aa | cori.local.bash | PASS | 0
reduction | bd9abd7e | cori.local.bash | PASS | 0
reduction | 9409a86f | cori.local.bash | PASS | 0
reduction | b9700a0f | cori.local.bash | PASS | 0
reduction | a605c970 | cori.local.bash | PASS | 0
reduction | 9ef915a9 | cori.local.bash | PASS | 0
reduction | 4f9e4242 | cori.local.bash | PASS | 0
reduction | e37befed | cori.local.bash | PASS | 0
reduction | 1e9b0ab5 | cori.local.bash | PASS | 0
reduction | 4e6d6f8a | cori.local.bash | PASS | 0
reduction | ad1e44af | cori.local.bash | PASS | 0
reduction | 49acf44b | cori.local.bash | PASS | 0
reduction | 4192750c | cori.local.bash | PASS | 0
reduction | 06584529 | cori.local.bash | PASS | 0
reduction | 82fd9bab | cori.local.bash | PASS | 0
reduction | 6140e8b4 | cori.local.bash | PASS | 0
reduction | ac509e2e | cori.local.bash | PASS | 0
reduction | 9c39818e | cori.local.bash | PASS | 0
reduction | 2cb3acd1 | cori.local.bash | PASS | 0
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 25/25 Percentage: 100.000%
Failed Tests: 0/25 Percentage: 0.000%
Writing Logfile to: /tmp/buildtest_sq87154s.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /global/homes/s/siddiq90/github/buildtest/buildtest.log
If we inspect one of these tests from each compiler group (gcc, intel) we will see OMP_NUM_THREADS is set in all tests along with the appropriate compiler flag.
1 #!/bin/bash
2 _EXEC=reduction.c.exe
3 export OMP_NUM_THREADS=4
4 module load intel/19.1.3.304
5 icc -qopenmp -o $_EXEC /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/openmp/src/reduction.c
6 ./$_EXEC
1 #!/bin/bash
2 _EXEC=reduction.c.exe
3 export OMP_NUM_THREADS=4
4 module load gcc/6.1.0
5 gcc -fopenmp -o $_EXEC /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/openmp/src/reduction.c
6 ./$_EXEC
Customize Run Line¶
buildtest will define variable _EXEC
in the job script that can be used to reference
the generated binary. By default, buildtest will run the program standalone, but sometimes you
may want to customize how job is run. This may include passing arguments or running
binary through a job/mpi launcher. The run
property expects user to specify how to launch
program. buildtest will change directory to the called script before running executable. The compiled
executable will be present in local directory which can be accessed via ./$_EXEC
. In example below
we pass arguments 1 3 5
for gcc group and 100 200
for compiler gcc/10.2.0-37fmsw7
.
version: "1.0"
buildspecs:
custom_run_by_compilers:
type: compiler
description: Customize binary launch based on compiler
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/argc.c"
compilers:
name: ["^(builtin_gcc|gcc)"]
default:
gcc:
run: ./$_EXEC 1 3 5
config:
gcc/10.2.0-37fmsw7:
run: ./$_EXEC 100 120
If we build this test and see generated test, we notice buildtest customized the run line
for launching binary. buildtest will directly replace content in run
section into the
shell-script. If no run
field is specified buildtest will run the binary in standalone mode (./$_EXEC
).
1 #!/bin/bash
2
3
4 # name of executable
5 _EXEC=argc.c.exe
6 # Loading modules
7 module load gcc/10.2.0-37fmsw7
8 # Compilation Line
9 gcc -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/argc.c
10
11
12 # Run executable
13 ./$_EXEC 100 120
MPI Example¶
In this example we run a MPI Laplace code using 4 process on a KNL node using
the intel/19.1.2.254
compiler. This test is run on Cori through batch queue
system. We can define #SBATCH parameters using sbatch
property. This program
is compiled using mpiicc
wrapper this can be defined using cc
parameter.
Currently, buildtest cannot detect if program is serial or MPI to infer appropriate
compiler wrapper. If cc
wasn’t specified, buildtest would infer icc as compiler
wrapper for C program. This program is run using srun
job launcher, we can control
how test is executed using the run
property. This test required we swap intel
modules and load impi/2020 module.
1version: "1.0"
2buildspecs:
3 laplace_mpi:
4 type: compiler
5 description: Laplace MPI code in C
6 executor: slurm.knl_debug
7 tags: ["mpi"]
8 source: src/laplace_mpi.c
9 compilers:
10 name: ["^(intel/19.1.2.254)$"]
11 default:
12 all:
13 sbatch: ["-N 1", "-n 4"]
14 run: srun -n 4 $_EXEC
15 intel:
16 cc: mpiicc
17 cflags: -O3
18 config:
19 intel/19.1.2.254:
20 module:
21 load: [impi/2020]
22 swap: [intel, intel/19.1.2.254]
Shown below is a sample build for this buildspec, buildtest will dispatch and poll job until its complete.
$ buildtest build -b buildspecs/apps/mpi/laplace_mpi.yml
User: siddiq90
Hostname: cori02
Platform: Linux
Current Time: 2021/06/11 09:11:16
buildtest path: /global/homes/s/siddiq90/github/buildtest/bin/buildtest
buildtest version: 0.9.5
python path: /global/homes/s/siddiq90/.conda/envs/buildtest/bin/python
python version: 3.8.8
Test Directory: /global/u1/s/siddiq90/github/buildtest/var/tests
Configuration File: /global/u1/s/siddiq90/.buildtest/config.yml
Command: /global/homes/s/siddiq90/github/buildtest/bin/buildtest build -b buildspecs/apps/mpi/laplace_mpi.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
+---------------------------------------------------------------------------------+
| Discovered Buildspecs |
+=================================================================================+
| /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/mpi/laplace_mpi.yml |
+---------------------------------------------------------------------------------+
Discovered Buildspecs: 1
Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+---------------------------------------------------------------------------------
compiler-v1.0.schema.json | True | /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/mpi/laplace_mpi.yml
name description
----------- ---------------------
laplace_mpi Laplace MPI code in C
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | compiler | testpath
-------------+----------+----------+----------------------+---------+------------------+----------------------------------------------------------------------------------------------------------------------
laplace_mpi | a6087b86 | compiler | cori.slurm.knl_debug | ['mpi'] | intel/19.1.2.254 | /global/u1/s/siddiq90/github/buildtest/var/tests/cori.slurm.knl_debug/laplace_mpi/laplace_mpi/0/laplace_mpi_build.sh
+---------------------+
| Stage: Running Test |
+---------------------+
[laplace_mpi] JobID: 43308598 dispatched to scheduler
name | id | executor | status | returncode
-------------+----------+----------------------+----------+--------------
laplace_mpi | a6087b86 | cori.slurm.knl_debug | N/A | -1
Polling Jobs in 30 seconds
________________________________________
Job Queue: [43308598]
Pending Jobs
________________________________________
+-------------+----------------------+----------+-----------+
| name | executor | jobID | jobstate |
+-------------+----------------------+----------+-----------+
| laplace_mpi | cori.slurm.knl_debug | 43308598 | COMPLETED |
+-------------+----------------------+----------+-----------+
Polling Jobs in 30 seconds
________________________________________
Job Queue: []
Completed Jobs
________________________________________
+-------------+----------------------+----------+-----------+
| name | executor | jobID | jobstate |
+-------------+----------------------+----------+-----------+
| laplace_mpi | cori.slurm.knl_debug | 43308598 | COMPLETED |
+-------------+----------------------+----------+-----------+
+---------------------------------------------+
| Stage: Final Results after Polling all Jobs |
+---------------------------------------------+
name | id | executor | status | returncode
-------------+----------+----------------------+----------+--------------
laplace_mpi | a6087b86 | cori.slurm.knl_debug | PASS | 0
+----------------------+
| Stage: Test Summary |
+----------------------+
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Writing Logfile to: /tmp/buildtest_wgptyp8v.log
A copy of logfile can be found at $BUILDTEST_ROOT/buildtest.log - /global/homes/s/siddiq90/github/buildtest/buildtest.log
The generated test is as follows, note that buildtest will insert the #SBATCH directives at the top of script, and module load
are done before module swap
command.
1#!/bin/bash
2#SBATCH -N 1
3#SBATCH -n 4
4#SBATCH --job-name=laplace_mpi
5#SBATCH --output=laplace_mpi.out
6#SBATCH --error=laplace_mpi.err
7_EXEC=laplace_mpi.c.exe
8module load impi/2020
9module swap intel intel/19.1.2.254
10mpiicc -O3 -o $_EXEC /global/u1/s/siddiq90/github/buildtest-cori/buildspecs/apps/mpi/src/laplace_mpi.c
11srun -n 4 $_EXEC
The master script that buildtest will invoke is the following, notice that our generated script (shown above) is invoked via sbatch with its
options. The options sbatch -q debug --clusters=cori -C knl,quad,cache
was inserted by our executor configuration. We add the --parsable
option for Slurm jobs in order to get the JobID when this script is invoked so that buildtest can poll the job.
1#!/bin/bash
2source /global/u1/s/siddiq90/github/buildtest/var/executor/cori.slurm.knl_debug/before_script.sh
3sbatch --parsable -q debug --clusters=cori -C knl,quad,cache /global/u1/s/siddiq90/github/buildtest/var/tests/cori.slurm.knl_debug/laplace_mpi/laplace_mpi/0/stage/laplace_mpi.sh
4returncode=$?
5exit $returncode
Pre/Post sections for build and run section¶
The compiler schema comes with pre_build
, post_build
, pre_run
and
post_run
fields where you can insert commands before and after build
or
run
section. The build section is where we compile code, and run
section is where compiled binary is executed.
Shown below is an example buildspec with pre/post section.
version: "1.0"
buildspecs:
pre_post_build_run:
type: compiler
description: example using pre_build, post_build, pre_run, post_run example
executor: generic.local.bash
tags: [tutorials, compile]
source: "src/hello.c"
compilers:
name: ["^(builtin_gcc)$"]
default:
gcc:
cflags: -Wall
all:
pre_build: |
echo "This is a pre-build section"
gcc --version
post_build: |
echo "This is post-build section"
pre_run: |
echo "This is pre-run section"
export FOO=BAR
post_run: |
echo "This is post-run section"
The format of the test structure is as follows.
#!{shebang path} -- defaults to #!/bin/bash depends on executor name (local.bash, local.sh)
{job directives} -- sbatch or bsub field
{environment variables} -- env field
{variable declaration} -- vars field
{module commands} -- modules field
{pre build commands} -- pre_build field
{compile program} -- build field
{post build commands} -- post_build field
{pre run commands} -- pre_run field
{run executable} -- run field
{post run commands} -- post_run field
The generated test for this buildspec is the following:
#!/bin/bash
# name of executable
_EXEC=hello.c.exe
### START OF PRE BUILD SECTION ###
echo "This is a pre-build section"
gcc --version
### END OF PRE BUILD SECTION ###
# Compilation Line
gcc -Wall -o $_EXEC /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/src/hello.c
### START OF POST BUILD SECTION ###
echo "This is post-build section"
### END OF POST BUILD SECTION ###
### START OF PRE RUN SECTION ###
echo "This is pre-run section"
export FOO=BAR
### END OF PRE RUN SECTION ###
# Run executable
./$_EXEC
### START OF POST RUN SECTION ###
echo "This is post-run section"
### END OF POST RUN SECTION ###