Compiler Schema¶
The compiler schema is used for compilation of programs, currently we support
single source file compilation. In order to use the compiler schema you must set type: compiler
in your
sub-schema. See compiler schema docs
Compilation Examples¶
We assume the reader has basic understanding of Global Schema validation. Shown below is the schema definition for compiler schema:
"$id": "compiler-v1.0.schema.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "compiler schema version 1.0",
"description": "The compiler schema is of ``type: compiler`` in sub-schema which is used for compiling and running programs",
"type": "object",
"required": ["type", "build", "executor"],
"additionalProperties": false,
The required fields for compiler schema are type, build, and executor.
The compiler schema is a JSON object defined by "type": "object"
which is
similar to the script schema.
Shown below are 6 test examples performing Hello World compilation with C, C++, and Fortran using GNU compiler
version: "1.0"
buildspecs:
hello_f:
type: compiler
description: "Hello World Fortran Compilation"
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.f90"
name: gnu
fflags: -Wall
hello_c:
type: compiler
description: "Hello World C Compilation"
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.c"
name: gnu
cflags: -Wall
hello_cplusplus:
type: compiler
description: "Hello World C++ Compilation"
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.cpp"
name: gnu
cxxflags: -Wall
cc_example:
type: compiler
description: Example by using cc to set C compiler
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.c"
name: gnu
cc: gcc
fc_example:
type: compiler
description: Example by using fc to set Fortran compiler
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.f90"
name: gnu
fc: gfortran
cxx_example:
type: compiler
description: Example by using cxx to set C++ compiler
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/hello.cpp"
name: gnu
cxx: g++
The tests hello_f
, hello_c
and hello_cplusplus
rely on buildtest to
detect compiler wrappers while tests cc_example
, fc_example
, cxx_example
rely on user to specify compiler wrappers manually.
The compiler
object is start of compilation section, the required
keys are source
and name
. The source key requires an input program for
compilation, this can be a file relative to buildspec file or an absolute path.
In this example our source examples are in src
directory. The name
field
informs buildtest to auto-detect compiler wrappers (cc
, fc
, cxx
).
The compilation pattern buildtest utilizes is the following:
# C example
$cc $cppflags $cflags -o <executable> $SOURCE $ldflags
# Fortran example
$cxx $cppflags $cxxflags -o <executable> $SOURCE $ldflags
# Fortran example
$fc $cppflags $fflags -o <executable> $SOURCE $ldflags
If you specify cc
, fc
and cxx
field attributes you are responsible for
selecting the correct compiler wrapper. You can use cflags
, cxxflags
and
fflags
field to pass compiler options to C, C++ and Fortran compilers.
Shown below is an example build for the buildspec example
$ buildtest build -b tutorials/compilers/gnu_hello.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
Discovered Buildspecs:
/Users/siddiq90/Documents/buildtest/tutorials/compilers/gnu_hello.yml
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+-----------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/buildtest/tutorials/compilers/gnu_hello.yml
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | testpath
-----------------+----------+----------+------------+--------------------------+--------------------------------------------------------------------------------------------------------
hello_f | 3e4017b8 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_f/3/stage/generate.sh
hello_c | 33ba91d2 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_c/3/stage/generate.sh
hello_cplusplus | b7ffc06c | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_cplusplus/3/stage/generate.sh
cc_example | e565abb3 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/cc_example/3/stage/generate.sh
fc_example | cf7c3505 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/fc_example/3/stage/generate.sh
cxx_example | 6dcf90b8 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/cxx_example/3/stage/generate.sh
+----------------------+
| Stage: Running Test |
+----------------------+
name | id | executor | status | returncode | testpath
-----------------+----------+------------+----------+--------------+--------------------------------------------------------------------------------------------------------
hello_f | 3e4017b8 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_f/3/stage/generate.sh
hello_c | 33ba91d2 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_c/3/stage/generate.sh
hello_cplusplus | b7ffc06c | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/hello_cplusplus/3/stage/generate.sh
cc_example | e565abb3 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/cc_example/3/stage/generate.sh
fc_example | cf7c3505 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/fc_example/3/stage/generate.sh
cxx_example | 6dcf90b8 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/gnu_hello/cxx_example/3/stage/generate.sh
+----------------------+
| Stage: Test Summary |
+----------------------+
Executed 6 tests
Passed Tests: 6/6 Percentage: 100.000%
Failed Tests: 0/6 Percentage: 0.000%
The generated test for test name hello_f is the following:
#!/bin/bash
source /Users/siddiq90/Documents/buildtest/var/executors/local.bash/before_script.sh
gfortran -Wall -o hello.f90.exe src/hello.f90
./hello.f90.exe
source /Users/siddiq90/Documents/buildtest/var/executors/local.bash/after_script.sh
buildtest will fill in the compilation line based on compilation pattern. buildtest, will detect the file extensions and perform a lookup to find the programming language, and finally generate the appropriate C, C++, or Fortran compilation based on language detected.
buildtest detects the programming language and it finds .f90 file extension
and infers it must be Fortran program, hence gfortran
was selected. The
executable name is generated by adding .exe
to end of source file name
so we get hello.f90.exe
. Finally, we run the executable.
File Extension Language Table¶
Shown below is the file extension table for your reference
Language |
File Extension |
---|---|
C |
.c |
C++ |
.cc .cxx .cpp .c++ |
Fortran |
.f90 .F90 .f95 .f .F .FOR .for .FTN .ftn |
Passing Arguments¶
If you want to pass options to executable command use the args
key. Shown
below is an example test
version: "1.0"
buildspecs:
executable_arguments:
type: compiler
description: Passing arguments example
executor: local.bash
tags: [tutorials, compile]
build:
source: "src/argc.c"
name: gnu
cflags: -Wall
run:
args: "1 2 3"
The exec_args will pass options to the executable, use this if your binary requires input arguments. Shown below is a generated test:
#!/bin/bash
gcc -Wall -o argc.c.exe /global/u1/s/siddiq90/tutorials/examples/serial/src/argc.c
./argc.c.exe 1 2 3
OpenMP Example¶
Here is an example OpenMP reduction test that runs on 1 node using 32 tasks on a haswell node:
version: "1.0"
buildspecs:
reduction:
type: compiler
executor: slurm.debug
sbatch: ["-N 1", "--ntasks-per-node 32", "-C haswell", "-t 1"]
module:
- "module load PrgEnv-gnu"
env:
OMP_NUM_THREADS: 32
OMP_PROC_BIND: spread
OMP_PLACES: cores
build:
source: src/reduction.c
name: gnu
cflags: -fopenmp
tags: [openmp]
In this example, we use the SlurmExecutor slurm.debug
, the source file is
src/reduction.c
that is relative to buildspec file. The environment variables
are defined using env
section. To enable openmp flag, for GNU compilers we
pass -fopenmp
to C compiler. Finally we classify this test using tags
key which is set to openmp.
The generated test looks as follows:
#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node 32
#SBATCH -C haswell
#SBATCH -t 1
export OMP_NUM_THREADS=32
export OMP_PROC_BIND=spread
export OMP_PLACES=cores
module load PrgEnv-gnu
gcc -fopenmp -o reduction.c.exe /global/u1/s/siddiq90/buildtest-cori/apps/openmp/src/reduction.c
./reduction.c.exe
MPI Example¶
In this example we run a MPI Laplace code using 4 process on a KNL node using
the module PrgEnv-intel
. The executable is launched using srun
, that
is set via launcher
field. The source code src/laplace_mpi.c
must be run
with 4 process, for this test we allocate 1 node with 4 tasks.
The name
field is a required field, buildtest uses this field to select the
appropriate subclass, when you set name: intel
buildtest will select the IntelCompiler
subclass which sets the cc
, fc
and cxx
variables automatically. If you
want to specify your compiler variables you can use cc
, fc
and cxx
fields
and buildtest will honor your options.
version: "1.0"
buildspecs:
laplace_mpi:
type: compiler
description: Laplace MPI code in C
sbatch: ["-C knl", "-N 1", "-n 4"]
executor: slurm.debug
tags: ["mpi"]
module:
- "module load PrgEnv-intel"
build:
name: intel
source: src/laplace_mpi.c
cflags: -O3
run:
launcher: srun -n 4
The generated test is as follows:
#!/bin/bash
#SBATCH -C knl
#SBATCH -N 1
#SBATCH -n 4
module load PrgEnv-intel
icc -O3 -o laplace_mpi.c.exe /global/u1/s/siddiq90/buildtest-cori/apps/mpi/src/laplace_mpi.c
srun -n 4 ./laplace_mpi.c.exe
Shown below is a sample build for this buildspec:
$ buildtest build -b mpi/laplace_mpi.yml
Paths:
__________
Prefix: /global/u1/s/siddiq90/cache
Buildspec Search Path: ['/global/u1/s/siddiq90/buildtest/tutorials']
Test Directory: /global/u1/s/siddiq90/cache/tests
+-------------------------------+
| Stage: Discovered Buildspecs |
+-------------------------------+
/global/u1/s/siddiq90/buildtest-cori/apps/mpi/laplace_mpi.yml
+----------------------+
| Stage: Building Test |
+----------------------+
Name | Schema File | Test Path | Buildspec
-------------+---------------------------+--------------------------------------------------------------+---------------------------------------------------------------
laplace_mpi | compiler-v1.0.schema.json | /global/u1/s/siddiq90/cache/tests/laplace_mpi/laplace_mpi.sh | /global/u1/s/siddiq90/buildtest-cori/apps/mpi/laplace_mpi.yml
+----------------------+
| Stage: Running Test |
+----------------------+
[laplace_mpi] job dispatched to scheduler
[laplace_mpi] acquiring job id in 2 seconds
name | executor | status | returncode | testpath
-------------+-------------+----------+--------------+--------------------------------------------------------------
laplace_mpi | slurm.debug | N/A | 0 | /global/u1/s/siddiq90/cache/tests/laplace_mpi/laplace_mpi.sh
Polling Jobs in 10 seconds
________________________________________
[laplace_mpi]: JobID 33306420 in COMPLETED state
Polling Jobs in 10 seconds
________________________________________
+---------------------------------------------+
| Stage: Final Results after Polling all Jobs |
+---------------------------------------------+
name | executor | status | returncode | testpath
-------------+-------------+----------+--------------+--------------------------------------------------------------
laplace_mpi | slurm.debug | PASS | 0 | /global/u1/s/siddiq90/cache/tests/laplace_mpi/laplace_mpi.sh
+----------------------+
| Stage: Test Summary |
+----------------------+
Executed 1 tests
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
OpenACC Examples¶
Next, we will make use of an OpenACC vector addition example shown below is an example test
version: "1.0"
buildspecs:
vecadd_gnu:
type: compiler
description: Vector Addition example with GNU compiler
tags: [tutorials, compile]
executor: local.bash
build:
name: gnu
source: src/vecAdd.c
cflags: -fopenacc
ldflags: -lm
status:
regex:
stream: stdout
exp: "^final result: 1.000000$"
To compile OpenACC program with gnu compiler we must use -fopenacc
flag, this
program requires linking with math library so we can specify linker flags (ldflags)
using ldflags: -lm
.
The output of this test will generate a single line output as follows:
final result: 1.000000
The status
field with regex
is used for checking output stream using stream: stdout
and exp
key to specify regular expression to use. If we are to build this test,
you will notice the run section will have a Status of PASS
$ buildtest build -b tutorials/compilers/vecadd.yml
+-------------------------------+
| Stage: Discovering Buildspecs |
+-------------------------------+
Discovered Buildspecs:
/Users/siddiq90/Documents/buildtest/tutorials/compilers/vecadd.yml
+---------------------------+
| Stage: Parsing Buildspecs |
+---------------------------+
schemafile | validstate | buildspec
---------------------------+--------------+--------------------------------------------------------------------
compiler-v1.0.schema.json | True | /Users/siddiq90/Documents/buildtest/tutorials/compilers/vecadd.yml
+----------------------+
| Stage: Building Test |
+----------------------+
name | id | type | executor | tags | testpath
------------+----------+----------+------------+--------------------------+------------------------------------------------------------------------------------------------
vecadd_gnu | 1a0f6265 | compiler | local.bash | ['tutorials', 'compile'] | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/vecadd/vecadd_gnu/3/stage/generate.sh
+----------------------+
| Stage: Running Test |
+----------------------+
name | id | executor | status | returncode | testpath
------------+----------+------------+----------+--------------+------------------------------------------------------------------------------------------------
vecadd_gnu | 1a0f6265 | local.bash | PASS | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/vecadd/vecadd_gnu/3/stage/generate.sh
+----------------------+
| Stage: Test Summary |
+----------------------+
Executed 1 tests
Passed Tests: 0/1 Percentage: 0.000%
Failed Tests: 1/1 Percentage: 100.000%
The regular expression is performed using re.search, for example if we can change
the exp
field as follows:
exp: "^final result: 0.99$"
Next if we re-run test we will notice the Status is FAIL
even though we
have a Return Code of 0:
+----------------------+
| Stage: Running Test |
+----------------------+
name | executor | status | returncode | testpath
------------+------------+----------+--------------+------------------------------------------------------------------------------------------
vecadd_gnu | local.bash | FAIL | 0 | /Users/siddiq90/Documents/buildtest/var/tests/local.bash/vecadd/vecadd_gnu/run_script.sh
In the next example, we extend the previous buildspec test to run at Cori GPU
machine using Slurm scheduler. We use the executor slurm.gpu
where our executor
is defined as follows:
gpu:
description: submit jobs to GPU partition
options: ["-C gpu"]
cluster: escori
In order to submit job to the Cori GPU cluster we must use sbatch -C gpu -M escori
which
is what slurm.gpu
executor is doing.
In this example we make use of module
field to load modules into the test, for
this test we load the modules cuda
and gcc/8.1.1-openacc-gcc-8-branch-20190215
.
This test will launch job via srun
and check job state code is COMPLETED
.
version: "1.0"
buildspecs:
vecadd_openacc_gnu:
type: compiler
description: Vector Addition example with GNU compiler
executor: slurm.gpu
sbatch: ["-G 1", "-t 5", "-N 1"]
module:
- "module load cuda"
- "module load gcc/8.1.1-openacc-gcc-8-branch-20190215"
build:
name: gnu
source: src/vecAdd.c
cflags: -fopenacc
ldflags: -lm
run:
launcher: srun
status:
slurm_job_state: COMPLETED
buildtest will generate the following test, buildtest will add the #SBATCH directives
followed by module commands. The executable is run via srun
because we specify the launcher
field.
#!/bin/bash
#SBATCH -G 1
#SBATCH -t 5
#SBATCH -N 1
module load cuda
module load gcc/8.1.1-openacc-gcc-8-branch-20190215
gcc -fopenacc -o vecAdd.c.exe /global/u1/s/siddiq90/buildtest-cori/apps/openacc/src/vecAdd.c -lm
srun ./vecAdd.c.exe
In this next example, we build same test using hpcsdk
compiler by NVIDIA that recently acquired PGI compiler. At Cori, we must load hpcsdk
and cuda
module in order to use the hpcsdk compiler. The name
is a
required field however buildtest will ignore since we specify
cc
field. NVIDIA changed their compiler names instead of pgcc
we must use
nvc
with flag -acc
to offload to GPU. For CoriGPU we must use
srun
to acquire GPU access hence launcher
field is set to srun.
version: "1.0"
buildspecs:
vecadd_hpcsdk_gnu:
type: compiler
description: Vector Addition example with hpcsdk (pgi) compiler
executor: slurm.gpu
sbatch: ["-G 1", "-t 5", "-N 1"]
module:
- "module load hpcsdk"
- "module load cuda"
build:
name: pgi
cc: nvc
source: src/vecAdd.c
cflags: -acc
ldflags: -lm
run:
launcher: srun
Pre/Post sections for build and run section¶
The compiler schema comes with pre_build
, post_build
, pre_run
and
post_run
fields where you can insert commands before and after build
or
run
section. The build section is where we compile code, and run
section is where compiled binary is executed.
Shown below is an example buildspec with pre/post section.
version: "1.0"
buildspecs:
pre_post_build_run:
type: compiler
description: example using pre_build, post_build, pre_run, post_run example
executor: local.bash
tags: [tutorials, compile]
pre_build: |
echo "This is a pre-build section"
gcc --version
build:
source: "src/hello.c"
name: gnu
cflags: -Wall
post_build: |
echo "This is post-build section"
pre_run: |
echo "This is pre-run section"
export FOO=BAR
post_run: |
echo "This is post-run section"
The format of the test structure is the following:
#!{shebang path} -- defaults to #!/bin/bash depends on executor name (local.bash, local.sh)
{job directives} -- sbatch or bsub field
{environment variables} -- env field
{variable declaration} -- vars field
{module commands} -- modules field
{pre build commands} -- pre_build field
{compile program} -- build field
{post build commands} -- post_build field
{pre run commands} -- pre_run field
{run executable} -- run field
{post run commands} -- post_run field
The generated test for this buildspec is the following:
#!/bin/bash
echo "This is a pre-build section"
gcc --version
gcc -Wall -o hello.c.exe /Users/siddiq90/Documents/buildtest/tutorials/compilers/src/hello.c
echo "This is post-build section"
echo "This is pre-run section"
export FOO=BAR
./hello.c.exe
echo "This is post-run section"