Buildtest Tutorial on AWS
Setup
This section of tutorial will be completed in Amazon Web Services (AWS) environment. Once you have access to system, please Installing buildtest and then proceed to the next section.
Once you are done, please run the following commands to ensure you are using the correct configuration file
export BUILDTEST_CONFIGFILE=$BUILDTEST_ROOT/buildtest/settings/aws.yml
buildtest config path
You can verify the configuration is valid by runnning the following:
buildtest config validate
If you see no errors, then you are ready to proceed to the next section.
Hello World Compilation
Let’s start off with a simple hello world compilation in C using the GNU compiler. We have the following buildspec
that will compile a code source code hello.c using the gcc compiler wrapper.
buildspecs:
hello_world_example:
type: script
executor: generic.local.bash
description: Hello world compilation in C
run: |
gcc hello.c -o hello
./hello
The source code is the following
#include <stdio.h>
int main() {
printf("Hello, World in C\n");
return 0;
}
Let’s try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/hello.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/hello.yml
Buildspec Paths: ['/home/lbladmin/Documents/buildtest/aws_tutorial']
Updating buildspec cache file: /home/lbladmin/Documents/buildtest/var/buildspecs/cache.json
╭───────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:20 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/hello.yml │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═══════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟───────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/hello.yml ║
╟───────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═══════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/hello.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_example/516718ee │ script │ generic.local.bash │ None │ None │ None │ Hello world compilation in C │ /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/hello.yml │
└──────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────┴───────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_example/516718ee: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_example/516718ee does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_example/516718ee │
└──────────────────────────────┘
hello_world_example/516718ee: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/stage
hello_world_example/516718ee: Running Test via command: bash hello_world_example_build.sh
hello_world_example/516718ee: Test completed in 0.050824 seconds with returncode: 0
hello_world_example/516718ee: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example.out
hello_world_example/516718ee: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_example/516718ee │ generic.local.bash │ PASS │ 0 │ 0.051 │
└──────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_k4_6t_re.log
$ buildtest inspect query -o -t hello_world_example
──────────────────────────────────────────────────────────────────────────────── hello_world_example/516718ee-aafe-4807-9ac1-cc2b3f1065c2 ─────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C
State: PASS
Returncode: 0
Runtime: 0.050824 sec
Starttime: 2024/06/17 18:50:20
Endtime: 2024/06/17 18:50:20
Command: bash hello_world_example_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello/hello_world_example/516718ee/hello_world_example.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_k4_6t_re.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello, World in C
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
# Content of run section
gcc hello.c -o hello
./hello
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Multi Compiler Test
Buildtest supports multiple test creation based on compiler selection using compilers property. This can be useful to test
a single test with multiple compilers. In this next example, we will attempt to compile the Hello World test in previous example using
2 versions of GNU compiler. The name property is used to search for compilers in the configuration file. The Defining Compilers section
covers in detail how to define compilers in buildtest configuration.
buildspecs:
hello_world_multi_compiler:
type: script
executor: generic.local.bash
description: Hello world compilation in C, C++ and Fortran with multiple compilers
compilers:
name: ['gcc']
run: |
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
Let’s try building this example, you will see there are now 2 tests created, one for each compiler.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/multi_compiler_hello.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml
╭─────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:22 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/multi_compil │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml ║
╟──────────────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_multi_compiler/9b1c9f5b │ script │ generic.local.bash │ gcc_11.4.0 │ None │ None │ Hello world compilation in C, C++ and Fortran with │ /home/lbladmin/Documents/buildtest/aws_tutorial/hello_worl │
│ │ │ │ │ │ │ multiple compilers │ d/multi_compiler_hello.yml │
├─────────────────────────────────────┼────────┼────────────────────┼────────────┼───────┼───────┼───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ hello_world_multi_compiler/b43967d7 │ script │ generic.local.bash │ gcc_12.3.0 │ None │ None │ Hello world compilation in C, C++ and Fortran with │ /home/lbladmin/Documents/buildtest/aws_tutorial/hello_worl │
│ │ │ │ │ │ │ multiple compilers │ d/multi_compiler_hello.yml │
└─────────────────────────────────────┴────────┴────────────────────┴────────────┴───────┴───────┴───────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_multi_compiler/9b1c9f5b: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b
hello_world_multi_compiler/b43967d7: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_multi_compiler/b43967d7 does not have any dependencies adding test to queue
hello_world_multi_compiler/9b1c9f5b does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_multi_compiler/b43967d7 │
│ hello_world_multi_compiler/9b1c9f5b │
└─────────────────────────────────────┘
hello_world_multi_compiler/b43967d7: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/stage
hello_world_multi_compiler/9b1c9f5b: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/stage
hello_world_multi_compiler/b43967d7: Running Test via command: bash hello_world_multi_compiler_build.sh
hello_world_multi_compiler/9b1c9f5b: Running Test via command: bash hello_world_multi_compiler_build.sh
hello_world_multi_compiler/9b1c9f5b: Test completed in 0.494109 seconds with returncode: 0
hello_world_multi_compiler/9b1c9f5b: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler.out
hello_world_multi_compiler/9b1c9f5b: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler.err
hello_world_multi_compiler/b43967d7: Test completed in 0.574126 seconds with returncode: 0
hello_world_multi_compiler/b43967d7: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler.out
hello_world_multi_compiler/b43967d7: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_multi_compiler/b43967d7 │ generic.local.bash │ PASS │ 0 │ 0.574 │
├─────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ hello_world_multi_compiler/9b1c9f5b │ generic.local.bash │ PASS │ 0 │ 0.494 │
└─────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_e7uu69js.log
Take note in the generated test output, the variables BUILDTEST_CC, BUILDTEST_CXX and BUILDTEST_FC are using the compiler wrappers for each compiler.
$ buildtest inspect query -o -t hello_world_multi_compiler/
───────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/b43967d7-d27a-47c5-8737-a14a7c8bbbef ─────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 0.574126 sec
Starttime: 2024/06/17 18:50:22
Endtime: 2024/06/17 18:50:22
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/b43967d7/hello_world_multi_compiler.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_e7uu69js.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
BUILDTEST_CC="/usr/bin/gcc-12"
BUILDTEST_CXX="/usr/bin/g++-12"
BUILDTEST_FC="/usr/bin/gfortran-12"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/9b1c9f5b-b8c6-46c9-a664-19f7fa5b6c2c ─────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 0.494109 sec
Starttime: 2024/06/17 18:50:22
Endtime: 2024/06/17 18:50:22
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/9b1c9f5b/hello_world_multi_compiler.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_e7uu69js.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
BUILDTEST_CC="/usr/bin/gcc"
BUILDTEST_CXX="/usr/bin/g++"
BUILDTEST_FC="/usr/bin/gfortran"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
We can see the compiler configuration using the command buildtest config compilers list --yaml which will print the output in YAML format.
Shown below is the compiler declaration
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml config compilers list --yaml
gcc:
gcc_11.4.0:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
gcc_12.3.0:
cc: /usr/bin/gcc-12
cxx: /usr/bin/g++-12
fc: /usr/bin/gfortran-12
OpenMP Test with custom compiler configuration
In this next example, we will compile a OpenMP code using GNU compiler and specify custom compiler flags and environment variable. We will work with a Hello World OpenMP code that uses OpenMP pragma to parallelize the code. The source code is the following
// OpenMP program to print Hello World
// using C language
// OpenMP header
#include <omp.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[])
{
// Beginning of parallel region
#pragma omp parallel
{
printf("Hello World... from thread = %d\n",
omp_get_thread_num());
}
// Ending of parallel region
}
In order to build OpenMP code, we need to use -fopenmp flag to enable OpenMP support. We will set OMP_NUM_THREADS environment variable
to specify number of OpenMP theads which will differ for each compiler test. We introduce a keyword config that allows us to specify
customize compiler flags and environment variables. The env property is used to set environment variable for each compiler test and cflags is
used to set compiler flags inorder to compile the test. This value is stored in environment BUILDTEST_CFLAGS that will be used in run section
for compiling the source code.
buildspecs:
hello_world_openmp_custom_compiler:
type: script
executor: 'generic.local.bash'
description: Hello World OpenMP example with custom compiler settings
compilers:
name: ["gcc"]
config:
gcc_12.3.0:
env:
OMP_NUM_THREADS: "2"
cflags: "-O1 -fopenmp"
gcc_11.4.0:
env:
OMP_NUM_THREADS: "6"
cflags: "-O2 -fopenmp"
run: |
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
Let’s try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/openmp_example_custom_compiler.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/openmp_example_custom_compiler.yml
╭─────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:29 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/openmp_example_custom_co │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔════════════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟────────────────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/openmp_example_custom_compiler.yml ║
╟────────────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚════════════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/openmp_example_custom_compiler.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/2e7a2ed5 │ script │ generic.local.bash │ gcc_11.4.0 │ None │ None │ Hello World OpenMP example with custom compiler │ /home/lbladmin/Documents/buildtest/aws_tutorial/openm │
│ │ │ │ │ │ │ settings │ p_example_custom_compiler.yml │
├─────────────────────────────────────────────┼────────┼────────────────────┼────────────┼───────┼───────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ hello_world_openmp_custom_compiler/bedcd5b5 │ script │ generic.local.bash │ gcc_12.3.0 │ None │ None │ Hello World OpenMP example with custom compiler │ /home/lbladmin/Documents/buildtest/aws_tutorial/openm │
│ │ │ │ │ │ │ settings │ p_example_custom_compiler.yml │
└─────────────────────────────────────────────┴────────┴────────────────────┴────────────┴───────┴───────┴────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_openmp_custom_compiler/2e7a2ed5: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5
hello_world_openmp_custom_compiler/bedcd5b5: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_openmp_custom_compiler/2e7a2ed5 does not have any dependencies adding test to queue
hello_world_openmp_custom_compiler/bedcd5b5 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/2e7a2ed5 │
│ hello_world_openmp_custom_compiler/bedcd5b5 │
└─────────────────────────────────────────────┘
hello_world_openmp_custom_compiler/bedcd5b5: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/stage
hello_world_openmp_custom_compiler/2e7a2ed5: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/stage
hello_world_openmp_custom_compiler/2e7a2ed5: Running Test via command: bash hello_world_openmp_custom_compiler_build.sh
hello_world_openmp_custom_compiler/bedcd5b5: Running Test via command: bash hello_world_openmp_custom_compiler_build.sh
hello_world_openmp_custom_compiler/2e7a2ed5: Test completed in 0.066805 seconds with returncode: 0
hello_world_openmp_custom_compiler/2e7a2ed5: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler.out
hello_world_openmp_custom_compiler/2e7a2ed5: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler.err
hello_world_openmp_custom_compiler/bedcd5b5: Test completed in 0.070316 seconds with returncode: 0
hello_world_openmp_custom_compiler/bedcd5b5: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler.out
hello_world_openmp_custom_compiler/bedcd5b5: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/2e7a2ed5 │ generic.local.bash │ PASS │ 0 │ 0.067 │
├─────────────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ hello_world_openmp_custom_compiler/bedcd5b5 │ generic.local.bash │ PASS │ 0 │ 0.070 │
└─────────────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_x83ex5kh.log
We will see in the generated test the values OMP_NUM_THREADS and BUILDTEST_CFLAGS are set for each compiler test. The OMP_NUM_THREADS
will impact the number of threads used to run code therefore we will see different output for each compiler test.
$ buildtest inspect query -o -t hello_world_openmp_custom_compiler/
───────────────────────────────────────────────────────────────────────── hello_world_openmp_custom_compiler/bedcd5b5-5820-4c7e-830a-4997cb27d4f2 ─────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello World OpenMP example with custom compiler settings
State: PASS
Returncode: 0
Runtime: 0.070316 sec
Starttime: 2024/06/17 18:50:29
Endtime: 2024/06/17 18:50:29
Command: bash hello_world_openmp_custom_compiler_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/bedcd5b5/hello_world_openmp_custom_compiler.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_x83ex5kh.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello World... from thread = 0
Hello World... from thread = 1
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
BUILDTEST_CC="/usr/bin/gcc-12"
BUILDTEST_CXX="/usr/bin/g++-12"
BUILDTEST_FC="/usr/bin/gfortran-12"
BUILDTEST_CFLAGS="-O1 -fopenmp"
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
export OMP_NUM_THREADS="2"
# Content of run section
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────────────────────────────────────────────────────────────────────── hello_world_openmp_custom_compiler/2e7a2ed5-465d-4d27-885a-58c7800a0e3e ─────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello World OpenMP example with custom compiler settings
State: PASS
Returncode: 0
Runtime: 0.066805 sec
Starttime: 2024/06/17 18:50:29
Endtime: 2024/06/17 18:50:29
Command: bash hello_world_openmp_custom_compiler_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/2e7a2ed5/hello_world_openmp_custom_compiler.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_x83ex5kh.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello World... from thread = 3
Hello World... from thread = 5
Hello World... from thread = 2
Hello World... from thread = 4
Hello World... from thread = 1
Hello World... from thread = 0
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
BUILDTEST_CC="/usr/bin/gcc"
BUILDTEST_CXX="/usr/bin/g++"
BUILDTEST_FC="/usr/bin/gfortran"
BUILDTEST_CFLAGS="-O2 -fopenmp"
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
export OMP_NUM_THREADS="6"
# Content of run section
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Testing a MPI Code
This image comes with several MPI flavors such as OpenMPI, mvapich2. In the next example we will run a MPI Proc Name test
on login node that will run with 8 processes. The source code is available in the $HOME/examples directory as part of the image.
buildspecs:
mpiprocname:
type: script
executor: generic.local.bash
description: Run MPI Proc Name test
run: |
cd $HOME/examples/mpi-procname
mpicc -o mpiprocname mpiprocname.c
mpirun -np 8 ./mpiprocname
rm mpiprocname
We can run this test using the following command
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/mpiproc.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/mpiproc.yml
╭──────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:24 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/mpiproc.yml │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/mpiproc.yml ║
╟─────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/mpiproc.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ mpiprocname/7801542b │ script │ generic.local.bash │ None │ None │ None │ Run MPI Proc Name test │ /home/lbladmin/Documents/buildtest/aws_tutorial/mpiproc.yml │
└──────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴────────────────────────┴─────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
mpiprocname/7801542b: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
mpiprocname/7801542b does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ mpiprocname/7801542b │
└──────────────────────┘
mpiprocname/7801542b: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/stage
mpiprocname/7801542b: Running Test via command: bash mpiprocname_build.sh
mpiprocname/7801542b: Test completed in 0.844294 seconds with returncode: 0
mpiprocname/7801542b: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname.out
mpiprocname/7801542b: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ mpiprocname/7801542b │ generic.local.bash │ PASS │ 0 │ 0.844 │
└──────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_kbnu_bz5.log
We can see this test prints Hello World message from each process.
$ buildtest inspect query -o mpiprocname
──────────────────────────────────────────────────────────────────────────────────── mpiprocname/7801542b-a666-4b9b-aff6-81dcd5a6ac4d ─────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run MPI Proc Name test
State: PASS
Returncode: 0
Runtime: 0.844294 sec
Starttime: 2024/06/17 18:50:24
Endtime: 2024/06/17 18:50:25
Command: bash mpiprocname_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/7801542b/mpiprocname.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_kbnu_bz5.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello, world. I am 2 of 8 on e4spro-cluster
Hello, world. I am 3 of 8 on e4spro-cluster
Hello, world. I am 0 of 8 on e4spro-cluster
Hello, world. I am 4 of 8 on e4spro-cluster
Hello, world. I am 1 of 8 on e4spro-cluster
Hello, world. I am 7 of 8 on e4spro-cluster
Hello, world. I am 5 of 8 on e4spro-cluster
Hello, world. I am 6 of 8 on e4spro-cluster
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
MPI Job Submission
This cluster comes with PBS/Torque scheduler. We can run the MPI test to batch scheduler. Shown below is an example
buildspec. We will use an executor named a torque executor named generic.torque.e4spro-cluster that is mapped to
a queue named e4spro-cluster. The pbs property is used to specify PBS directives. We will run this test
on a single node using 2 processors with a wall time of 1hr.
buildspecs:
mpi_job_submission:
type: script
executor: generic.torque.e4spro-cluster
description: Run MPI Proc Name test
pbs: ["-l nodes=1:ppn=2,walltime=1:00"]
run: |
cd $HOME/examples/mpi-procname
mpicc -o mpiprocname mpiprocname.c
mpirun -np 2 ./mpiprocname
rm mpiprocname
We can run this test by running the following commands. The --pollinterval 10 will be used to poll job every 10 sec and
retrieve job status. Buildtest will keep polling job until job is complete. The --display output --display test will show
content of output, error and generated test files during the build phase. You will notice the generated build script (_build.sh)
will invoke qsub command to a generated test.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/mpi_job_submission.yml --pollinterval 10 --display output --display test
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml --pollinterval=10 --display output --display test
╭─────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:35 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml - │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml ║
╟────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ mpi_job_submission/25048a6e │ script │ generic.torque.e4spro-cluster │ None │ None │ None │ Run MPI Proc Name test │ /home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml │
└─────────────────────────────┴────────┴───────────────────────────────┴──────────┴───────┴───────┴────────────────────────┴────────────────────────────────────────────────────────────────────────┘
Batch Job Builders
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ executor ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ mpi_job_submission/25048a6e │ generic.torque.e4spro-cluster │ /home/lbladmin/Documents/buildtest/aws_tutorial/mpi_job_submission.yml │
└─────────────────────────────┴───────────────────────────────┴────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
mpi_job_submission/25048a6e: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e
──────────────────────────────────────────────────────────────────────────────────── mpi_job_submission/25048a6e: Start of Test Script ────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
#PBS -l nodes=1:ppn=2,walltime=1:00
#PBS -N mpi_job_submission
#PBS -o mpi_job_submission.o
#PBS -e mpi_job_submission.e
# Content of run section
cd $HOME/examples/mpi-procname
mpicc -o mpiprocname mpiprocname.c
mpirun -np 2 ./mpiprocname
rm mpiprocname
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────── mpi_job_submission/25048a6e: Start of Build Script ────────────────────────────────────────────────────────────────────────────────────
#!/bin/bash
# Function to handle all signals and perform cleanup
function cleanup() {
echo "Signal trapped. Performing cleanup before exiting."
exitcode=$?
echo "buildtest: command \`$BASH_COMMAND' failed (exit code: $exitcode)"
exit $exitcode
}
# Trap all signals and call the cleanup function
trap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU
export BUILDTEST_TEST_NAME=mpi_job_submission
export BUILDTEST_TEST_ROOT=/home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e
export BUILDTEST_BUILDSPEC_DIR=/home/lbladmin/Documents/buildtest/aws_tutorial
export BUILDTEST_STAGE_DIR=/home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e/stage
# source executor startup script
source /home/lbladmin/Documents/buildtest/var/executor/generic.torque.e4spro-cluster/before_script.sh
# Run generated script
/usr/local/bin/qsub -q e4spro-cluster /home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e/stage/mpi_job_submission.sh
# Get return code
returncode=$?
# Exit with return code
exit $returncode
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
mpi_job_submission/25048a6e does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ mpi_job_submission/25048a6e │
└─────────────────────────────┘
mpi_job_submission/25048a6e: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e/stage
mpi_job_submission/25048a6e: Running Test via command: bash mpi_job_submission_build.sh
────────────────────────────────────────────────────────────────────────────────────── mpi_job_submission/25048a6e: Start of Output ───────────────────────────────────────────────────────────────────────────────────────
3290003.e4spro-cluster
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
mpi_job_submission/25048a6e: JobID: 3290003.e4spro-cluster dispatched to scheduler
Polling Jobs in 10 seconds
Pending and Suspended Jobs (1)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ builder ┃ executor ┃ jobid ┃ jobstate ┃ runtime ┃ elapsedtime ┃ pendtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ mpi_job_submission/25048a6e │ generic.torque.e4spro-cluster │ 3290003.e4spro-cluster │ Q │ 10.046 │ 0 │ 10.03 │
└─────────────────────────────┴───────────────────────────────┴────────────────────────┴──────────┴─────────┴─────────────┴──────────┘
Polling Jobs in 10 seconds
mpi_job_submission/25048a6e: Job 3290003.e4spro-cluster is complete!
mpi_job_submission/25048a6e: Test completed in 0 seconds with returncode: 0
mpi_job_submission/25048a6e: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e/mpi_job_submission.o
mpi_job_submission/25048a6e: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.torque.e4spro-cluster/mpi_job_submission/mpi_job_submission/25048a6e/mpi_job_submission.e
Completed Jobs (1)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ builder ┃ executor ┃ jobid ┃ jobstate ┃ runtime ┃ elapsedtime ┃ pendtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ mpi_job_submission/25048a6e │ generic.torque.e4spro-cluster │ 3290003.e4spro-cluster │ C │ 0 │ 0 │ 10.03 │
└─────────────────────────────┴───────────────────────────────┴────────────────────────┴──────────┴─────────┴─────────────┴──────────┘
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ mpi_job_submission/25048a6e │ generic.torque.e4spro-cluster │ PASS │ 0 │ 0.000 │
└─────────────────────────────┴───────────────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_z6rcivu8.log
OSU Microbenchmark
The OSU Microbenchmark is a collection of MPI-based benchmarks developed at Ohio State University.
This benchmark is used to measure performance of MPI libraries and is a popular benchmark suite for MPI. We will run the osu_bw test
which measures bandwidth. The test requires we specify 2 processes.
In this next example, we will run 2 tests, the first is simply invoking the osu_bw test with 2 processes, and the second test will
run the same test and use comparison_operators to compare the bandwidth result. The metrics property is used to
capture performance metrics that can be used for comparison. We will use the assert_ge status check that
will do a greater than or equal comparison with reference value. The metrics will capture performance results for message length 16384 and
perform a comparison with reference value of 10000. If the test result is greater than or equal to 10000, the test will pass otherwise it will fail.
buildspecs:
osu_bandwidth:
type: script
executor: generic.local.bash
description: Run OSU Bandwidth Test
run: mpirun -np 2 osu_bw
osu_bandwidth_perf:
type: script
executor: generic.local.bash
description: Run OSU Bandwidth Performance Test
run: mpirun -np 2 osu_bw
metrics:
osu_bw:
type: float
regex:
exp: '^16384\s+([\d.]+)$'
stream: stdout
item: 1
status:
assert_ge:
comparisons:
- name: osu_bw
ref: 10000
Let’s run the test and see the results.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/osu_bandwidth_test.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml
╭────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:26 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml ║
╟────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ osu_bandwidth/5b6a6022 │ script │ generic.local.bash │ None │ None │ None │ Run OSU Bandwidth Test │ /home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml │
├─────────────────────────────┼────────┼────────────────────┼──────────┼───────┼───────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ osu_bandwidth_perf/7b147273 │ script │ generic.local.bash │ None │ None │ None │ Run OSU Bandwidth Performance Test │ /home/lbladmin/Documents/buildtest/aws_tutorial/osu_bandwidth_test.yml │
└─────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴────────────────────────────────────┴────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
osu_bandwidth/5b6a6022: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022
osu_bandwidth_perf/7b147273: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
osu_bandwidth/5b6a6022 does not have any dependencies adding test to queue
osu_bandwidth_perf/7b147273 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ osu_bandwidth/5b6a6022 │
│ osu_bandwidth_perf/7b147273 │
└─────────────────────────────┘
osu_bandwidth/5b6a6022: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/stage
osu_bandwidth_perf/7b147273: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/stage
osu_bandwidth_perf/7b147273: Running Test via command: bash osu_bandwidth_perf_build.sh
osu_bandwidth/5b6a6022: Running Test via command: bash osu_bandwidth_build.sh
osu_bandwidth_perf/7b147273: Test completed in 1.066617 seconds with returncode: 0
osu_bandwidth_perf/7b147273: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf.out
osu_bandwidth_perf/7b147273: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf.err
osu_bandwidth_perf/7b147273: testing metric: osu_bw if 9291.32 >= 10000.0 - Check: False
osu_bandwidth_perf/7b147273: ge check: False
osu_bandwidth/5b6a6022: Test completed in 1.075386 seconds with returncode: 0
osu_bandwidth/5b6a6022: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth.out
osu_bandwidth/5b6a6022: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ osu_bandwidth_perf/7b147273 │ generic.local.bash │ FAIL │ 0 │ 1.067 │
├─────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ osu_bandwidth/5b6a6022 │ generic.local.bash │ PASS │ 0 │ 1.075 │
└─────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%
Adding 2 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_ay4679yy.log
Take note in the output of the second test, we will see a list of performance metrics captured in table output. If metrics is not captured, the value will be undefined.
$ buildtest inspect query -o osu_bandwidth osu_bandwidth_perf
─────────────────────────────────────────────────────────────────────────────────── osu_bandwidth/5b6a6022-53b8-449c-adda-5f1341711fdd ────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run OSU Bandwidth Test
State: PASS
Returncode: 0
Runtime: 1.075386 sec
Starttime: 2024/06/17 18:50:26
Endtime: 2024/06/17 18:50:27
Command: bash osu_bandwidth_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/5b6a6022/osu_bandwidth.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_ay4679yy.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
# OSU MPI Bandwidth Test v7.3
# Size Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1 5.77
2 11.10
4 18.22
8 44.51
16 86.73
32 175.56
64 341.96
128 611.27
256 1413.40
512 2440.32
1024 3179.99
2048 4395.94
4096 6680.31
8192 7974.70
16384 8754.07
32768 9285.44
65536 16754.46
131072 16720.23
262144 13243.39
524288 12519.46
1048576 12644.59
2097152 12278.21
4194304 12180.92
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────────────────────────────────────────────────────────────────────────────── osu_bandwidth_perf/7b147273-72e1-4439-9b5f-f6a64ac4cef0 ─────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run OSU Bandwidth Performance Test
State: FAIL
Returncode: 0
Runtime: 1.066617 sec
Starttime: 2024/06/17 18:50:26
Endtime: 2024/06/17 18:50:27
Command: bash osu_bandwidth_perf_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/7b147273/osu_bandwidth_perf.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_ay4679yy.log
Metrics
┏━━━━━━━━┳━━━━━━━━━┓
┃ Name ┃ Value ┃
┡━━━━━━━━╇━━━━━━━━━┩
│ osu_bw │ 9291.32 │
└────────┴─────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
# OSU MPI Bandwidth Test v7.3
# Size Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1 5.81
2 10.66
4 22.52
8 45.15
16 92.11
32 180.50
64 362.67
128 712.86
256 1262.76
512 2401.10
1024 3307.80
2048 4857.36
4096 6495.22
8192 8303.84
16384 9291.32
32768 9459.18
65536 12795.71
131072 16870.14
262144 13644.36
524288 12525.87
1048576 12634.52
2097152 12676.00
4194304 12385.52
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Tensorflow Test
In this next exercise, we provide a tensorflow test in python that train a model and generate a predicition model. We have the following source code
from tensorflow import keras
from tensorflow.keras import layers
print("Testing TensorFlow")
inputs = keras.Input(shape=(784,), name="digits")
x = layers.Dense(64, activation="relu", name="dense_1")(inputs)
x = layers.Dense(64, activation="relu", name="dense_2")(x)
outputs = layers.Dense(10, activation="softmax", name="predictions")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
# Load a toy dataset for the sake of this example
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
# Preprocess the data (these are Numpy arrays)
x_train = x_train.reshape(60000, 784).astype("float32") / 255
x_test = x_test.reshape(10000, 784).astype("float32") / 255
y_train = y_train.astype("float32")
y_test = y_test.astype("float32")
# Reserve 10,000 samples for validation
x_val = x_train[-10000:]
y_val = y_train[-10000:]
x_train = x_train[:-10000]
y_train = y_train[:-10000]
# Specify the training configuration (optimizer, loss, metrics)
model.compile(
optimizer=keras.optimizers.RMSprop(), # Optimizer
# Loss function to minimize
loss=keras.losses.SparseCategoricalCrossentropy(),
# List of metrics to monitor
metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
# Train the model by slicing the data into "batches"
# of size "batch_size", and repeatedly iterating over
# the entire dataset for a given number of "epochs"
print("# Fit model on training data")
history = model.fit(
x_train,
y_train,
batch_size=64,
epochs=3,
# We pass some validation for
# monitoring validation loss and metrics
# at the end of each epoch
validation_data=(x_val, y_val),
)
# The returned "history" object holds a record
# of the loss values and metric values during training
print("\nhistory dict:", history.history)
# Evaluate the model on the test data using `evaluate`
print("\n# Evaluate on test data")
results = model.evaluate(x_test, y_test, batch_size=128)
print("test loss, test acc:", results)
# Generate predictions (probabilities -- the output of the last layer)
# on new data using `predict`
print("\n# Generate predictions for 3 samples")
predictions = model.predict(x_test[:3])
print("predictions shape:", predictions.shape)
The buildspec for this test is the following, where we run the test using python3 interpreter.
buildspecs:
run_tensorflow_model:
type: script
executor: generic.local.bash
description: Run TensorFlow Model
run: python3 tensorflow_model.py
Please run the test yourself and inspect the output. You will want to run the following commands:
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/tensorflow.yml
buildtest inspect query -o run_tensorflow_model
Running Tests in Containers
Buildtest has support for running tests in containers such as docker and singularity. In this
next examples, we will show how to run a hello world test in docker and singularity container. Let’s start off with docker, we
introduce a new property container that allows us to specify the container settings. The platform and image are required properties
that specifies the container platform (e.g., docker, singularity, podman) and the image name.
buildspecs:
hello_world_docker:
type: script
executor: generic.local.bash
description: run hello-world container with docker
container:
platform: "docker"
image: hello-world
run: echo 'Test Complete!'
Let’s run the test and inspect the output, you will notice the test output will print a message from the container. This test will run in a docker container,
if you look at the generated test content you will see a docker run invocation.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world.yml
╭────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:30 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world.yml │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world.yml ║
╟─────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/tutorials/containers/hello_world.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_docker/950a4841 │ script │ generic.local.bash │ None │ None │ None │ run hello-world container with docker │ /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world.yml │
└─────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴───────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_docker/950a4841: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_docker/950a4841 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_docker/950a4841 │
└─────────────────────────────┘
hello_world_docker/950a4841: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/stage
hello_world_docker/950a4841: Running Test via command: bash hello_world_docker_build.sh
hello_world_docker/950a4841: Test completed in 0.251264 seconds with returncode: 0
hello_world_docker/950a4841: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker.out
hello_world_docker/950a4841: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_docker/950a4841 │ generic.local.bash │ PASS │ 0 │ 0.251 │
└─────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_1ht7a34d.log
$ buildtest inspect query -o -t hello_world_docker
───────────────────────────────────────────────────────────────────────────────── hello_world_docker/950a4841-48d0-482a-bac0-c4ccfbc85665 ─────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: run hello-world container with docker
State: PASS
Returncode: 0
Runtime: 0.251264 sec
Starttime: 2024/06/17 18:50:30
Endtime: 2024/06/17 18:50:30
Command: bash hello_world_docker_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/hello_world_docker.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_1ht7a34d.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Test Complete!
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
# Content of run section
/usr/bin/docker run --rm -v /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/950a4841/stage:/buildtest hello-world
echo 'Test Complete!'
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
In the next example, we will run the same test in a singularity container. To do this we will simply change the
platform to singularity and specify the image name. Since singularity can pull images from different registries we will specify
docker::// prefix to pull the image from docker hub.
buildspecs:
hello_world_singularity:
type: script
executor: generic.local.bash
description: run hello-world container using singularity
container:
platform: "singularity"
image: 'docker://hello-world'
run: echo 'Test Complete!'
Buildtest will invoke singularity run and bind mount the stage directory into the container and execute test from the container. Take note that
that singularity will volume mount test into /buildtest in the container and then run test from that directory.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world_singularity.yml
$ buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world_singularity.yml
╭─────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:32 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/aws.yml build -b /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world_sing │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world_singularity.yml ║
╟─────────────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/tutorials/containers/hello_world_singularity.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_singularity/ce01e347 │ script │ generic.local.bash │ None │ None │ None │ run hello-world container using singularity │ /home/lbladmin/Documents/buildtest/tutorials/containers/hello_world_singulari │
│ │ │ │ │ │ │ │ ty.yml │
└──────────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_singularity/ce01e347: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 4 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_singularity/ce01e347 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_singularity/ce01e347 │
└──────────────────────────────────┘
hello_world_singularity/ce01e347: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/stage
hello_world_singularity/ce01e347: Running Test via command: bash hello_world_singularity_build.sh
hello_world_singularity/ce01e347: Test completed in 0.20001 seconds with returncode: 0
hello_world_singularity/ce01e347: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity.out
hello_world_singularity/ce01e347: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_singularity/ce01e347 │ generic.local.bash │ PASS │ 0 │ 0.200 │
└──────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_2937xdzg.log
$ buildtest inspect query -o -t hello_world_singularity
────────────────────────────────────────────────────────────────────────────── hello_world_singularity/ce01e347-6fc9-4df9-aa6d-cbd278c79ad7 ───────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: run hello-world container using singularity
State: PASS
Returncode: 0
Runtime: 0.20001 sec
Starttime: 2024/06/17 18:50:32
Endtime: 2024/06/17 18:50:32
Command: bash hello_world_singularity_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/hello_world_singularity.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_2937xdzg.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Test Complete!
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
# Content of run section
/usr/local/bin/singularity run -B /home/lbladmin/Documents/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/ce01e347/stage:/buildtest docker://hello-world
echo 'Test Complete!'
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
In this last example, we will run a test using a container executor by defining a custom executor based on a container
image. In all of our previous examples, we were running tests using executor generic.local.bash which is a generic executor
that runs tests on the local system using bash shell.
Let’s take a look at the executor configuration by running the following command. The container keyword under executors section is used
to define container executors. We can specify arbitrary name for the executor and specify the container image and platform.
$ buildtest -c $BUILDTEST_ROOT/buildtest/settings/container_executor.yml config executors list --yaml
executors:
container:
python:
description: submit jobs on python container
image: python:3.11.0
platform: docker
ubuntu:
description: submit jobs on ubuntu container
image: ubuntu:20.04
platform: docker
local:
bash:
description: submit jobs on local machine using bash shell
shell: bash
We have the following buildspec that will run test using a custom executor generic.container.ubuntu.
buildspecs:
ubuntu_container_example:
type: script
executor: generic.container.ubuntu
description: run test in a container executor
run: |
echo "USER: " $(whoami)
echo "HOMEDIR: " $HOME
echo "WORKDIR: " $(pwd)
df -h
echo "************************"
ls -l
echo "************************"
cat /etc/os-release
Let’s run the test and inspect the output. You will notice the test is run in a ubuntu 20.04 container. In the output of df -h you will
see the filesystem is from the container image with an entry /buildtest that is bind mounted from the host system.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/container_executor/ubuntu.yml
$ buildtest -c $BUILDTEST_ROOT/buildtest/settings/container_executor.yml build -b $BUILDTEST_ROOT/tutorials/containers/container_executor/ubuntu.yml
╭─────────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: lbladmin │
│ Hostname: e4spro-cluster │
│ Platform: Linux │
│ Current Time: 2024/06/17 18:50:34 │
│ buildtest path: /home/lbladmin/Documents/buildtest/bin/buildtest │
│ buildtest version: 2.0 │
│ python path: /usr/bin/python3 │
│ python version: 3.10.12 │
│ Configuration File: /home/lbladmin/Documents/buildtest/buildtest/settings/container_executor.yml │
│ Test Directory: /home/lbladmin/Documents/buildtest/var/tests │
│ Report File: /home/lbladmin/Documents/buildtest/var/report.json │
│ Command: /home/lbladmin/Documents/buildtest/bin/buildtest -c /home/lbladmin/Documents/buildtest/buildtest/settings/container_executor.yml build -b /home/lbladmin/Documents/buildtest/tutorials/containers/c │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═══════════════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟───────────────────────────────────────────────────────────────────────────────────────╢
║ /home/lbladmin/Documents/buildtest/tutorials/containers/container_executor/ubuntu.yml ║
╟───────────────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═══════════════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/lbladmin/Documents/buildtest/tutorials/containers/container_executor/ubuntu.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/abf3ecf3 │ script │ generic.container.ubuntu │ None │ None │ None │ run test in a container executor │ /home/lbladmin/Documents/buildtest/tutorials/containers/container_executor/ubuntu │
│ │ │ │ │ │ │ │ .yml │
└───────────────────────────────────┴────────┴──────────────────────────┴──────────┴───────┴───────┴──────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────┘
Batch Job Builders
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ executor ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/abf3ecf3 │ generic.container.ubuntu │ /home/lbladmin/Documents/buildtest/tutorials/containers/container_executor/ubuntu.yml │
└───────────────────────────────────┴──────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────
ubuntu_container_example/abf3ecf3: Creating Test Directory: /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3
────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 1 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────
ubuntu_container_example/abf3ecf3 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/abf3ecf3 │
└───────────────────────────────────┘
ubuntu_container_example/abf3ecf3: Current Working Directory : /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/stage
ubuntu_container_example/abf3ecf3: Running Test via command: bash ubuntu_container_example_build.sh
ubuntu_container_example/abf3ecf3: Test completed in 0.242506 seconds with returncode: 0
ubuntu_container_example/abf3ecf3: Writing output file - /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example.out
ubuntu_container_example/abf3ecf3: Writing error file - /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ ubuntu_container_example/abf3ecf3 │ generic.container.ubuntu │ PASS │ 0 │ 0.243 │
└───────────────────────────────────┴──────────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/lbladmin/Documents/buildtest/var/report.json
Writing Logfile to /home/lbladmin/Documents/buildtest/var/logs/buildtest_voydqejz.log
$ buildtest inspect query -o -t -b ubuntu_container_example
────────────────────────────────────────────────────────────────────────────── ubuntu_container_example/abf3ecf3-af0c-44c9-bbed-e70ecf3277fe ──────────────────────────────────────────────────────────────────────────────
Executor: generic.container.ubuntu
Description: run test in a container executor
State: PASS
Returncode: 0
Runtime: 0.242506 sec
Starttime: 2024/06/17 18:50:34
Endtime: 2024/06/17 18:50:34
Command: bash ubuntu_container_example_build.sh
Test Script: /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example.sh
Build Script: /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example_build.sh
Output File: /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example.out
Error File: /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/ubuntu_container_example.err
Log File: /home/lbladmin/Documents/buildtest/var/logs/buildtest_voydqejz.log
────────────────────────────────────────────────────────────────────────────────────────────────────── Output File: ──────────────────────────────────────────────────────────────────────────────────────────────────────
USER: root
HOMEDIR: /root
WORKDIR: /buildtest
Filesystem Size Used Avail Use% Mounted on
overlay 146G 117G 30G 80% /
/dev/root 146G 117G 30G 80% /buildtest
tmpfs 64M 0 64M 0% /dev
tmpfs 1.6G 168K 1.6G 1% /etc/hosts
shm 63M 0 63M 0% /dev/shm
devtmpfs 7.9G 0 7.9G 0% /dev/tty
************************
total 16
-rw-rw-r-- 1 root root 208 Jun 14 16:32 python_container.yml
-rw-rw-r-- 1 root root 379 Jun 14 16:32 ubuntu.yml
-rwxr-xr-x 1 root root 216 Jun 17 18:50 ubuntu_container_example.sh
-rwxr-xr-x 1 root root 1320 Jun 17 18:50 ubuntu_container_example_build.sh
************************
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────────── Test File: ───────────────────────────────────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
# Content of run section
echo "USER: " $(whoami)
echo "HOMEDIR: " $HOME
echo "WORKDIR: " $(pwd)
df -h
echo "************************"
ls -l
echo "************************"
cat /etc/os-release
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────────────────────────────────────────────────────────────────────────── Build Script File: ───────────────────────────────────────────────────────────────────────────────────────────────────
#!/bin/bash
# Function to handle all signals and perform cleanup
function cleanup() {
echo "Signal trapped. Performing cleanup before exiting."
exitcode=$?
echo "buildtest: command \`$BASH_COMMAND' failed (exit code: $exitcode)"
exit $exitcode
}
# Trap all signals and call the cleanup function
trap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU
export BUILDTEST_TEST_NAME=ubuntu_container_example
export BUILDTEST_TEST_ROOT=/home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3
export BUILDTEST_BUILDSPEC_DIR=/home/lbladmin/Documents/buildtest/tutorials/containers/container_executor
export BUILDTEST_STAGE_DIR=/home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/stage
# source executor startup script
source /home/lbladmin/Documents/buildtest/var/executor/generic.container.ubuntu/before_script.sh
# Run generated script
/usr/bin/docker run -it --rm -v /home/lbladmin/Documents/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/abf3ecf3/stage:/buildtest -w /buildtest ubuntu:20.04 bash -c /buildtest/ubuntu_contai
# Get return code
returncode=$?
# Exit with return code
exit $returncode
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────