Buildtest Tutorial on AWS
Setup
This section of tutorial will be completed in Amazon Web Services (AWS) environment. Once you have access to system, please Installing buildtest and then proceed to the next section.
Once you are done, please run the following commands to ensure you are using the correct configuration file
export BUILDTEST_CONFIGFILE=$BUILDTEST_ROOT/buildtest/settings/aws.yml
buildtest config path
You can verify the configuration is valid by runnning the following:
buildtest config validate
If you see no errors, then you are ready to proceed to the next section.
Hello World Compilation
Let’s start off with a simple hello world compilation in C using the GNU compiler. We have the following buildspec
that will compile a code source code hello.c using the gcc compiler wrapper.
buildspecs:
hello_world_example:
type: script
executor: generic.local.bash
description: Hello world compilation in C
run: |
gcc hello.c -o hello
./hello
The source code is the following
#include <stdio.h>
int main() {
printf("Hello, World in C\n");
return 0;
}
Let’s try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/hello.yml
$ buildtest build -b /home/ubuntu/buildtest/aws_tutorial/hello_world/hello.yml
╭───────────────────────────────────────────────────── buildtest summary ─────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 18:34:44 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/aws_tutorial/hello_world/hello.yml │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═══════════════════════════════════════════════════════════╗
║ buildspec ║
╟───────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/aws_tutorial/hello_world/hello.yml ║
╟───────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═══════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/aws_tutorial/hello_world/hello.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_example/3626d4ed │ script │ generic.local.bash │ None │ None │ None │ Hello world compilation in C │ /home/ubuntu/buildtest/aws_tutorial/hello_world/hello.yml │
└──────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────┴───────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_example/3626d4ed: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_example/3626d4ed does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_example/3626d4ed │
└──────────────────────────────┘
hello_world_example/3626d4ed: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/stage
hello_world_example/3626d4ed: Running Test via command: bash hello_world_example_build.sh
hello_world_example/3626d4ed: Test completed in 0.155718 seconds with returncode: 0
hello_world_example/3626d4ed: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.out
hello_world_example/3626d4ed: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_example/3626d4ed │ generic.local.bash │ PASS │ 0 │ 0.156 │
└──────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_e37i8a5i.log
$ buildtest inspect query -o -t hello_world_example
──────────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_example/3626d4ed-30e9-4b02-84fa-113ddf305922 ─────────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C
State: PASS
Returncode: 0
Runtime: 0.155718 sec
Starttime: 2024/02/28 18:34:44
Endtime: 2024/02/28 18:34:44
Command: bash hello_world_example_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_e37i8a5i.log
─────────────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.out ───────────────────────────────────────────────────────────────────────
Hello, World in C
──────────────────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello/hello_world_example/3626d4ed/hello_world_example.sh ─────────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
# Content of run section
gcc hello.c -o hello
./hello
Multi Compiler Test
Buildtest supports multiple test creation based on compiler selection using compilers property. This can be useful to test
a single test with multiple compilers. In this next example, we will attempt to compile the Hello World test in previous example using
2 versions of GNU compiler. The name property is used to search for compilers in the configuration file. The Defining Compilers section
covers in detail how to define compilers in buildtest configuration.
buildspecs:
hello_world_multi_compiler:
type: script
executor: generic.local.bash
description: Hello world compilation in C, C++ and Fortran with multiple compilers
compilers:
name: ['gcc']
run: |
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
Let’s try building this example, you will see there are now 2 tests created, one for each compiler.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/multi_compiler_hello.yml
$ buildtest build -b /home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml
╭──────────────────────────────────────────────────────────── buildtest summary ─────────────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 18:34:46 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml ║
╟──────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_multi_compiler/7ea7082b │ script │ generic.local.bash │ gcc_11.4.0 │ None │ None │ Hello world compilation in C, C++ and Fortran with multiple compilers │ /home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml │
├─────────────────────────────────────┼────────┼────────────────────┼────────────┼───────┼───────┼───────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ hello_world_multi_compiler/543d59fb │ script │ generic.local.bash │ gcc_9.4.0 │ None │ None │ Hello world compilation in C, C++ and Fortran with multiple compilers │ /home/ubuntu/buildtest/aws_tutorial/hello_world/multi_compiler_hello.yml │
└─────────────────────────────────────┴────────┴────────────────────┴────────────┴───────┴───────┴───────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_multi_compiler/7ea7082b: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b
hello_world_multi_compiler/543d59fb: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_multi_compiler/7ea7082b does not have any dependencies adding test to queue
hello_world_multi_compiler/543d59fb does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_multi_compiler/7ea7082b │
│ hello_world_multi_compiler/543d59fb │
└─────────────────────────────────────┘
hello_world_multi_compiler/7ea7082b: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/stage
hello_world_multi_compiler/543d59fb: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/stage
hello_world_multi_compiler/543d59fb: Running Test via command: bash hello_world_multi_compiler_build.sh
hello_world_multi_compiler/7ea7082b: Running Test via command: bash hello_world_multi_compiler_build.sh
hello_world_multi_compiler/7ea7082b: Test completed in 0.545868 seconds with returncode: 0
hello_world_multi_compiler/7ea7082b: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.out
hello_world_multi_compiler/7ea7082b: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.err
hello_world_multi_compiler/543d59fb: Test completed in 0.571412 seconds with returncode: 0
hello_world_multi_compiler/543d59fb: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.out
hello_world_multi_compiler/543d59fb: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_multi_compiler/7ea7082b │ generic.local.bash │ PASS │ 0 │ 0.546 │
├─────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ hello_world_multi_compiler/543d59fb │ generic.local.bash │ PASS │ 0 │ 0.571 │
└─────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_bknghril.log
Take note in the generated test output, the variables BUILDTEST_CC, BUILDTEST_CXX and BUILDTEST_FC are using the compiler wrappers for each compiler.
$ buildtest inspect query -o -t hello_world_multi_compiler/
───────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/7ea7082b-1c26-4e52-b62b-3108f192b860 ─────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 0.545868 sec
Starttime: 2024/02/28 18:34:46
Endtime: 2024/02/28 18:34:46
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_bknghril.log
──────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.out ─────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/7ea7082b/hello_world_multi_compiler.sh ──────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc"
BUILDTEST_CXX="/usr/bin/g++"
BUILDTEST_FC="/usr/bin/gfortran"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
───────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/543d59fb-5a91-43b4-8c21-4d99917729db ─────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 0.571412 sec
Starttime: 2024/02/28 18:34:46
Endtime: 2024/02/28 18:34:46
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_bknghril.log
──────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.out ─────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/543d59fb/hello_world_multi_compiler.sh ──────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc-9"
BUILDTEST_CXX="/usr/bin/g++-9"
BUILDTEST_FC="/usr/bin/gfortran-9"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
───────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/337a9268-35a4-4182-914c-3c9d985e05a8 ─────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 16.082103 sec
Starttime: 2024/02/28 18:24:33
Endtime: 2024/02/28 18:24:49
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_26_np7tl.log
──────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler.out ─────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/337a9268/hello_world_multi_compiler.sh ──────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc"
BUILDTEST_CXX="/usr/bin/g++"
BUILDTEST_FC="/usr/bin/gfortran"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
───────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_multi_compiler/432aa86e-b3db-460f-85a3-a25238a83672 ─────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello world compilation in C, C++ and Fortran with multiple compilers
State: PASS
Returncode: 0
Runtime: 23.025996 sec
Starttime: 2024/02/28 18:24:33
Endtime: 2024/02/28 18:24:56
Command: bash hello_world_multi_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_26_np7tl.log
──────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler.out ─────────────────────────────────────────────────────────
Hello, World in C
Hello, World in C++
Hello, World in Fortran
────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/multi_compiler_hello/hello_world_multi_compiler/432aa86e/hello_world_multi_compiler.sh ──────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc-9"
BUILDTEST_CXX="/usr/bin/g++-9"
BUILDTEST_FC="/usr/bin/gfortran-9"
BUILDTEST_CFLAGS=
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
# Content of run section
$BUILDTEST_CC hello.c -o hello_c
$BUILDTEST_CXX hello.cpp -o hello_cpp
$BUILDTEST_FC hello.f90 -o hello_f90
./hello_c
./hello_cpp
./hello_f90
We can see the compiler configuration using the command buildtest config compilers list --yaml which will print the output in YAML format.
Shown below is the compiler declaration
$ buildtest config compilers list --yaml
gcc:
gcc_11.4.0:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
fc: /usr/bin/gfortran
gcc_9.4.0:
cc: /usr/bin/gcc-9
cxx: /usr/bin/g++-9
fc: /usr/bin/gfortran-9
OpenMP Test with custom compiler configuration
In this next example, we will compile a OpenMP code using GNU compiler and specify custom compiler flags and environment variable. We will work with a Hello World OpenMP code that uses OpenMP pragma to parallelize the code. The source code is the following
// OpenMP program to print Hello World
// using C language
// OpenMP header
#include <omp.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[])
{
// Beginning of parallel region
#pragma omp parallel
{
printf("Hello World... from thread = %d\n",
omp_get_thread_num());
}
// Ending of parallel region
}
In order to build OpenMP code, we need to use -fopenmp flag to enable OpenMP support. We will set OMP_NUM_THREADS environment variable
to specify number of OpenMP theads which will differ for each compiler test. We introduce a keyword config that allows us to specify
customize compiler flags and environment variables. The env property is used to set environment variable for each compiler test and cflags is
used to set compiler flags inorder to compile the test. This value is stored in environment BUILDTEST_CFLAGS that will be used in run section
for compiling the source code.
buildspecs:
hello_world_openmp_custom_compiler:
type: script
executor: 'generic.local.bash'
description: Hello World OpenMP example with custom compiler settings
compilers:
name: ["gcc"]
config:
gcc_9.4.0:
env:
OMP_NUM_THREADS: "2"
cflags: "-O1 -fopenmp"
gcc_11.4.0:
env:
OMP_NUM_THREADS: "6"
cflags: "-O2 -fopenmp"
run: |
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
Let’s try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/openmp_example_custom_compiler.yml
$ buildtest build -b /home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml
╭─────────────────────────────────────────────────────────── buildtest summary ────────────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 21:07:52 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟────────────────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml ║
╟────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/265d3928 │ script │ generic.local.bash │ gcc_11.4.0 │ None │ None │ Hello World OpenMP example with custom compiler settings │ /home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml │
├─────────────────────────────────────────────┼────────┼────────────────────┼────────────┼───────┼───────┼──────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ hello_world_openmp_custom_compiler/10a8dbc6 │ script │ generic.local.bash │ gcc_9.4.0 │ None │ None │ Hello World OpenMP example with custom compiler settings │ /home/ubuntu/buildtest/aws_tutorial/openmp_example_custom_compiler.yml │
└─────────────────────────────────────────────┴────────┴────────────────────┴────────────┴───────┴───────┴──────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_openmp_custom_compiler/265d3928: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928
hello_world_openmp_custom_compiler/10a8dbc6: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_openmp_custom_compiler/265d3928 does not have any dependencies adding test to queue
hello_world_openmp_custom_compiler/10a8dbc6 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/265d3928 │
│ hello_world_openmp_custom_compiler/10a8dbc6 │
└─────────────────────────────────────────────┘
hello_world_openmp_custom_compiler/265d3928: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/stage
hello_world_openmp_custom_compiler/10a8dbc6: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/stage
hello_world_openmp_custom_compiler/265d3928: Running Test via command: bash hello_world_openmp_custom_compiler_build.sh
hello_world_openmp_custom_compiler/10a8dbc6: Running Test via command: bash hello_world_openmp_custom_compiler_build.sh
hello_world_openmp_custom_compiler/10a8dbc6: Test completed in 0.133897 seconds with returncode: 0
hello_world_openmp_custom_compiler/10a8dbc6: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.out
hello_world_openmp_custom_compiler/10a8dbc6: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.err
hello_world_openmp_custom_compiler/265d3928: Test completed in 0.200693 seconds with returncode: 0
hello_world_openmp_custom_compiler/265d3928: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.out
hello_world_openmp_custom_compiler/265d3928: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_openmp_custom_compiler/265d3928 │ generic.local.bash │ PASS │ 0 │ 0.201 │
├─────────────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ hello_world_openmp_custom_compiler/10a8dbc6 │ generic.local.bash │ PASS │ 0 │ 0.134 │
└─────────────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_h0h25cyv.log
We will see in the generated test the values OMP_NUM_THREADS and BUILDTEST_CFLAGS are set for each compiler test. The OMP_NUM_THREADS
will impact the number of threads used to run code therefore we will see different output for each compiler test.
$ buildtest inspect query -o -t hello_world_openmp_custom_compiler/
───────────────────────────────────────────────────────────────────────────────────────────────── hello_world_openmp_custom_compiler/265d3928-c872-4ae0-8c21-4fd8268315f1 ─────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello World OpenMP example with custom compiler settings
State: PASS
Returncode: 0
Runtime: 0.200693 sec
Starttime: 2024/02/28 21:07:52
Endtime: 2024/02/28 21:07:52
Command: bash hello_world_openmp_custom_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_h0h25cyv.log
─────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.out ────────────────────────────────────────────
Hello World... from thread = 0
Hello World... from thread = 3
Hello World... from thread = 4
Hello World... from thread = 5
Hello World... from thread = 2
Hello World... from thread = 1
───────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/265d3928/hello_world_openmp_custom_compiler.sh ─────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc"
BUILDTEST_CXX="/usr/bin/g++"
BUILDTEST_FC="/usr/bin/gfortran"
BUILDTEST_CFLAGS="-O2 -fopenmp"
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
export OMP_NUM_THREADS="6"
# Content of run section
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
───────────────────────────────────────────────────────────────────────────────────────────────── hello_world_openmp_custom_compiler/10a8dbc6-68e6-440e-9844-fa095136edf6 ─────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Hello World OpenMP example with custom compiler settings
State: PASS
Returncode: 0
Runtime: 0.133897 sec
Starttime: 2024/02/28 21:07:52
Endtime: 2024/02/28 21:07:52
Command: bash hello_world_openmp_custom_compiler_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_h0h25cyv.log
─────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.out ────────────────────────────────────────────
Hello World... from thread = 0
Hello World... from thread = 1
───────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/openmp_example_custom_compiler/hello_world_openmp_custom_compiler/10a8dbc6/hello_world_openmp_custom_compiler.sh ─────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
BUILDTEST_CC="/usr/bin/gcc-9"
BUILDTEST_CXX="/usr/bin/g++-9"
BUILDTEST_FC="/usr/bin/gfortran-9"
BUILDTEST_CFLAGS="-O1 -fopenmp"
BUILDTEST_CXXFLAGS=
BUILDTEST_FFLAGS=
BUILDTEST_CPPFLAGS=
BUILDTEST_LDFLAGS=
export OMP_NUM_THREADS="2"
# Content of run section
$BUILDTEST_CC $BUILDTEST_CFLAGS -o openmp_hello openmp_hello.c
./openmp_hello
Testing a MPI Code
This image comes with several MPI flavors such as OpenMPI, mvapich2. In the next example we will run a MPI Proc Name test
on login node that will run with 8 processes. The source code is available in the $HOME/examples directory as part of the image.
buildspecs:
mpiprocname:
type: script
executor: generic.local.bash
description: Run MPI Proc Name test
run: |
cd $HOME/examples/mpi-procname
mpicc -o mpiprocname mpiprocname.c
mpirun -np 8 ./mpiprocname
rm mpiprocname
We can run this test using the following command
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/mpiproc.yml
$ buildtest build -b /home/ubuntu/buildtest/aws_tutorial/mpiproc.yml
╭──────────────────────────────────────────────── buildtest summary ────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 18:35:05 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/aws_tutorial/mpiproc.yml │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/aws_tutorial/mpiproc.yml ║
╟─────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/aws_tutorial/mpiproc.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ mpiprocname/cd40d34b │ script │ generic.local.bash │ None │ None │ None │ Run MPI Proc Name test │ /home/ubuntu/buildtest/aws_tutorial/mpiproc.yml │
└──────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴────────────────────────┴─────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
mpiprocname/cd40d34b: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
mpiprocname/cd40d34b does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ mpiprocname/cd40d34b │
└──────────────────────┘
mpiprocname/cd40d34b: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/stage
mpiprocname/cd40d34b: Running Test via command: bash mpiprocname_build.sh
mpiprocname/cd40d34b: Test completed in 0.413352 seconds with returncode: 0
mpiprocname/cd40d34b: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.out
mpiprocname/cd40d34b: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ mpiprocname/cd40d34b │ generic.local.bash │ PASS │ 0 │ 0.413 │
└──────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_dfq2ezbn.log
We can see this test prints Hello World message from each process.
$ buildtest inspect query -o mpiprocname
──────────────────────────────────────────────────────────────────────────────────────────────────────────── mpiprocname/cd40d34b-a3a8-4fe9-885e-b2a7b30b212b ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run MPI Proc Name test
State: PASS
Returncode: 0
Runtime: 0.413352 sec
Starttime: 2024/02/28 18:35:05
Endtime: 2024/02/28 18:35:05
Command: bash mpiprocname_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_dfq2ezbn.log
────────────────────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/mpiproc/mpiprocname/cd40d34b/mpiprocname.out ──────────────────────────────────────────────────────────────────────────────
Hello, world. I am 0 of 8 on ip-172-31-96-59
Hello, world. I am 3 of 8 on ip-172-31-96-59
Hello, world. I am 4 of 8 on ip-172-31-96-59
Hello, world. I am 5 of 8 on ip-172-31-96-59
Hello, world. I am 6 of 8 on ip-172-31-96-59
Hello, world. I am 7 of 8 on ip-172-31-96-59
Hello, world. I am 1 of 8 on ip-172-31-96-59
Hello, world. I am 2 of 8 on ip-172-31-96-59
OSU Microbenchmark
The OSU Microbenchmark is a collection of MPI-based benchmarks developed at Ohio State University.
This benchmark is used to measure performance of MPI libraries and is a popular benchmark suite for MPI. We will run the osu_bw test
which measures bandwidth. The test requires we specify 2 processes.
In this next example, we will run 2 tests, the first is simply invoking the osu_bw test with 2 processes, and the second test will
run the same test and use comparison_operators to compare the bandwidth result. The metrics property is used to
capture performance metrics that can be used for comparison. We will use the assert_ge status check that
will do a greater than or equal comparison with reference value. The metrics will capture performance results for message length 16384 and
perform a comparison with reference value of 10000. If the test result is greater than or equal to 10000, the test will pass otherwise it will fail.
buildspecs:
osu_bandwidth:
type: script
executor: generic.local.bash
description: Run OSU Bandwidth Test
run: mpirun -np 2 osu_bw
osu_bandwidth_perf:
type: script
executor: generic.local.bash
description: Run OSU Bandwidth Performance Test
run: mpirun -np 2 osu_bw
metrics:
osu_bw:
type: float
regex:
exp: '^16384\s+([\d.]+)$'
stream: stdout
item: 1
status:
assert_ge:
comparisons:
- name: osu_bw
ref: 10000
Let’s run the test and see the results.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/osu_bandwidth_test.yml
$ buildtest build -b /home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml
╭───────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 18:35:07 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔════════════════════════════════════════════════════════════╗
║ buildspec ║
╟────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml ║
╟────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml: VALID
Total builder objects created: 2
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ osu_bandwidth/87947870 │ script │ generic.local.bash │ None │ None │ None │ Run OSU Bandwidth Test │ /home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml │
├─────────────────────────────┼────────┼────────────────────┼──────────┼───────┼───────┼────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ osu_bandwidth_perf/55a7f00a │ script │ generic.local.bash │ None │ None │ None │ Run OSU Bandwidth Performance Test │ /home/ubuntu/buildtest/aws_tutorial/osu_bandwidth_test.yml │
└─────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴────────────────────────────────────┴────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
osu_bandwidth/87947870: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870
osu_bandwidth_perf/55a7f00a: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
osu_bandwidth/87947870 does not have any dependencies adding test to queue
osu_bandwidth_perf/55a7f00a does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ osu_bandwidth/87947870 │
│ osu_bandwidth_perf/55a7f00a │
└─────────────────────────────┘
osu_bandwidth_perf/55a7f00a: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/stage
osu_bandwidth/87947870: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/stage
osu_bandwidth_perf/55a7f00a: Running Test via command: bash osu_bandwidth_perf_build.sh
osu_bandwidth/87947870: Running Test via command: bash osu_bandwidth_build.sh
osu_bandwidth/87947870: Test completed in 2.477132 seconds with returncode: 0
osu_bandwidth/87947870: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.out
osu_bandwidth/87947870: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.err
osu_bandwidth_perf/55a7f00a: Test completed in 2.625078 seconds with returncode: 0
osu_bandwidth_perf/55a7f00a: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.out
osu_bandwidth_perf/55a7f00a: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.err
osu_bandwidth_perf/55a7f00a: testing metric: osu_bw if 6267.26 >= 10000.0 - Check: False
osu_bandwidth_perf/55a7f00a: ge check: False
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ osu_bandwidth/87947870 │ generic.local.bash │ PASS │ 0 │ 2.477 │
├─────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ osu_bandwidth_perf/55a7f00a │ generic.local.bash │ FAIL │ 0 │ 2.625 │
└─────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%
Adding 2 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_9zpt_39v.log
Take note in the output of the second test, we will see a list of performance metrics captured in table output. If metrics is not captured, the value will be undefined.
$ buildtest inspect query -o osu_bandwidth osu_bandwidth_perf
─────────────────────────────────────────────────────────────────────────────────────────────────────────── osu_bandwidth/87947870-1953-46f0-8cd0-ea526092df8b ────────────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run OSU Bandwidth Test
State: PASS
Returncode: 0
Runtime: 2.477132 sec
Starttime: 2024/02/28 18:35:07
Endtime: 2024/02/28 18:35:09
Command: bash osu_bandwidth_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_9zpt_39v.log
────────────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth/87947870/osu_bandwidth.out ───────────────────────────────────────────────────────────────────────
# OSU MPI Bandwidth Test v7.3
# Size Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1 5.65
2 3.57
4 9.26
8 33.72
16 20.76
32 161.53
64 133.98
128 705.10
256 534.20
512 708.01
1024 390.97
2048 899.16
4096 5240.26
8192 3275.13
16384 4527.77
32768 6407.88
65536 9635.86
131072 15361.67
262144 10873.81
524288 18213.85
1048576 10275.43
2097152 11507.50
4194304 4837.92
───────────────────────────────────────────────────────────────────────────────────────────────────────── osu_bandwidth_perf/55a7f00a-fcd9-42fb-88a9-5f616ad03d58 ─────────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run OSU Bandwidth Performance Test
State: FAIL
Returncode: 0
Runtime: 2.625078 sec
Starttime: 2024/02/28 18:35:07
Endtime: 2024/02/28 18:35:09
Command: bash osu_bandwidth_perf_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_9zpt_39v.log
Metrics
┏━━━━━━━━┳━━━━━━━━━┓
┃ Name ┃ Value ┃
┡━━━━━━━━╇━━━━━━━━━┩
│ osu_bw │ 6267.26 │
└────────┴─────────┘
───────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/osu_bandwidth_test/osu_bandwidth_perf/55a7f00a/osu_bandwidth_perf.out ──────────────────────────────────────────────────────────────────
# OSU MPI Bandwidth Test v7.3
# Size Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1 1.11
2 1.10
4 10.29
8 3.58
16 7.43
32 40.57
64 318.26
128 121.58
256 648.05
512 996.95
1024 3282.12
2048 2766.76
4096 2129.39
8192 5918.99
16384 6267.26
32768 2773.79
65536 22293.99
131072 25997.02
262144 26964.82
524288 23820.10
1048576 17338.45
2097152 11510.47
4194304 4020.37
Tensorflow Test
In this next exercise, we provide a tensorflow test in python that train a model and generate a predicition model. We have the following source code
from tensorflow import keras
from tensorflow.keras import layers
print("Testing TensorFlow")
inputs = keras.Input(shape=(784,), name="digits")
x = layers.Dense(64, activation="relu", name="dense_1")(inputs)
x = layers.Dense(64, activation="relu", name="dense_2")(x)
outputs = layers.Dense(10, activation="softmax", name="predictions")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
# Load a toy dataset for the sake of this example
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
# Preprocess the data (these are Numpy arrays)
x_train = x_train.reshape(60000, 784).astype("float32") / 255
x_test = x_test.reshape(10000, 784).astype("float32") / 255
y_train = y_train.astype("float32")
y_test = y_test.astype("float32")
# Reserve 10,000 samples for validation
x_val = x_train[-10000:]
y_val = y_train[-10000:]
x_train = x_train[:-10000]
y_train = y_train[:-10000]
# Specify the training configuration (optimizer, loss, metrics)
model.compile(
optimizer=keras.optimizers.RMSprop(), # Optimizer
# Loss function to minimize
loss=keras.losses.SparseCategoricalCrossentropy(),
# List of metrics to monitor
metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
# Train the model by slicing the data into "batches"
# of size "batch_size", and repeatedly iterating over
# the entire dataset for a given number of "epochs"
print("# Fit model on training data")
history = model.fit(
x_train,
y_train,
batch_size=64,
epochs=3,
# We pass some validation for
# monitoring validation loss and metrics
# at the end of each epoch
validation_data=(x_val, y_val),
)
# The returned "history" object holds a record
# of the loss values and metric values during training
print("\nhistory dict:", history.history)
# Evaluate the model on the test data using `evaluate`
print("\n# Evaluate on test data")
results = model.evaluate(x_test, y_test, batch_size=128)
print("test loss, test acc:", results)
# Generate predictions (probabilities -- the output of the last layer)
# on new data using `predict`
print("\n# Generate predictions for 3 samples")
predictions = model.predict(x_test[:3])
print("predictions shape:", predictions.shape)
The buildspec for this test is the following, where we run the test using python3 interpreter.
buildspecs:
run_tensorflow_model:
type: script
executor: generic.local.bash
description: Run TensorFlow Model
run: python3 tensorflow_model.py
Please run the test yourself and inspect the output. You will want to run the following commands:
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/tensorflow.yml
buildtest inspect query -o run_tensorflow_model
Running Tests in Containers
Buildtest has support for running tests in containers such as docker and singularity. In this
next examples, we will show how to run a hello world test in docker and singularity container. Let’s start off with docker, we
introduce a new property container that allows us to specify the container settings. The platform and image are required properties
that specifies the container platform (e.g., docker, singularity, podman) and the image name.
buildspecs:
hello_world_docker:
type: script
executor: generic.local.bash
description: run hello-world container with docker
container:
platform: "docker"
image: hello-world
run: echo 'Test Complete!'
Let’s run the test and inspect the output, you will notice the test output will print a message from the container. This test will run in a docker container,
if you look at the generated test content you will see a docker run invocation.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world.yml
$ buildtest build -b /home/ubuntu/buildtest/tutorials/containers/hello_world.yml
╭────────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 21:58:07 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/tutorials/containers/hello_world.yml │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/tutorials/containers/hello_world.yml ║
╟─────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/tutorials/containers/hello_world.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_docker/9345e5ec │ script │ generic.local.bash │ None │ None │ None │ run hello-world container with docker │ /home/ubuntu/buildtest/tutorials/containers/hello_world.yml │
└─────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴───────────────────────────────────────┴─────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_docker/9345e5ec: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_docker/9345e5ec does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_docker/9345e5ec │
└─────────────────────────────┘
hello_world_docker/9345e5ec: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/stage
hello_world_docker/9345e5ec: Running Test via command: bash hello_world_docker_build.sh
hello_world_docker/9345e5ec: Test completed in 0.840842 seconds with returncode: 0
hello_world_docker/9345e5ec: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.out
hello_world_docker/9345e5ec: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_docker/9345e5ec │ generic.local.bash │ PASS │ 0 │ 0.841 │
└─────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_qjlh5cbe.log
$ buildtest inspect query -o -t hello_world_docker
───────────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_docker/9345e5ec-656c-4671-b541-47889d15e21a ─────────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: run hello-world container with docker
State: PASS
Returncode: 0
Runtime: 0.840842 sec
Starttime: 2024/02/28 21:58:07
Endtime: 2024/02/28 21:58:08
Command: bash hello_world_docker_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_qjlh5cbe.log
───────────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.out ─────────────────────────────────────────────────────────────────────
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Test Complete!
────────────────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/hello_world_docker.sh ───────────────────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
# Content of run section
docker run -v /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world/hello_world_docker/9345e5ec/stage:/buildtest hello-world
echo 'Test Complete!'
In the next example, we will run the same test in a singularity container. To do this we will simply change the
platform to singularity and specify the image name. Since singularity can pull images from different registries we will specify
docker::// prefix to pull the image from docker hub.
buildspecs:
hello_world_singularity:
type: script
executor: generic.local.bash
description: run hello-world container using singularity
container:
platform: "singularity"
image: 'docker://hello-world'
run: echo 'Test Complete!'
Buildtest will invoke singularity run and bind mount the stage directory into the container and execute test from the container. Take note that
that singularity will volume mount test into /buildtest in the container and then run test from that directory.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world_singularity.yml
$ buildtest build -b /home/ubuntu/buildtest/tutorials/containers/hello_world_singularity.yml
╭──────────────────────────────────────────────────────────── buildtest summary ────────────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 21:58:10 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/aws.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest build -b /home/ubuntu/buildtest/tutorials/containers/hello_world_singularity.yml │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟─────────────────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/tutorials/containers/hello_world_singularity.yml ║
╟─────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/tutorials/containers/hello_world_singularity.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_singularity/8c4c7c79 │ script │ generic.local.bash │ None │ None │ None │ run hello-world container using singularity │ /home/ubuntu/buildtest/tutorials/containers/hello_world_singularity.yml │
└──────────────────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_singularity/8c4c7c79: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
hello_world_singularity/8c4c7c79 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ hello_world_singularity/8c4c7c79 │
└──────────────────────────────────┘
hello_world_singularity/8c4c7c79: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/stage
hello_world_singularity/8c4c7c79: Running Test via command: bash hello_world_singularity_build.sh
hello_world_singularity/8c4c7c79: Test completed in 0.443567 seconds with returncode: 0
hello_world_singularity/8c4c7c79: Writing output file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.out
hello_world_singularity/8c4c7c79: Writing error file - /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ hello_world_singularity/8c4c7c79 │ generic.local.bash │ PASS │ 0 │ 0.444 │
└──────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_s78629tb.log
$ buildtest inspect query -o -t hello_world_singularity
────────────────────────────────────────────────────────────────────────────────────────────────────── hello_world_singularity/8c4c7c79-b297-40ad-a5ff-bf810957f0c7 ───────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: run hello-world container using singularity
State: PASS
Returncode: 0
Runtime: 0.443567 sec
Starttime: 2024/02/28 21:58:10
Endtime: 2024/02/28 21:58:10
Command: bash hello_world_singularity_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.out
Error File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_s78629tb.log
────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.out ──────────────────────────────────────────────────────────
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Test Complete!
─────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/hello_world_singularity.sh ────────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
# Content of run section
singularity run -B /home/ubuntu/buildtest/var/tests/generic.local.bash/hello_world_singularity/hello_world_singularity/8c4c7c79/stage:/buildtest docker://hello-world
echo 'Test Complete!'
In this last example, we will run a test using a container executor by defining a custom executor based on a container
image. In all of our previous examples, we were running tests using executor generic.local.bash which is a generic executor
that runs tests on the local system using bash shell.
Let’s take a look at the executor configuration by running the following command. The container keyword under executors section is used
to define container executors. We can specify arbitrary name for the executor and specify the container image and platform.
$ buildtest -c $BUILDTEST_ROOT/buildtest/settings/container_executor.yml config executors list --yaml
executors:
container:
python:
description: submit jobs on python container
image: python:3.11.0
platform: docker
ubuntu:
description: submit jobs on ubuntu container
image: ubuntu:20.04
platform: docker
local:
bash:
description: submit jobs on local machine using bash shell
shell: bash
We have the following buildspec that will run test using a custom executor generic.container.ubuntu.
buildspecs:
ubuntu_container_example:
type: script
executor: generic.container.ubuntu
description: run test in a container executor
run: |
echo "USER: " $(whoami)
echo "HOMEDIR: " $HOME
echo "WORKDIR: " $(pwd)
df -h
echo "************************"
ls -l
echo "************************"
cat /etc/os-release
Let’s run the test and inspect the output. You will notice the test is run in a ubuntu 20.04 container. In the output of df -h you will
see the filesystem is from the container image with an entry /buildtest that is bind mounted from the host system.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/container_executor/ubuntu.yml
$ buildtest -c $BUILDTEST_ROOT/buildtest/settings/container_executor.yml build -b $BUILDTEST_ROOT/tutorials/containers/container_executor/ubuntu.yml
╭─────────────────────────────────────────────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ User: ubuntu │
│ Hostname: ip-172-31-96-59 │
│ Platform: Linux │
│ Current Time: 2024/02/28 21:58:12 │
│ buildtest path: /home/ubuntu/buildtest/bin/buildtest │
│ buildtest version: 1.8 │
│ python path: /usr/bin/python3 │
│ python version: 3.8.10 │
│ Configuration File: /home/ubuntu/buildtest/buildtest/settings/container_executor.yml │
│ Test Directory: /home/ubuntu/buildtest/var/tests │
│ Report File: /home/ubuntu/buildtest/var/report.json │
│ Command: /home/ubuntu/buildtest/bin/buildtest -c /home/ubuntu/buildtest/buildtest/settings/container_executor.yml build -b /home/ubuntu/buildtest/tutorials/containers/container_executor/ubuntu.yml │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Discovered buildspecs
╔═══════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟───────────────────────────────────────────────────────────────────────────╢
║ /home/ubuntu/buildtest/tutorials/containers/container_executor/ubuntu.yml ║
╟───────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚═══════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/ubuntu/buildtest/tutorials/containers/container_executor/ubuntu.yml: VALID
Total builder objects created: 1
Builders by type=script
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ description ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/6e8234e9 │ script │ generic.container.ubuntu │ None │ None │ None │ run test in a container executor │ /home/ubuntu/buildtest/tutorials/containers/container_executor/ubuntu.yml │
└───────────────────────────────────┴────────┴──────────────────────────┴──────────┴───────┴───────┴──────────────────────────────────┴───────────────────────────────────────────────────────────────────────────┘
Batch Job Builders
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder ┃ executor ┃ buildspecs ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/6e8234e9 │ generic.container.ubuntu │ /home/ubuntu/buildtest/tutorials/containers/container_executor/ubuntu.yml │
└───────────────────────────────────┴──────────────────────────┴───────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
ubuntu_container_example/6e8234e9: Creating Test Directory: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 1 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
ubuntu_container_example/6e8234e9 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ubuntu_container_example/6e8234e9 │
└───────────────────────────────────┘
ubuntu_container_example/6e8234e9: Current Working Directory : /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/stage
ubuntu_container_example/6e8234e9: Running Test via command: bash ubuntu_container_example_build.sh
ubuntu_container_example/6e8234e9: Test completed in 0.681839 seconds with returncode: 0
ubuntu_container_example/6e8234e9: Writing output file - /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.out
ubuntu_container_example/6e8234e9: Writing error file - /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.err
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ ubuntu_container_example/6e8234e9 │ generic.container.ubuntu │ PASS │ 0 │ 0.682 │
└───────────────────────────────────┴──────────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
Adding 1 test results to report file: /home/ubuntu/buildtest/var/report.json
Writing Logfile to /home/ubuntu/buildtest/var/logs/buildtest_8jxyimxt.log
$ buildtest inspect query -o -t -b ubuntu_container_example
────────────────────────────────────────────────────────────────────────────────────────────────────── ubuntu_container_example/6e8234e9-1782-409c-b395-75a1a73a5c53 ──────────────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.container.ubuntu
Description: run test in a container executor
State: PASS
Returncode: 0
Runtime: 0.681839 sec
Starttime: 2024/02/28 21:58:13
Endtime: 2024/02/28 21:58:13
Command: bash ubuntu_container_example_build.sh
Test Script: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.sh
Build Script: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example_build.sh
Output File: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.out
Error File: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.err
Log File: /home/ubuntu/buildtest/var/logs/buildtest_8jxyimxt.log
────────────────────────────────────────────────────────────── Output File: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.out ───────────────────────────────────────────────────────────────
USER: root
HOMEDIR: /root
WORKDIR: /buildtest
Filesystem Size Used Avail Use% Mounted on
overlay 194G 100G 95G 52% /
tmpfs 64M 0 64M 0% /dev
tmpfs 16G 0 16G 0% /sys/fs/cgroup
shm 64M 0 64M 0% /dev/shm
/dev/root 194G 100G 95G 52% /buildtest
tmpfs 16G 0 16G 0% /proc/acpi
tmpfs 16G 0 16G 0% /proc/scsi
tmpfs 16G 0 16G 0% /sys/firmware
************************
total 16
-rw-rw-r-- 1 1000 1000 208 Feb 28 18:20 python_container.yml
-rw-rw-r-- 1 1000 1000 379 Feb 28 18:20 ubuntu.yml
-rwxr-xr-x 1 1000 1000 233 Feb 28 21:58 ubuntu_container_example.sh
-rwxr-xr-x 1 1000 1000 1251 Feb 28 21:58 ubuntu_container_example_build.sh
************************
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
──────────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example.sh ────────────────────────────────────────────────────────────────
#!/usr/bin/bash
set -eo pipefail
# Content of run section
echo "USER: " $(whoami)
echo "HOMEDIR: " $HOME
echo "WORKDIR: " $(pwd)
df -h
echo "************************"
ls -l
echo "************************"
cat /etc/os-release
───────────────────────────────────────────────────────────── Test File: /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/ubuntu_container_example_build.sh ─────────────────────────────────────────────────────────────
#!/bin/bash
# Function to handle all signals and perform cleanup
function cleanup() {
echo "Signal trapped. Performing cleanup before exiting."
exitcode=$?
echo "buildtest: command \`$BASH_COMMAND' failed (exit code: $exitcode)"
exit $exitcode
}
# Trap all signals and call the cleanup function
trap cleanup SIGINT SIGTERM SIGHUP SIGQUIT SIGABRT SIGKILL SIGALRM SIGPIPE SIGTERM SIGTSTP SIGTTIN SIGTTOU
export BUILDTEST_TEST_NAME=ubuntu_container_example
export BUILDTEST_TEST_ROOT=/home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9
export BUILDTEST_BUILDSPEC_DIR=/home/ubuntu/buildtest/tutorials/containers/container_executor
export BUILDTEST_STAGE_DIR=/home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/stage
# source executor startup script
source /home/ubuntu/buildtest/var/executor/generic.container.ubuntu/before_script.sh
# Run generated script
docker run -it --rm -v /home/ubuntu/buildtest/var/tests/generic.container.ubuntu/ubuntu/ubuntu_container_example/6e8234e9/stage:/buildtest -w /buildtest ubuntu:20.04 bash -c /buildtest/ubuntu_container_example.sh
# Get return code
returncode=$?
# Exit with return code
exit $returncode