Defining Compilers

buildtest provides a mechanism to declare compilers in your configuration file, this is defined in compilers top-level section. The compilers should reflect compilers installed at your site. The compilers are declared within scope of a system since we assume compilers will vary across different HPC clusters.

Compiler Declaration

Shown below is a declaration of builtin_gcc provided by default.

compilers:
  compiler:
    gcc:
      builtin_gcc:
        cc: /usr/bin/gcc
        cxx: /usr/bin/g++
        fc: /usr/bin/gfortran

The compiler declaration is defined in section compiler followed by name of compiler in this case gcc. In the gcc section one can define all gnu compilers, which includes the name of the compiler in this example we call builtin_gcc as system compiler that defines C, C++ and Fortran compilers using cc, cxx and fc.

One can retrieve all compilers using buildtest config compilers list, there are few options for this command.

buildtest config compilers list --help
$ buildtest config compilers list --help
usage: buildtest [options] [COMMANDS] config compilers list [-h] [-j | -y]

optional arguments:
  -h, --help  show this help message and exit
  -j, --json  List compiler details in JSON format
  -y, --yaml  List compiler details in YAML format

buildtest can represent compiler output in JSON, YAML using the --json and --yaml. Shown below is an example output with these options

buildtest config compilers list --json
$ buildtest config compilers list --json
{
  "gcc": {
    "builtin_gcc": {
      "cc": "gcc",
      "fc": "gfortran",
      "cxx": "g++"
    }
  }
}
buildtest config compilers list --yaml
$ buildtest config compilers list --yaml
gcc:
  builtin_gcc:
    cc: gcc
    cxx: g++
    fc: gfortran

If you want to see a flat listing of the compilers as names you can simply run buildtest config compilers list as shown below

$ buildtest config compilers list
builtin_gcc

Detect Compilers (Experimental Feature)

Note

This feature relies on module system (Lmod, environment-modules) to search modulefiles and one must specify moduletool property to indicate how buildtest will search modules. If moduletool: lmod is set, buildtest will rely on Lmod spider using Lmodule API to detect and test all modules. If moduletool: environment-modules is set, buildtest will retrieve modules using output of module -t av.

Note

buildtest config compilers find will not update the buildtest configuration with new compilers, you will need to use --update option to write changes back to configuration file.

buildtest can detect compilers based on modulefiles and generate compiler section that way you don’t have to specify each compiler manually. This can be done via buildtest config compilers find command. Buildtest expects a key/value mapping when searching compiler names and regular expression using re.match for discovering compiler modules.

This can be demonstrated, by defining search pattern in the find section that expects a dictionary of key/value mapping between compiler names and their module names.

In example, below we define a pattern for gcc modules as ^(gcc) which will find all modules that start with name gcc.

 compilers:
   find:
     gcc: "^(gcc)"
   compiler:
     gcc:
       builtin:
         cc: /usr/bin/gcc
         cxx: /usr/bin/g++
         fc: /usr/bin/gfortran

In this system, we have two gcc modules and we will add both modules as compiler declaration in buildtest.

$ module -t av gcc
/Users/siddiq90/projects/spack/share/spack/lmod/darwin-catalina-x86_64/Core:
gcc/9.3.0-n7p74fd
gcc/10.2.0-37fmsw7

We will run buildtest config compilers find which will search all modules based on regular expression and add compilers in their respective group. In this example, buildtest automatically add gcc/9.2.0-n7p74fd and gcc/10.2.0-37fmsw7 modules as compiler instance. Depending on the compiler group, buildtest will update the properties cc, cxx, fc to the appropriate compiler wrapper. The module property defines the module configuration to be used to access the compiler, the load property is a list of modules to load. The purge property is a boolean that determines whether to run module purge prior to loading modules when using the compiler. If purge: true is set then we will do module purge.

buildtest config compilers find
 1 $ buildtest config compilers find
 2 MODULEPATH: /Users/siddiq90/projects/spack/share/spack/lmod/darwin-catalina-x86_64/Core:/usr/local/Cellar/lmod/8.6.14/modulefiles/Darwin:/usr/local/Cellar/lmod/8.6.14/modulefiles/Core
 3 ──────────────────────────────────────────────────────────────────────────────────────────────────────────── Detect Compilers ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
 4 gcc:
 5   builtin_gcc:
 6     cc: /usr/bin/gcc
 7     cxx: /usr/bin/g++
 8     fc: /usr/bin/gfortran
 9   gcc/10.2.0-37fmsw7:
10     cc: gcc
11     cxx: g++
12     fc: gfortran
13     module:
14       load:
15       - gcc/10.2.0-37fmsw7
16       purge: false
17   gcc/9.3.0-n7p74fd:
18     cc: gcc
19     cxx: g++
20     fc: gfortran
21     module:
22       load:
23       - gcc/9.3.0-n7p74fd
24       purge: false

You can use –detailed option to see how buildtest discovers compilers by searching the modules in MODULEPATH and testing each one with a regular expression. We can see in the output buildtest is applying a regular expression with each modulefile and if there is a match, we add the compiler instance into the appropriate compiler group.

buildtest config compilers find --detailed
 1 $ buildtest config compilers find --detailed
 2 MODULEPATH: /Users/siddiq90/projects/spack/share/spack/lmod/darwin-catalina-x86_64/Core:/usr/local/Cellar/lmod/8.6.14/modulefiles/Darwin:/usr/local/Cellar/lmod/8.6.14/modulefiles/Core
 3 Searching modules via Lmod Spider
 4 Applying regex ^(gcc) with module: autoconf/2.69-3yrvwbu
 5 Applying regex ^(gcc) with module: autoconf-archive/2019.01.06-qoeupni
 6 Applying regex ^(gcc) with module: automake/1.16.2-vjjvnh7
 7 Applying regex ^(gcc) with module: berkeley-db/18.1.40-zixsuu6
 8 Applying regex ^(gcc) with module: bzip2/1.0.8-uem3fk5
 9 Applying regex ^(gcc) with module: diffutils/3.7-67w5vu5
10 Applying regex ^(gcc) with module: gcc/9.3.0-n7p74fd
11 Applying regex ^(gcc) with module: gcc/10.2.0-37fmsw7
12 Applying regex ^(gcc) with module: gdbm/1.18.1-qcqdlzf
13 Applying regex ^(gcc) with module: gmp/6.1.2-pstkmss
14 Applying regex ^(gcc) with module: isl/0.21-v6cpwya
15 Applying regex ^(gcc) with module: isl/0.20-ypts4jg
16 Applying regex ^(gcc) with module: libiconv/1.16-3kkozjq
17 Applying regex ^(gcc) with module: libsigsegv/2.12-dg5wkck
18 Applying regex ^(gcc) with module: libtool/2.4.6-sp423u5
19 Applying regex ^(gcc) with module: lmod
20 Applying regex ^(gcc) with module: m4/1.4.18-wctmckj
21 Applying regex ^(gcc) with module: mpc/1.1.0-xid3nuo
22 Applying regex ^(gcc) with module: mpc/1.1.0-sqfmp67
23 Applying regex ^(gcc) with module: mpfr/3.1.6-nm4h2fx
24 Applying regex ^(gcc) with module: mpfr/4.0.2-6in3dph
25 Applying regex ^(gcc) with module: ncurses/6.2-g5wyknv
26 Applying regex ^(gcc) with module: perl/5.32.0-hlmfvxi
27 Applying regex ^(gcc) with module: pkgconf/1.7.3-pxfp6qy
28 Applying regex ^(gcc) with module: readline/8.0-d4acjhu
29 Applying regex ^(gcc) with module: settarg
30 Applying regex ^(gcc) with module: zlib/1.2.11-id3vwmq
31 Applying regex ^(gcc) with module: zstd/1.4.5-2tk5glw
32   Discovered Modules
33 ┏━━━━━━━━━━━━━━━━━━━━┓
34 ┃ Name               ┃
35 ┡━━━━━━━━━━━━━━━━━━━━┩
36 │ gcc/9.3.0-n7p74fd  │
37 ├────────────────────┤
38 │ gcc/10.2.0-37fmsw7 │
39 └────────────────────┘
40 [DEBUG] Executing module command: bash -l -c "module load gcc/9.3.0-n7p74fd  "
41 [DEBUG] Return Code: 0
42 [DEBUG] Executing module command: bash -l -c "module load gcc/10.2.0-37fmsw7  "
43 [DEBUG] Return Code: 0
44 ──────────────────────────────────────────────────────────────────────────────────────────────────────────── Detect Compilers ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
45 gcc:
46   builtin_gcc:
47     cc: /usr/bin/gcc
48     cxx: /usr/bin/g++
49     fc: /usr/bin/gfortran
50   gcc/10.2.0-37fmsw7:
51     cc: gcc
52     cxx: g++
53     fc: gfortran
54     module:
55       load:
56       - gcc/10.2.0-37fmsw7
57       purge: false
58   gcc/9.3.0-n7p74fd:
59     cc: gcc
60     cxx: g++
61     fc: gfortran
62     module:
63       load:
64       - gcc/9.3.0-n7p74fd
65       purge: false

Module Purge

We can configure each compiler instance to run module purge behavior by setting purge property as part of the compilers section. buildtest will set purge: true in each of the compiler section when running buildtest config compilers find. The purge property is optional, if its not defined then buildtest will assume purge: false as the value

 1 system:
 2   generic:
 3     hostnames:
 4     - .*
 5     description: Generic System
 6     moduletool: lmod
 7     executors:
 8       local:
 9         bash:
10           description: submit jobs on local machine using bash shell
11           shell: bash
12         sh:
13           description: submit jobs on local machine using sh shell
14           shell: sh
15         csh:
16           description: submit jobs on local machine using csh shell
17           shell: csh
18         zsh:
19           description: submit jobs on local machine using zsh shell
20           shell: zsh
21         python:
22           description: submit jobs on local machine using python shell
23           shell: python
24     compilers:
25       purge: true
26       find:
27         gcc: ^(gcc)
28       compiler:
29         gcc:
30           builtin_gcc:
31             cc: /usr/bin/gcc
32             fc: /usr/bin/gfortran
33             cxx: /usr/bin/g++

Now take a look at generated compilers upon running buildtest config compiler find, you will see purge: true is set in each compiler instance

 1      buildtest config compilers find
 2    MODULEPATH: /Users/siddiq90/projects/spack/share/spack/lmod/darwin-catalina-x86_64/Core:/usr/local/Cellar/lmod/8.6.14/modulefiles/Darwin:/usr/local/Cellar/lmod/8.6.14/modulefiles/Core
 3    ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Detect Compilers ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 4    gcc:
 5      builtin_gcc:
 6        cc: /usr/bin/gcc
 7        cxx: /usr/bin/g++
 8        fc: /usr/bin/gfortran
 9      gcc/10.2.0-37fmsw7:
10        cc: gcc
11        cxx: g++
12        fc: gfortran
13        module:
14          load:
15          - gcc/10.2.0-37fmsw7
16          purge: true
17      gcc/9.3.0-n7p74fd:
18        cc: gcc
19        cxx: g++
20        fc: gfortran
21        module:
22          load:
23          - gcc/9.3.0-n7p74fd
24          purge: true

Enable Programming Environments

If you have a Cray based system, you will be using the Cray Programming Environments (PrgEnv-*) modulefiles to access the compilers which is the recommended way to use compilers in Cray environment. In buildtest, you can enable programming environment support which will detect and test PrgEnv-* modules. If the modules are present, buildtest will automatically add the the PrgEnv-* modules into compiler instance.

To demonstrate this let’s take a look at the following configuration that is available on Cori. To enable programming environment support, we set enable_prgenv: true which is a boolean that enables support for Programming Environments. The property prgenv_modules is a mapping of compiler groups to the corresponding PrgEnv-* modulefile. For instance PrgEnv-gnu is the programming environment modulefile that will load the GNU compiler on Cray systems.

system:
  muller:
    hostnames:
      - login01
      - login02
    moduletool: lmod
    poolsize: 8
    max_jobs: 10
    pager: False
    buildspecs:
      rebuild: False
      count: 15
      format: "name,description"
      terse: False
    report:
      count: 25
      terse: False
      format: "name,id,state,runtime,returncode"
    compilers:
      enable_prgenv: true
      prgenv_modules:
        gcc: PrgEnv-gnu
        cray: PrgEnv-cray
        nvhpc: PrgEnv-nvhpc
      find:
        gcc: '^(gcc)'
        cray: '^(cce)'
        nvhpc: '^(nvhpc)'
      compiler:
        gcc:
          builtin_gcc:
            cc: /usr/bin/gcc
            cxx: /usr/bin/g++
            fc: /usr/bin/gfortran
    executors:
      defaults:
        pollinterval: 30
        maxpendtime: 86400
      local:
        bash:
          description: submit jobs on local machine using bash shell
          shell: bash
        sh:
          description: submit jobs on local machine using sh shell
          shell: sh
        csh:
          description: submit jobs on local machine using csh shell
          shell: csh
        zsh:
          description: submit jobs on local machine using zsh shell
          shell: zsh
      slurm:
        debug:
          description: submit jobs on perlmutter using slurm debug queue
          qos: debug
  perlmutter:
    hostnames:
      - 'login(0[1-9]|[1-3][0-9]|40)'
    moduletool: lmod
    poolsize: 8
    max_jobs: 10
    pager: False
    buildspecs:
      rebuild: False
      count: 15
      format: "name,description"
      terse: False
    report:
      count: 25
      terse: False
      format: "name,id,state,runtime,returncode"
    compilers:
      enable_prgenv: true
      prgenv_modules:
        gcc: PrgEnv-gnu
        cray: PrgEnv-cray
        nvhpc: PrgEnv-nvhpc
      find:
        gcc: '^(gcc)'
        cray: '^(cce)'
        nvhpc: '^(nvhpc)'
      compiler:
        gcc:
          builtin_gcc:
            cc: /usr/bin/gcc
            cxx: /usr/bin/g++
            fc: /usr/bin/gfortran
    executors:
      defaults:
        pollinterval: 30
        maxpendtime: 86400
      local:
        bash:
          description: submit jobs on local machine using bash shell
          shell: bash
        sh:
          description: submit jobs on local machine using sh shell
          shell: sh
        csh:
          description: submit jobs on local machine using csh shell
          shell: csh
        zsh:
          description: submit jobs on local machine using zsh shell
          shell: zsh
      slurm:
        debug:
          description: submit jobs on perlmutter using slurm debug queue
          qos: debug

Now let’s run buildtest config compilers find –detailed and take note of the generated compilers, you will see that PrgEnv-* modules will be found in each compiler instance under the module, load section. Furthermore, you will see the cray wrappers cc, CC, and ftn are used instead of the compiler wrappers when defining a compiler instance that uses a Programming Environment module.

buildtest config compilers find --detailed
 buildtest config compilers find --detailed
MODULEPATH: /opt/cray/pe/perftools/21.12.0/modulefiles:/opt/cray/pe/craype-targets/default/modulefiles:/opt/cray/ari/modulefiles:/opt/cray/pe/modulefiles:/opt/cray/modulefiles:/opt/modulefiles:/global/common/software/nersc/cle7up03/modulefiles:/global/common/software/nersc/cle7up03/extra_modulefiles:/global/common/cori_cle7up03/ftg/modulefiles
Searching modules by parsing content of command: module av -t
  Discovered Modules
┏━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                ┃
┡━━━━━━━━━━━━━━━━━━━━━┩
│ gcc/7.3.0           │
├─────────────────────┤
│ gcc/8.1.0           │
├─────────────────────┤
│ gcc/8.3.0           │
├─────────────────────┤
│ gcc/10.3.0          │
├─────────────────────┤
│ gcc/11.2.0          │
├─────────────────────┤
│ craype/2.6.2        │
├─────────────────────┤
│ craype/2.7.10       │
├─────────────────────┤
│ intel/19.0.3.199    │
├─────────────────────┤
│ intel/19.1.2.254    │
├─────────────────────┤
│ intel/19.1.0.166    │
├─────────────────────┤
│ intel/19.1.1.217    │
├─────────────────────┤
│ intel/19.1.2.275    │
├─────────────────────┤
│ intel/19.1.3.304    │
├─────────────────────┤
│ upcxx/2021.9.0      │
├─────────────────────┤
│ upcxx/2022.3.0      │
├─────────────────────┤
│ upcxx/2022.9.0      │
├─────────────────────┤
│ upcxx/bleeding-edge │
├─────────────────────┤
│ upcxx/nightly       │
└─────────────────────┘
[DEBUG] Executing module command: bash -l -c "module load gcc/7.3.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load gcc/8.1.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load gcc/8.3.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load gcc/10.3.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load gcc/11.2.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load craype/2.6.2  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load craype/2.7.10  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.0.3.199  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.1.2.254  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.1.0.166  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.1.1.217  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.1.2.275  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load intel/19.1.3.304  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load upcxx/2021.9.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load upcxx/2022.3.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load upcxx/2022.9.0  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load upcxx/bleeding-edge  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load upcxx/nightly  "
[DEBUG] Return Code: 0
Testing Programming Environment Modules
[DEBUG] Executing module command: bash -l -c "module load PrgEnv-gnu  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load PrgEnv-cray  "
[DEBUG] Return Code: 0
[DEBUG] Executing module command: bash -l -c "module load PrgEnv-intel  "
[DEBUG] Return Code: 0
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Detect Compilers ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
cray:
  craype/2.6.2:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-cray
      - craype/2.6.2
      purge: false
  craype/2.7.10:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-cray
      - craype/2.7.10
      purge: false
gcc:
  builtin_gcc:
    cc: /usr/bin/gcc
    cxx: /usr/bin/g++
    fc: /usr/bin/gfortran
  gcc/10.3.0:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-gnu
      - gcc/10.3.0
      purge: false
  gcc/11.2.0:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-gnu
      - gcc/11.2.0
      purge: false
  gcc/7.3.0:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-gnu
      - gcc/7.3.0
      purge: false
  gcc/8.1.0:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-gnu
      - gcc/8.1.0
      purge: false
  gcc/8.3.0:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-gnu
      - gcc/8.3.0
      purge: false
intel:
  intel/19.0.3.199:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.0.3.199
      purge: false
  intel/19.1.0.166:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.1.0.166
      purge: false
  intel/19.1.1.217:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.1.1.217
      purge: false
  intel/19.1.2.254:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.1.2.254
      purge: false
  intel/19.1.2.275:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.1.2.275
      purge: false
  intel/19.1.3.304:
    cc: cc
    cxx: CC
    fc: ftn
    module:
      load:
      - PrgEnv-intel
      - intel/19.1.3.304
      purge: false
upcxx:
  upcxx/2021.9.0:
    cc: upcxx
    cxx: upcxx
    fc: None
    module:
      load:
      - upcxx/2021.9.0
      purge: false
  upcxx/2022.3.0:
    cc: upcxx
    cxx: upcxx
    fc: None
    module:
      load:
      - upcxx/2022.3.0
      purge: false
  upcxx/2022.9.0:
    cc: upcxx
    cxx: upcxx
    fc: None
    module:
      load:
      - upcxx/2022.9.0
      purge: false
  upcxx/bleeding-edge:
    cc: upcxx
    cxx: upcxx
    fc: None
    module:
      load:
      - upcxx/bleeding-edge
      purge: false
  upcxx/nightly:
    cc: upcxx
    cxx: upcxx
    fc: None
    module:
      load:
      - upcxx/nightly
      purge: false

Test Compilers (Experimental Feature)

Next we run buildtest config compilers test which test each compiler instance by performing module load test and show an output of each compiler.

buildtest config compilers test
$ buildtest config compilers test

                Compilers Test Pass
┏━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name                   ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩                                                                                                    [0/1858]
│ 1   │ PrgEnv-gnu/6.0.5                │     ✅ │
│ 2   │ PrgEnv-gnu/6.0.10               │     ✅ │
│ 3   │ gcc/7.3.0                       │     ✅ │
│ 4   │ gcc/8.1.0                       │     ✅ │
│ 5   │ gcc/8.3.0                       │     ✅ │
│ 6   │ gcc/10.3.0                      │     ✅ │
│ 7   │ gcc/11.2.0                      │     ✅ │
│ 8   │ PrgEnv-cray/6.0.5               │     ✅ │
│ 9   │ PrgEnv-cray/6.0.10              │     ✅ │
│ 10  │ PrgEnv-intel/6.0.5              │     ✅ │
│ 11  │ PrgEnv-intel/6.0.10             │     ✅ │
│ 12  │ intel/19.0.3.199                │     ✅ │
│ 13  │ intel/19.1.2.254                │     ✅ │
│ 14  │ intel/19.1.0.166                │     ✅ │
│ 15  │ intel/19.1.1.217                │     ✅ │
│ 16  │ intel/19.1.2.275                │     ✅ │
│ 17  │ intel/19.1.3.304                │     ✅ │
│ 18  │ upcxx/2021.9.0                  │     ✅ │
│ 19  │ upcxx/2022.3.0                  │     ✅ │
│ 20  │ upcxx/bleeding-edge             │     ✅ │
│ 21  │ upcxx/nightly                   │     ✅ │
│ 22  │ upcxx-bupc-narrow/2021.9.0      │     ✅ │
│ 23  │ upcxx-bupc-narrow/2022.3.0      │     ✅ │
│ 24  │ upcxx-bupc-narrow/bleeding-edge │     ✅ │
│ 25  │ upcxx-extras/2020.3.0           │     ✅ │
│ 26  │ upcxx-extras/2020.3.8           │     ✅ │
│ 27  │ upcxx-extras/2022.3.0           │     ✅ │
│ 28  │ upcxx-extras/master             │     ✅ │
└─────┴─────────────────────────────────┴────────┘
            Compilers Test Fail
┏━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name            ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 1   │ upcxx-gpu/2021.9.0       │     ❌ │
│ 2   │ upcxx-gpu/2022.3.0       │     ❌ │
│ 3   │ upcxx-gpu/nightly        │     ❌ │
│ 4   │ upcxx-gpu-1rail/2021.9.0 │     ❌ │
│ 5   │ upcxx-gpu-1rail/nightly  │     ❌ │
└─────┴──────────────────────────┴────────┘

If you want to test specific compilers instead of testing all compilers you can pass name of compiler as a positional argument to buildtest config compilers test and buildtest will only test the selected compiler. Shown below is an example where we only test compiler gcc/9.1.01

buildtest config compilers test gcc/9.1.0
$ buildtest config compilers test gcc/9.1.0
Skipping test for compiler: builtin_gcc
Skipping test for compiler: gcc/9.3.0
Skipping test for compiler: gcc/11.1.0
Skipping test for compiler: gcc/7.5.0
Skipping test for compiler: gcc/12.1.0
Skipping test for compiler: gcc/11.2.0
Skipping test for compiler: gcc/10.2.0
      Compilers Test Pass
┏━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 1   │ gcc/9.1.0     │     ✅ │
└─────┴───────────────┴────────┘

Removing Compilers

The buildtest config compilers remove command allows you to remove a compiler from the configuration file. This command takes a positional argument that includes the name of compiler you want to remove. buildtest will search for compiler name in configuration file and attempt to remove it if it’s found.

This command can be used when you find compilers are out of date, for instance you can run buildtest config compilers test to test all compilers and show which compilers failed during test. In output below we have one failed compiler name nvhpc-mixed/23.7.

$ buildtest config compilers test
Compiler 'builtin_gcc' has no 'modules' defined therefore skipping test
        Compilers Test Pass
┏━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name     ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 1   │ builtin_gcc       │     ✅ │
│ 2   │ gcc/11.2.0        │     ✅ │
│ 3   │ gcc/10.3.0        │     ✅ │
│ 4   │ gcc/12.2.0        │     ✅ │
│ 5   │ gcc/12.1.0        │     ✅ │
│ 6   │ cce/15.0.1        │     ✅ │
│ 7   │ cce/15.0.0        │     ✅ │
│ 8   │ nvhpc/21.11       │     ✅ │
│ 9   │ nvhpc/21.9        │     ✅ │
│ 10  │ nvhpc/22.7        │     ✅ │
│ 11  │ nvhpc-mixed/21.11 │     ✅ │
│ 12  │ nvhpc-mixed/21.9  │     ✅ │
│ 13  │ nvhpc-mixed/22.7  │     ✅ │
│ 14  │ nvhpc/23.1        │     ✅ │
│ 15  │ nvhpc-mixed/23.1  │     ✅ │
└─────┴───────────────────┴────────┘
        Compilers Test Fail
┏━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name    ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 1   │ nvhpc-mixed/23.7 │     ❌ │
└─────┴──────────────────┴────────┘

Note we have tab completion for compilers names that can be removed which can help select the compiler you want to remove.

$ buildtest config compilers remove
builtin_gcc        cce/15.0.1         gcc/11.2.0         gcc/12.2.0         nvhpc/21.9         nvhpc/23.1         nvhpc-mixed/21.9   nvhpc-mixed/23.1
cce/15.0.0         gcc/10.3.0         gcc/12.1.0         nvhpc/21.11        nvhpc/22.7         nvhpc-mixed/21.11  nvhpc-mixed/22.7   nvhpc-mixed/23.7

Let’s remove the compiler nvhpc-mixed/23.7 by running the following, take note buildtest will update the configuration file

$ buildtest config compilers remove nvhpc-mixed/23.7
──────────────────────────────────────────────────────────────────────────────── Removing compiler name: nvhpc-mixed/23.7 ────────────────────────────────────────────────────────────────────────────────
cc: cc
cxx: CC
fc: ftn
module:
  load:
  - PrgEnv-nvidia
  - nvhpc-mixed/23.7
  purge: false

Updating configuration file: /global/u1/s/siddiq90/gitrepos/buildtest-nersc/config.yml

Now if we rerun the compiler tests, we see all compilers have passed checks

$ buildtest config compilers test
Compiler 'builtin_gcc' has no 'modules' defined therefore skipping test
        Compilers Test Pass
┏━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ No. ┃ Compiler Name     ┃ Status ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 1   │ builtin_gcc       │     ✅ │
│ 2   │ gcc/11.2.0        │     ✅ │
│ 3   │ gcc/10.3.0        │     ✅ │
│ 4   │ gcc/12.2.0        │     ✅ │
│ 5   │ gcc/12.1.0        │     ✅ │
│ 6   │ cce/15.0.1        │     ✅ │
│ 7   │ cce/15.0.0        │     ✅ │
│ 8   │ nvhpc/21.11       │     ✅ │
│ 9   │ nvhpc/21.9        │     ✅ │
│ 10  │ nvhpc/22.7        │     ✅ │
│ 11  │ nvhpc-mixed/21.11 │     ✅ │
│ 12  │ nvhpc-mixed/21.9  │     ✅ │
│ 13  │ nvhpc-mixed/22.7  │     ✅ │
│ 14  │ nvhpc/23.1        │     ✅ │
│ 15  │ nvhpc-mixed/23.1  │     ✅ │
└─────┴───────────────────┴────────┘