Note

Please see Tutorials Setup before you proceed with this section

Buildtest Spack Integration

Note

This feature is in active development.

buildtest can use spack to build test where one can use spack to install packages followed by running any test. You must set type: spack in buildspec to use the spack schema for validating the buildspec test. Currently, we have spack.schema.json JSON schema that defines the structure of how tests are to be written in buildspec. Shown below is the schema header. The required properties are type, executor and spack.

{
  "$id": "spack.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "spack schema version",
  "description": "The spack schema is referenced using ``type: spack`` which is used for generating tests using spack package manager",
  "type": "object",
  "required": [
    "type",
    "executor",
    "spack"
  ],
  "additionalProperties": false,

Install Specs

Let’s start off with a simple example where we create a test that can spack install zlib. Shown below is a test named install_zlib. The spack keyword is a JSON object, in this test we define the root of spack using the root keyword which informs buildtest where spack is located. buildtest will automatically check the path and source the startup script. The install field is a JSON object that contains a specs property which is a list of strings types that are name of spack packages to install. Each item in the specs property will be added as a separate spack install command. In the second test, we run the same example except we clone spack and install zlib. When root property is not specified buildtest will clone buildtest and automatically source the setup script.

The schema is designed to mimic spack commands which will be clear with more examples.

buildspecs:
  install_specs_example:
    type: spack
    executor: generic.local.bash
    description: "Install zlib from an existing spack instance"
    tags: [spack]
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib']

  clone_spack_and_install_zlib:
    type: spack
    executor: generic.local.bash
    description: "Clone spack and install zlib spec"
    tags: [spack]
    spack:
      install:
        specs: ['zlib']

Let’s build this test by running the following

buildtest build -b /home/spack/buildtest/examples/spack/install_specs.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/install_specs.yml 
Buildspec Paths: ['/home/spack/buildtest/examples']
Updating buildspec cache file: /home/spack/buildtest/var/buildspecs/cache.json
╭─────────────────────── buildtest summary ────────────────────────╮
│                                                                  │
│ User:               spack                                        │
│ Hostname:           356ea13b3433                                 │
│ Platform:           Linux                                        │
│ Current Time:       2022/06/29 17:25:02                          │
│ buildtest path:     /home/spack/buildtest/bin/buildtest          │
│ buildtest version:  0.14.0                                       │
│ python path:        /home/spack/pyenv/buildtest/bin/python3      │
│ python version:     3.8.6                                        │
│ Configuration File:                                              │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml     │
│ Test Directory:     /home/spack/runs                             │
│ Report File:        /home/spack/buildtest/var/report.json        │
│ Command:            /home/spack/buildtest/bin/buildtest build -b │
│ /home/spack/buildtest/examples/spack/install_specs.yml           │
│                                                                  │
╰──────────────────────────────────────────────────────────────────╯
─────────────────────  Discovering Buildspecs ──────────────────────
                  Discovered buildspecs                   
╔════════════════════════════════════════════════════════╗
║ buildspec                                              ║
╟────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/install_specs.yml ║
╚════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
──────────────────────── Parsing Buildspecs ────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/install_specs.yml: VALID
Total builder objects created: 2
Total compiler builder: 0
Total script builder: 0
Total spack builder: 2
                       Spack Builder Details                        
┏━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ execut… ┃ compil… ┃ nodes ┃ procs   ┃ descr… ┃ builds… ┃
┡━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│ instal… │ generi… │ None    │ None  │ Install │ /home… │         │
│         │         │         │       │ zlib    │        │         │
│         │         │         │       │ from an │        │         │
│         │         │         │       │ existi… │        │         │
│         │         │         │       │ spack   │        │         │
│         │         │         │       │ instan… │        │         │
├─────────┼─────────┼─────────┼───────┼─────────┼────────┼─────────┤
│ clone_… │ generi… │ None    │ None  │ Clone   │ /home… │         │
│         │         │         │       │ spack   │        │         │
│         │         │         │       │ and     │        │         │
│         │         │         │       │ install │        │         │
│         │         │         │       │ zlib    │        │         │
│         │         │         │       │ spec    │        │         │
└─────────┴─────────┴─────────┴───────┴─────────┴────────┴─────────┘
────────────────────────── Building Test ───────────────────────────
install_specs_example/c637d366: Creating test directory: /home/spack
/runs/generic.local.bash/install_specs/install_specs_example/c637d36
6
install_specs_example/c637d366: Creating the stage directory: /home/
spack/runs/generic.local.bash/install_specs/install_specs_example/c6
37d366/stage
install_specs_example/c637d366: Writing build script: /home/spack/ru
ns/generic.local.bash/install_specs/install_specs_example/c637d366/i
nstall_specs_example_build.sh
clone_spack_and_install_zlib/53e91673: Creating test directory: /hom
e/spack/runs/generic.local.bash/install_specs/clone_spack_and_instal
l_zlib/53e91673
clone_spack_and_install_zlib/53e91673: Creating the stage directory:
/home/spack/runs/generic.local.bash/install_specs/clone_spack_and_in
stall_zlib/53e91673/stage
clone_spack_and_install_zlib/53e91673: Writing build script: /home/s
pack/runs/generic.local.bash/install_specs/clone_spack_and_install_z
lib/53e91673/clone_spack_and_install_zlib_build.sh
────────────────────────── Running Tests ───────────────────────────
Spawning 4 processes for processing builders
─────────────────────────── Iteration 1 ────────────────────────────
install_specs_example/c637d366 does not have any dependencies adding
test to queue
clone_spack_and_install_zlib/53e91673 does not have any dependencies
adding test to queue
install_specs_example/c637d366: Running Test via command: bash 
--norc --noprofile -eo pipefail install_specs_example_build.sh
clone_spack_and_install_zlib/53e91673: Running Test via command: 
bash --norc --noprofile -eo pipefail 
clone_spack_and_install_zlib_build.sh
install_specs_example/c637d366: Test completed in 0.899835 seconds
install_specs_example/c637d366: Test completed with returncode: 0
install_specs_example/c637d366: Writing output file -  /home/spack/r
uns/generic.local.bash/install_specs/install_specs_example/c637d366/
install_specs_example.out
install_specs_example/c637d366: Writing error file - /home/spack/run
s/generic.local.bash/install_specs/install_specs_example/c637d366/in
stall_specs_example.err
clone_spack_and_install_zlib/53e91673: Test completed in 93.305672 
seconds
clone_spack_and_install_zlib/53e91673: Test completed with returncode: 0
clone_spack_and_install_zlib/53e91673: Writing output file -  /home/spack
/runs/generic.local.bash/install_specs/clone_spack_and_install_zlib/53e91
673/clone_spack_and_install_zlib.out
clone_spack_and_install_zlib/53e91673: Writing error file - /home/spack/r
uns/generic.local.bash/install_specs/clone_spack_and_install_zlib/53e9167
3/clone_spack_and_install_zlib.err
In this iteration we are going to run the following tests: [install_specs_example/c637d366, clone_spack_and_install_zlib/53e91673]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃           ┃           ┃
┃            ┃            ┃        ┃ turnCode,  ┃           ┃           ┃
┃            ┃            ┃        ┃ Regex,     ┃           ┃           ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCo… ┃ runtime   ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━┩
│ clone_spac │ generic.l… │ PASS   │ N/A N/A    │ 0         │ 93.305672 │
│ k_and_inst │            │        │ N/A        │           │           │
│ all_zlib/5 │            │        │            │           │           │
│ 3e91673    │            │        │            │           │           │
├────────────┼────────────┼────────┼────────────┼───────────┼───────────┤
│ install_sp │ generic.l… │ PASS   │ N/A N/A    │ 0         │ 0.899835  │
│ ecs_exampl │            │        │ N/A        │           │           │
│ e/c637d366 │            │        │            │           │           │
└────────────┴────────────┴────────┴────────────┴───────────┴───────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_24v80ou9.log

Let’s inspect the generated script and output file via buildtest inspect query command. We notice that buildtest will source spack setup script and install zlib which is automatically installed from the buildcache. In the second test we clone spack and it will install zlib from source.

buildtest inspect query -o -t install_specs_example clone_spack_and_install_zlib
$ buildtest inspect query -o -t install_specs_example clone_spack_and_install_zlib 
────── install_specs_example/c637d366-da1a-428a-a9c7-46ce17b6344b ───────
Executor: generic.local.bash
Description: Install zlib from an existing spack instance
State: PASS
Returncode: 0
Runtime: 0.899835 sec
Starttime: 2022/06/29 17:25:03
Endtime: 2022/06/29 17:25:03
Command: bash --norc --noprofile -eo pipefail 
install_specs_example_build.sh
Test Script: /home/spack/runs/generic.local.bash/install_specs/install_sp
ecs_example/c637d366/install_specs_example.sh
Build Script: /home/spack/runs/generic.local.bash/install_specs/install_s
pecs_example/c637d366/install_specs_example_build.sh
Output File: /home/spack/runs/generic.local.bash/install_specs/install_sp
ecs_example/c637d366/install_specs_example.out
Error File: /home/spack/runs/generic.local.bash/install_specs/install_spe
cs_example/c637d366/install_specs_example.err
Log File: /home/spack/buildtest/var/logs/buildtest_24v80ou9.log
─ Output File: /home/spack/runs/generic.local.bash/install_specs/insta… ─
[+] /home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/install_specs/install… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack install  zlib                                                      
─── clone_spack_and_install_zlib/53e91673-45d8-4fb0-b09b-a4fb6cfa9f6c ───
Executor: generic.local.bash
Description: Clone spack and install zlib spec
State: PASS
Returncode: 0
Runtime: 93.305672 sec
Starttime: 2022/06/29 17:25:03
Endtime: 2022/06/29 17:26:36
Command: bash --norc --noprofile -eo pipefail 
clone_spack_and_install_zlib_build.sh
Test Script: /home/spack/runs/generic.local.bash/install_specs/clone_spac
k_and_install_zlib/53e91673/clone_spack_and_install_zlib.sh
Build Script: /home/spack/runs/generic.local.bash/install_specs/clone_spa
ck_and_install_zlib/53e91673/clone_spack_and_install_zlib_build.sh
Output File: /home/spack/runs/generic.local.bash/install_specs/clone_spac
k_and_install_zlib/53e91673/clone_spack_and_install_zlib.out
Error File: /home/spack/runs/generic.local.bash/install_specs/clone_spack
_and_install_zlib/53e91673/clone_spack_and_install_zlib.err
Log File: /home/spack/buildtest/var/logs/buildtest_24v80ou9.log
─ Output File: /home/spack/runs/generic.local.bash/install_specs/clone… ─
==> Bootstrapping clingo from pre-built binaries                         
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_
==> Installing "clingo-bootstrap@spack%gcc@10.2.1~docs~ipo+python+static_
==> Bootstrapping patchelf from pre-built binaries                       
==> Bootstrapping patchelf@0.13.1:0.13.99 %gcc target=x86_64 from sources
==> Installing patchelf-0.13.1-fw7rhbzvu2cwstdjpennwm7hdlegz47u          
==> No binary for patchelf-0.13.1-fw7rhbzvu2cwstdjpennwm7hdlegz47u found:
==> Fetching https://mirror.spack.io/_source-cache/archive/08/08c0237e89b
==> No patches needed for patchelf                                       
==> patchelf: Executing phase: 'autoreconf'                              
==> patchelf: Executing phase: 'configure'                               
==> patchelf: Executing phase: 'build'                                   
==> patchelf: Executing phase: 'install'                                 
==> patchelf: Successfully installed patchelf-0.13.1-fw7rhbzvu2cwstdjpenn
  Fetch: 1.11s.  Build: 8.75s.  Total: 9.86s.                            
[+] /home/spack/.spack/bootstrap/store/linux-ubuntu18.04-x86_64/gcc-7.5.0
==> Installing zlib-1.2.12-m4fqokrwbprfci4rptvgevnklpdo3dhd              
==> No binary for zlib-1.2.12-m4fqokrwbprfci4rptvgevnklpdo3dhd found: ins
==> Fetching https://mirror.spack.io/_source-cache/archive/91/91844808532
==> Applied patch /home/spack/runs/generic.local.bash/install_specs/clone
==> zlib: Executing phase: 'install'                                     
==> zlib: Successfully installed zlib-1.2.12-m4fqokrwbprfci4rptvgevnklpdo
  Fetch: 0.27s.  Build: 3.27s.  Total: 3.54s.                            
[+] /home/spack/runs/generic.local.bash/install_specs/clone_spack_and_ins
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/install_specs/clone_s… ─
#!/bin/bash                                                              
git clone https://github.com/spack/spack                                 
source ./spack/share/spack/setup-env.sh                                  
spack install  zlib

Spack Environment

buildtest can generate scripts to make use of spack environments which can be useful if you want to install or test specs in an isolated environment.

Currently, we can create spack environment (spack env create) via name, directory and manifest file (spack.yaml, spack.lock) and pass any options to spack env create command. Furthermore, we can activate existing spack environment via name or directory using spack env activate and pass options to the command. buildtest can remove spack environments automatically before creating spack environment or one can explicitly specify by name.

Create a Spack Environment by name

In this next example, we will create a spack environment named m4_zlib that will install m4 and zlib spec. The create field is a JSON object that maps to spack env create command which can pass some arguments in the form of key/value pairs. The name property in create section is used to create a spack environment by name. The activate property maps to spack env activate command which is used to activate a spack environment. The name property is of type: string which is name of spack environment you want to activate.

The compiler_find: true is a boolean that determines if we need to find compilers in spack via spack compiler find. This can be useful if you need to find compilers so spack can install specs with a preferred compiler otherwise spack may have issues concretizing or install specs. buildtest will run spack compiler find after sourcing spack.

Note

The compiler_find option may not be useful if your compilers are already defined in one of your configuration scopes or spack.yaml that is part of your spack environment.

The option field can pass any command line arguments to spack install command and this field is available for other properties.

buildspecs:
  install_in_spack_env:
    type: spack
    executor: generic.local.bash
    description: "Install m4 and zlib in a spack environment named m4_zlib"
    tags: [spack]
    spack:
      root: $HOME/spack
      compiler_find: true
      env:
        create:
          name: 'm4_zlib'
        specs:
          - 'm4'
          - 'zlib'
        activate:
          name: m4_zlib
        concretize: true
      install:
        option: '--keep-prefix'

If we build this test and see generated test we see that buildtest will create a spack environment m4_zlib and activate the environment, add m4 and zlib, concretize the environment and install the specs.

buildtest build -b /home/spack/buildtest/examples/spack/env_install.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_install.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:26:38                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/env_install.yml                  │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                 Discovered buildspecs                  
╔══════════════════════════════════════════════════════╗
║ buildspec                                            ║
╟──────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_install.yml ║
╚══════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_install.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ install… │ generic… │ None     │ None  │ Install │ /home/s… │         │
│          │          │          │       │ m4 and  │          │         │
│          │          │          │       │ zlib in │          │         │
│          │          │          │       │ a spack │          │         │
│          │          │          │       │ enviro… │          │         │
│          │          │          │       │ named   │          │         │
│          │          │          │       │ m4_zlib │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
install_in_spack_env/904d2353: Creating test directory: /home/spack/runs/
generic.local.bash/env_install/install_in_spack_env/904d2353
install_in_spack_env/904d2353: Creating the stage directory: /home/spack/
runs/generic.local.bash/env_install/install_in_spack_env/904d2353/stage
install_in_spack_env/904d2353: Writing build script: /home/spack/runs/gen
eric.local.bash/env_install/install_in_spack_env/904d2353/install_in_spac
k_env_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
install_in_spack_env/904d2353 does not have any dependencies adding test 
to queue
install_in_spack_env/904d2353: Running Test via command: bash --norc 
--noprofile -eo pipefail install_in_spack_env_build.sh
install_in_spack_env/904d2353: Test completed in 4.531373 seconds
install_in_spack_env/904d2353: Test completed with returncode: 0
install_in_spack_env/904d2353: Writing output file -  /home/spack/runs/ge
neric.local.bash/env_install/install_in_spack_env/904d2353/install_in_spa
ck_env.out
install_in_spack_env/904d2353: Writing error file - /home/spack/runs/gene
ric.local.bash/env_install/install_in_spack_env/904d2353/install_in_spack
_env.err
In this iteration we are going to run the following tests: [install_in_spack_env/904d2353]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ install_in │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 4.531373 │
│ _spack_env │            │        │ N/A        │            │          │
│ /904d2353  │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_7jdvol0q.log
buildtest inspect query -t install_in_spack_env
$ buildtest inspect query -t install_in_spack_env 
─────── install_in_spack_env/904d2353-8315-4746-8f82-7feb113ba458 ───────
Executor: generic.local.bash
Description: Install m4 and zlib in a spack environment named m4_zlib
State: PASS
Returncode: 0
Runtime: 4.531373 sec
Starttime: 2022/06/29 17:26:39
Endtime: 2022/06/29 17:26:43
Command: bash --norc --noprofile -eo pipefail 
install_in_spack_env_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_install/install_in_s
pack_env/904d2353/install_in_spack_env.sh
Build Script: /home/spack/runs/generic.local.bash/env_install/install_in_
spack_env/904d2353/install_in_spack_env_build.sh
Output File: /home/spack/runs/generic.local.bash/env_install/install_in_s
pack_env/904d2353/install_in_spack_env.out
Error File: /home/spack/runs/generic.local.bash/env_install/install_in_sp
ack_env/904d2353/install_in_spack_env.err
Log File: /home/spack/buildtest/var/logs/buildtest_7jdvol0q.log
─ Test File: /home/spack/runs/generic.local.bash/env_install/install_i… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack compiler find                                                      
spack env create  m4_zlib                                                
spack env activate  m4_zlib                                              
spack add m4                                                             
spack add zlib                                                           
spack concretize -f                                                      
spack install --keep-prefix

Creating Spack Environment in Directory

We can create spack environment from a directory using the dir property that is available as part of create and activate field. In this next example we create a spack environment in our $HOME directory and concretize m4 in the spack environment

buildspecs:
  spack_env_directory:
    type: spack
    executor: generic.local.bash
    description: "create spack environment in directory"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          dir: $HOME/spack-envs/m4
        activate:
          dir: $HOME/spack-envs/m4
        specs:
          - 'm4'
      install:
        option: ''
    post_cmds: |
      spack find
      rm -rf $HOME/spack-envs/m4

When creating spack environment using directory, buildtest will automatically add the -d option which is required when creating spack environments. However, one can also pass this using the option field. Shown below is the build and generated script after running test.

buildtest build -b /home/spack/buildtest/examples/spack/env_create_directory.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_create_directory.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:26:44                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/env_create_directory.yml         │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                      Discovered buildspecs                      
╔═══════════════════════════════════════════════════════════════╗
║ buildspec                                                     ║
╟───────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_create_directory.yml ║
╚═══════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_create_directory.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ spack_e… │ generic… │ None     │ None  │ create  │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ enviro… │          │         │
│          │          │          │       │ in      │          │         │
│          │          │          │       │ direct… │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
spack_env_directory/6c67e3cf: Creating test directory: /home/spack/runs/g
eneric.local.bash/env_create_directory/spack_env_directory/6c67e3cf
spack_env_directory/6c67e3cf: Creating the stage directory: /home/spack/r
uns/generic.local.bash/env_create_directory/spack_env_directory/6c67e3cf/
stage
spack_env_directory/6c67e3cf: Writing build script: /home/spack/runs/gene
ric.local.bash/env_create_directory/spack_env_directory/6c67e3cf/spack_en
v_directory_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
spack_env_directory/6c67e3cf does not have any dependencies adding test 
to queue
spack_env_directory/6c67e3cf: Running Test via command: bash --norc 
--noprofile -eo pipefail spack_env_directory_build.sh
spack_env_directory/6c67e3cf: Test completed in 2.344294 seconds
spack_env_directory/6c67e3cf: Test completed with returncode: 0
spack_env_directory/6c67e3cf: Writing output file -  /home/spack/runs/gen
eric.local.bash/env_create_directory/spack_env_directory/6c67e3cf/spack_e
nv_directory.out
spack_env_directory/6c67e3cf: Writing error file - /home/spack/runs/gener
ic.local.bash/env_create_directory/spack_env_directory/6c67e3cf/spack_env
_directory.err
In this iteration we are going to run the following tests: [spack_env_directory/6c67e3cf]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ spack_env_ │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 2.344294 │
│ directory/ │            │        │ N/A        │            │          │
│ 6c67e3cf   │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_ag113i__.log
buildtest inspect query -o -t spack_env_directory
$ buildtest inspect query -o -t spack_env_directory 
─────── spack_env_directory/6c67e3cf-a2e2-4b58-a0a0-5f100fb996ad ────────
Executor: generic.local.bash
Description: create spack environment in directory
State: PASS
Returncode: 0
Runtime: 2.344294 sec
Starttime: 2022/06/29 17:26:44
Endtime: 2022/06/29 17:26:46
Command: bash --norc --noprofile -eo pipefail 
spack_env_directory_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_create_directory/spa
ck_env_directory/6c67e3cf/spack_env_directory.sh
Build Script: /home/spack/runs/generic.local.bash/env_create_directory/sp
ack_env_directory/6c67e3cf/spack_env_directory_build.sh
Output File: /home/spack/runs/generic.local.bash/env_create_directory/spa
ck_env_directory/6c67e3cf/spack_env_directory.out
Error File: /home/spack/runs/generic.local.bash/env_create_directory/spac
k_env_directory/6c67e3cf/spack_env_directory.err
Log File: /home/spack/buildtest/var/logs/buildtest_ag113i__.log
─ Output File: /home/spack/runs/generic.local.bash/env_create_director… ─
==> Updating view at /home/spack/spack-envs/m4/.spack-env/view           
==> Created environment in /home/spack/spack-envs/m4                     
==> You can activate this environment with:                              
==>   spack env activate /home/spack/spack-envs/m4                       
==> Adding m4 to environment /home/spack/spack-envs/m4                   
==> Updating view at /home/spack/spack-envs/m4/.spack-env/view           
==> Concretized m4                                                       
[+]  mkc3u4x  m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2
[+]  lbrx7ln      ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_6
==> Installing environment /home/spack/spack-envs/m4                     
==> All of the packages are already installed                            
==> Updating view at /home/spack/spack-envs/m4/.spack-env/view           
==> In environment /home/spack/spack-envs/m4                             
==> Root specs                                                           
m4                                                                       
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------        
libsigsegv@2.12                                                          
m4@1.4.18                                                                
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/env_create_directory/… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env create  -d /home/spack/spack-envs/m4                           
spack env activate  -d /home/spack/spack-envs/m4                         
spack add m4                                                             
spack install                                                            
                                                                         
                                                                         
######## START OF POST COMMANDS ########                                 
spack find                                                               
rm -rf $HOME/spack-envs/m4                                               
######## END OF POST COMMANDS   ########

Create Spack Environment from Manifest File (spack.yaml, spack.lock)

Spack can create environments from spack.yaml or spack.lock which can be used if you have a spack configuration that works for your system and want to write a buildspec. While creating a spack environment, you can use the manifest property to specify path to your spack.yaml or spack.lock.

Note

buildtest will not enforce that manifest names be spack.yaml or spack.lock since spack allows one to create spack environment from arbitrary name so long as it is a valid spack configuration.

Shown below is an example buildspec that generates a test from a manifest file. The manifest property is of type: string and this is only available as part of create property.

buildspecs:
  spack_env_create_from_manifest:
    type: spack
    executor: generic.local.bash
    description: "Create spack environment from spack.yaml"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          name: 'manifest_example'
          manifest: "$BUILDTEST_ROOT/examples/spack/example/spack.yaml"
        activate:
          name: 'manifest_example'
        concretize: true

If we build this test and inspect the generated script we see spack env create command will create an environment manifest_example using the manifest file that we provided from the spack.yaml.

buildtest build -b /home/spack/buildtest/examples/spack/env_create_manifest.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_create_manifest.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:26:47                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/env_create_manifest.yml          │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                     Discovered buildspecs                      
╔══════════════════════════════════════════════════════════════╗
║ buildspec                                                    ║
╟──────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_create_manifest.yml ║
╚══════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_create_manifest.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ spack_e… │ generic… │ None     │ None  │ Create  │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ enviro… │          │         │
│          │          │          │       │ from    │          │         │
│          │          │          │       │ spack.… │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
spack_env_create_from_manifest/aaca792c: Creating test directory: /home/s
pack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_ma
nifest/aaca792c
spack_env_create_from_manifest/aaca792c: Creating the stage directory: /h
ome/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_fr
om_manifest/aaca792c/stage
spack_env_create_from_manifest/aaca792c: Writing build script: /home/spac
k/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manif
est/aaca792c/spack_env_create_from_manifest_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
spack_env_create_from_manifest/aaca792c does not have any dependencies 
adding test to queue
spack_env_create_from_manifest/aaca792c: Running Test via command: bash 
--norc --noprofile -eo pipefail spack_env_create_from_manifest_build.sh
spack_env_create_from_manifest/aaca792c: Test completed in 4.026969 
seconds
spack_env_create_from_manifest/aaca792c: Test completed with returncode: 
0
spack_env_create_from_manifest/aaca792c: Writing output file -  /home/spa
ck/runs/generic.local.bash/env_create_manifest/spack_env_create_from_mani
fest/aaca792c/spack_env_create_from_manifest.out
spack_env_create_from_manifest/aaca792c: Writing error file - /home/spack
/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manife
st/aaca792c/spack_env_create_from_manifest.err
In this iteration we are going to run the following tests: [spack_env_create_from_manifest/aaca792c]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ spack_env_ │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 4.026969 │
│ create_fro │            │        │ N/A        │            │          │
│ m_manifest │            │        │            │            │          │
│ /aaca792c  │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_8hqm6wf8.log
buildtest inspect query -o -t spack_env_create_from_manifest
$ buildtest inspect query -o -t spack_env_create_from_manifest 
── spack_env_create_from_manifest/aaca792c-4625-48d1-9b43-eb47b8d636d0 ──
Executor: generic.local.bash
Description: Create spack environment from spack.yaml
State: PASS
Returncode: 0
Runtime: 4.026969 sec
Starttime: 2022/06/29 17:26:47
Endtime: 2022/06/29 17:26:51
Command: bash --norc --noprofile -eo pipefail 
spack_env_create_from_manifest_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_create_manifest/spac
k_env_create_from_manifest/aaca792c/spack_env_create_from_manifest.sh
Build Script: /home/spack/runs/generic.local.bash/env_create_manifest/spa
ck_env_create_from_manifest/aaca792c/spack_env_create_from_manifest_build
.sh
Output File: /home/spack/runs/generic.local.bash/env_create_manifest/spac
k_env_create_from_manifest/aaca792c/spack_env_create_from_manifest.out
Error File: /home/spack/runs/generic.local.bash/env_create_manifest/spack
_env_create_from_manifest/aaca792c/spack_env_create_from_manifest.err
Log File: /home/spack/buildtest/var/logs/buildtest_8hqm6wf8.log
─ Output File: /home/spack/runs/generic.local.bash/env_create_manifest… ─
==> Created environment 'manifest_example' in /home/spack/spack/var/spack
==> You can activate this environment with:                              
==>   spack env activate manifest_example                                
==> Concretized m4                                                       
 -   mkc3u4x  m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2
 -   lbrx7ln      ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_6
==> Concretized zlib                                                     
 -   smoyzzo  zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu
==> Concretized python                                                   
 -   szj7juk  python@3.8.6%gcc@7.5.0+bz2+ctypes+dbm~debug+libxml2+lzma~ni
 -   fvfpt26      ^bzip2@1.0.8%gcc@7.5.0+shared arch=linux-ubuntu18.04-x8
 -   otkkten          ^diffutils@3.7%gcc@7.5.0 arch=linux-ubuntu18.04-x86
 -   jearpk4              ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04
 -   ba7brxj      ^expat@2.2.10%gcc@7.5.0+libbsd arch=linux-ubuntu18.04-x
 -   u6ue7vw          ^libbsd@0.10.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86
 -   4av4gyw      ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64   
 -   t54jzdy          ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_
 -   crhlefo              ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=li
 -   4sh6pym                  ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu1
 -   lbb45to      ^gettext@0.21%gcc@7.5.0+bzip2+curses+git~libunistring+l
 -   yn2r3wf          ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu1
 -   komekkm              ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-
 -   smoyzzo              ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch
 -   uwe6tb5          ^tar@1.32%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64  
 -   hyhbnrm      ^libffi@3.3%gcc@7.5.0 patches=26f26c6f29a7ce9bf370ad3ab
 -   wuyj2ax      ^libuuid@1.0.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64 
 -   es377uq      ^openssl@1.1.1h%gcc@7.5.0+systemcerts arch=linux-ubuntu
 -   zfdvt2j          ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=li
 -   4ihuiaz              ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubunt
 -   rhv2o7b      ^sqlite@3.33.0%gcc@7.5.0+column_metadata+fts~functions~
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/env_create_manifest/s… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env create  manifest_example /home/spack/buildtest/examples/spack/e
spack env activate  manifest_example                                     
spack concretize -f

Removing Spack Environments

buildtest can remove spack environments which can be used if you are periodically running the same test where one is creating the same environment. buildtest can automatically remove spack environment using the property remove_environment which will remove the environment before creating it with same name. This field is part of the create field and only works if one is creating spack environments by name.

Alternately, buildtest provides the rm field which can be used for removing environment explicitly. In the rm field, the name is a required field which is the name of the spack environment to remove. The name field is of type: string Shown below are two example tests where we remove spack environment using the remove_environment and rm field.

buildspecs:
  remove_environment_automatically:
    type: spack
    executor: generic.local.bash
    description: "remove spack environment automatically before creating a new environment"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          remove_environment: true
          name: remove_environment
        activate:
          name: remove_environment
        specs:
          - 'bzip2'
        concretize: true

  remove_environment_explicit:
    type: spack
    executor: generic.local.bash
    description: "remove spack environment explicitly using the 'rm' property"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        rm:
          name: dummy
        create:
          name: dummy
        activate:
          name: dummy
        specs:
          - 'bzip2'
        concretize: true

Let’s build this by running the following

buildtest build -b /home/spack/buildtest/examples/spack/remove_environment_example.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/remove_environment_example.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:26:52                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/remove_environment_example.yml   │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                         Discovered buildspecs                         
╔═════════════════════════════════════════════════════════════════════╗
║ buildspec                                                           ║
╟─────────────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/remove_environment_example.yml ║
╚═════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/remove_environment_example.yml: 
VALID
Total builder objects created: 2
Total compiler builder: 0
Total script builder: 0
Total spack builder: 2
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ remove_… │ generic… │ None     │ None  │ remove  │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ enviro… │          │         │
│          │          │          │       │ automa… │          │         │
│          │          │          │       │ before  │          │         │
│          │          │          │       │ creati… │          │         │
│          │          │          │       │ a new   │          │         │
│          │          │          │       │ enviro… │          │         │
├──────────┼──────────┼──────────┼───────┼─────────┼──────────┼─────────┤
│ remove_… │ generic… │ None     │ None  │ remove  │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ enviro… │          │         │
│          │          │          │       │ explic… │          │         │
│          │          │          │       │ using   │          │         │
│          │          │          │       │ the     │          │         │
│          │          │          │       │ 'rm'    │          │         │
│          │          │          │       │ proper… │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
remove_environment_automatically/c5bb2a6f: Creating test directory: /home
/spack/runs/generic.local.bash/remove_environment_example/remove_environm
ent_automatically/c5bb2a6f
remove_environment_automatically/c5bb2a6f: Creating the stage directory: 
/home/spack/runs/generic.local.bash/remove_environment_example/remove_env
ironment_automatically/c5bb2a6f/stage
remove_environment_automatically/c5bb2a6f: Writing build script: /home/sp
ack/runs/generic.local.bash/remove_environment_example/remove_environment
_automatically/c5bb2a6f/remove_environment_automatically_build.sh
remove_environment_explicit/a1ddfbc0: Creating test directory: /home/spac
k/runs/generic.local.bash/remove_environment_example/remove_environment_e
xplicit/a1ddfbc0
remove_environment_explicit/a1ddfbc0: Creating the stage directory: /home
/spack/runs/generic.local.bash/remove_environment_example/remove_environm
ent_explicit/a1ddfbc0/stage
remove_environment_explicit/a1ddfbc0: Writing build script: /home/spack/r
uns/generic.local.bash/remove_environment_example/remove_environment_expl
icit/a1ddfbc0/remove_environment_explicit_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
remove_environment_explicit/a1ddfbc0 does not have any dependencies 
adding test to queue
remove_environment_automatically/c5bb2a6f does not have any dependencies 
adding test to queue
remove_environment_explicit/a1ddfbc0: Running Test via command: bash 
--norc --noprofile -eo pipefail remove_environment_explicit_build.sh
remove_environment_automatically/c5bb2a6f: Running Test via command: bash
--norc --noprofile -eo pipefail remove_environment_automatically_build.sh
remove_environment_automatically/c5bb2a6f: Test completed in 2.409277 
seconds
remove_environment_automatically/c5bb2a6f: Test completed with 
returncode: 0
remove_environment_automatically/c5bb2a6f: Writing output file -  /home/s
pack/runs/generic.local.bash/remove_environment_example/remove_environmen
t_automatically/c5bb2a6f/remove_environment_automatically.out
remove_environment_automatically/c5bb2a6f: Writing error file - /home/spa
ck/runs/generic.local.bash/remove_environment_example/remove_environment_
automatically/c5bb2a6f/remove_environment_automatically.err
remove_environment_explicit/a1ddfbc0: Test completed in 2.442686 seconds
remove_environment_explicit/a1ddfbc0: Test completed with returncode: 0
remove_environment_explicit/a1ddfbc0: Writing output file -  /home/spack/
runs/generic.local.bash/remove_environment_example/remove_environment_exp
licit/a1ddfbc0/remove_environment_explicit.out
remove_environment_explicit/a1ddfbc0: Writing error file - /home/spack/ru
ns/generic.local.bash/remove_environment_example/remove_environment_expli
cit/a1ddfbc0/remove_environment_explicit.err
In this iteration we are going to run the following tests: [remove_environment_explicit/a1ddfbc0, remove_environment_automatically/c5bb2a6f]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ remove_env │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 2.442686 │
│ ironment_e │            │        │ N/A        │            │          │
│ xplicit/a1 │            │        │            │            │          │
│ ddfbc0     │            │        │            │            │          │
├────────────┼────────────┼────────┼────────────┼────────────┼──────────┤
│ remove_env │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 2.409277 │
│ ironment_a │            │        │ N/A        │            │          │
│ utomatical │            │        │            │            │          │
│ ly/c5bb2a6 │            │        │            │            │          │
│ f          │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_09m1bwsv.log

If we build and look at the generated te, we notice that spack will remove environments names: remove_environment, dummy.

buildtest inspect query -t remove_environment_automatically remove_environment_explicit
$ buildtest inspect query -t remove_environment_automatically remove_environment_explicit 
─ remove_environment_automatically/c5bb2a6f-e008-4a47-9831-d4cd02b08e73 ─
Executor: generic.local.bash
Description: remove spack environment automatically before creating a new
environment
State: PASS
Returncode: 0
Runtime: 2.409277 sec
Starttime: 2022/06/29 17:26:52
Endtime: 2022/06/29 17:26:54
Command: bash --norc --noprofile -eo pipefail 
remove_environment_automatically_build.sh
Test Script: /home/spack/runs/generic.local.bash/remove_environment_examp
le/remove_environment_automatically/c5bb2a6f/remove_environment_automatic
ally.sh
Build Script: /home/spack/runs/generic.local.bash/remove_environment_exam
ple/remove_environment_automatically/c5bb2a6f/remove_environment_automati
cally_build.sh
Output File: /home/spack/runs/generic.local.bash/remove_environment_examp
le/remove_environment_automatically/c5bb2a6f/remove_environment_automatic
ally.out
Error File: /home/spack/runs/generic.local.bash/remove_environment_exampl
e/remove_environment_automatically/c5bb2a6f/remove_environment_automatica
lly.err
Log File: /home/spack/buildtest/var/logs/buildtest_09m1bwsv.log
─ Test File: /home/spack/runs/generic.local.bash/remove_environment_ex… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env rm -y remove_environment                                       
spack env create  remove_environment                                     
spack env activate  remove_environment                                   
spack add bzip2                                                          
spack concretize -f                                                      
─── remove_environment_explicit/a1ddfbc0-0016-4c2b-91cc-ef37dae94d01 ────
Executor: generic.local.bash
Description: remove spack environment explicitly using the 'rm' property
State: PASS
Returncode: 0
Runtime: 2.442686 sec
Starttime: 2022/06/29 17:26:52
Endtime: 2022/06/29 17:26:54
Command: bash --norc --noprofile -eo pipefail 
remove_environment_explicit_build.sh
Test Script: /home/spack/runs/generic.local.bash/remove_environment_examp
le/remove_environment_explicit/a1ddfbc0/remove_environment_explicit.sh
Build Script: /home/spack/runs/generic.local.bash/remove_environment_exam
ple/remove_environment_explicit/a1ddfbc0/remove_environment_explicit_buil
d.sh
Output File: /home/spack/runs/generic.local.bash/remove_environment_examp
le/remove_environment_explicit/a1ddfbc0/remove_environment_explicit.out
Error File: /home/spack/runs/generic.local.bash/remove_environment_exampl
e/remove_environment_explicit/a1ddfbc0/remove_environment_explicit.err
Log File: /home/spack/buildtest/var/logs/buildtest_09m1bwsv.log
─ Test File: /home/spack/runs/generic.local.bash/remove_environment_ex… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env rm -y dummy                                                    
spack env create  dummy                                                  
spack env activate  dummy                                                
spack add bzip2                                                          
spack concretize -f

Pre and Post Commands

The spack schema supports ability to write arbitrary shell script content using the pre_cmds and post_cmds field that are of type: string and buildtest will insert the content into the test exactly as it is defined by these two fields.

In this next example, we will test an installation of zlib by cloning spack from upstream and use pre_cmds field to specify where we will clone spack. In this example, we will clone spack under /tmp. Since we don’t have a valid root of spack since test hasn’t been run, we can ignore check for spack paths by specifying verify_spack: false which informs buildtest to skip spack path check. Generally, buildtest will raise an exception if path specified by root is invalid and if $SPACK_ROOT/share/spack/setup-env.sh doesn’t exist since this is the file that must be sourced.

The pre_cmds are shell commands that are run before sourcing spack, whereas the post_cmds are run at the very end of the script. In the post_cmds, we will spack find that will be run after spack install. We remove spack root ($SPACK_ROOT) so that this test can be rerun again.

buildspecs:
  run_pre_post_commands:
    type: spack
    executor: generic.local.bash
    description: "Install zlib"
    tags: [spack]
    pre_cmds: |
      cat /etc/os-release
      gcc --version
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib']
    post_cmds: |
      spack location -i gcc@6.5.0
      spack --version

If we build this test and inspect the generated script we should get the following result.

buildtest build -b /home/spack/buildtest/examples/spack/pre_post_cmds.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/pre_post_cmds.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:26:55                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/pre_post_cmds.yml                │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                  Discovered buildspecs                   
╔════════════════════════════════════════════════════════╗
║ buildspec                                              ║
╟────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/pre_post_cmds.yml ║
╚════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/pre_post_cmds.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ run_pre… │ generic… │ None     │ None  │ Install │ /home/s… │         │
│          │          │          │       │ zlib    │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
run_pre_post_commands/c6a63ba3: Creating test directory: /home/spack/runs
/generic.local.bash/pre_post_cmds/run_pre_post_commands/c6a63ba3
run_pre_post_commands/c6a63ba3: Creating the stage directory: /home/spack
/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c6a63ba3/sta
ge
run_pre_post_commands/c6a63ba3: Writing build script: /home/spack/runs/ge
neric.local.bash/pre_post_cmds/run_pre_post_commands/c6a63ba3/run_pre_pos
t_commands_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
run_pre_post_commands/c6a63ba3 does not have any dependencies adding test
to queue
run_pre_post_commands/c6a63ba3: Running Test via command: bash --norc 
--noprofile -eo pipefail run_pre_post_commands_build.sh
run_pre_post_commands/c6a63ba3: Test completed in 1.571065 seconds
run_pre_post_commands/c6a63ba3: Test completed with returncode: 0
run_pre_post_commands/c6a63ba3: Writing output file -  /home/spack/runs/g
eneric.local.bash/pre_post_cmds/run_pre_post_commands/c6a63ba3/run_pre_po
st_commands.out
run_pre_post_commands/c6a63ba3: Writing error file - /home/spack/runs/gen
eric.local.bash/pre_post_cmds/run_pre_post_commands/c6a63ba3/run_pre_post
_commands.err
In this iteration we are going to run the following tests: [run_pre_post_commands/c6a63ba3]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ run_pre_po │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 1.571065 │
│ st_command │            │        │ N/A        │            │          │
│ s/c6a63ba3 │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_xlxjubga.log
buildtest inspect query -o -t run_pre_post_commands
$ buildtest inspect query -o -t run_pre_post_commands 
────── run_pre_post_commands/c6a63ba3-9776-49ee-bec5-3fa905b7e474 ───────
Executor: generic.local.bash
Description: Install zlib
State: PASS
Returncode: 0
Runtime: 1.571065 sec
Starttime: 2022/06/29 17:26:55
Endtime: 2022/06/29 17:26:57
Command: bash --norc --noprofile -eo pipefail 
run_pre_post_commands_build.sh
Test Script: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_po
st_commands/c6a63ba3/run_pre_post_commands.sh
Build Script: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_p
ost_commands/c6a63ba3/run_pre_post_commands_build.sh
Output File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_po
st_commands/c6a63ba3/run_pre_post_commands.out
Error File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_pos
t_commands/c6a63ba3/run_pre_post_commands.err
Log File: /home/spack/buildtest/var/logs/buildtest_xlxjubga.log
─ Output File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_p… ─
NAME="Ubuntu"                                                            
VERSION="18.04.5 LTS (Bionic Beaver)"                                    
ID=ubuntu                                                                
ID_LIKE=debian                                                           
PRETTY_NAME="Ubuntu 18.04.5 LTS"                                         
VERSION_ID="18.04"                                                       
HOME_URL="https://www.ubuntu.com/"                                       
SUPPORT_URL="https://help.ubuntu.com/"                                   
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"                      
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/priva
VERSION_CODENAME=bionic                                                  
UBUNTU_CODENAME=bionic                                                   
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0                                  
Copyright (C) 2017 Free Software Foundation, Inc.                        
This is free software; see the source for copying conditions.  There is N
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOS
[+] /home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5
/home/spack/spack-workspace/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/g
0.16.3                                                                   
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre… ─
#!/bin/bash                                                              
                                                                         
                                                                         
######## START OF PRE COMMANDS ########                                  
cat /etc/os-release                                                      
gcc --version                                                            
                                                                         
######## END OF PRE COMMANDS   ########                                  
                                                                         
                                                                         
source /home/spack/spack/share/spack/setup-env.sh                        
spack install  zlib                                                      
                                                                         
                                                                         
######## START OF POST COMMANDS ########                                 
spack location -i gcc@6.5.0                                              
spack --version                                                          
######## END OF POST COMMANDS   ########

Configuring Spack Mirrors

We can add mirrors in the spack instance or spack environment using the mirror property which is available in the spack and env section. If the mirrror property is part of the env section, the mirror will be added to spack environment. The mirror is an object that expects a Key/Value pair where the key is the name of mirror and value is location of the spack mirror.

In this next example, we will define a mirror name e4s that points to https://cache.e4s.io as the mirror location. Internally, this translates to spack mirror add e4s https://cache.e4s.io command.

buildspecs:
  add_mirror:
    type: spack
    executor: generic.local.bash
    description: Declare spack mirror
    tags: [spack]
    spack:
      root: $HOME/spack
      mirror:
        spack_tutorial_mirror: /mirror
    post_cmds: |
      spack mirror list
      spack config blame mirrors

  add_mirror_in_spack_env:
    type: spack
    executor: generic.local.bash
    description: Declare spack mirror in spack environment
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          name: spack_mirror
        activate:
          name: spack_mirror
        mirror:
          spack_tutorial_mirror: /mirror
    post_cmds: |
      spack mirror list
      spack config blame mirrors

This test can be built by running:

buildtest build -b $BUILDTEST_ROOT/examples/spack/mirror_example.yml

If we look at the generated script for both tests, we see that mirror is added for both tests. Note that one can have mirrors defined in their spack.yaml or one of the configuration scopes defined by spack.

buildtest inspect query -o  -t add_mirror add_mirror_in_spack_env
$ buildtest inspect query -o  -t add_mirror add_mirror_in_spack_env 
───── add_mirror_in_spack_env/45b7ca8e-128c-439a-b008-6460decbdb47 ──────
Executor: generic.local.bash
Description: Declare spack mirror in spack environment
State: PASS
Returncode: 0
Runtime: 2.208137 sec
Starttime: 2022/06/29 17:26:57
Endtime: 2022/06/29 17:27:00
Command: bash --norc --noprofile -eo pipefail 
add_mirror_in_spack_env_build.sh
Test Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirro
r_in_spack_env/45b7ca8e/add_mirror_in_spack_env.sh
Build Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirr
or_in_spack_env/45b7ca8e/add_mirror_in_spack_env_build.sh
Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirro
r_in_spack_env/45b7ca8e/add_mirror_in_spack_env.out
Error File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror
_in_spack_env/45b7ca8e/add_mirror_in_spack_env.err
Log File: /home/spack/buildtest/var/logs/buildtest_syg52kp0.log
─ Output File: /home/spack/runs/generic.local.bash/mirror_example/add_… ─
==> Updating view at /home/spack/spack/var/spack/environments/spack_mirro
==> Created environment 'spack_mirror' in /home/spack/spack/var/spack/env
==> You can activate this environment with:                              
==>   spack env activate spack_mirror                                    
spack_tutorial_mirror    file:///mirror                                  
tutorial                 file:///mirror                                  
spack-public             https://mirror.spack.io                         
---                                                                  mirr
/home/spack/spack/var/spack/environments/spack_mirror/spack.yaml:10    sp
/home/spack/.spack/mirrors.yaml:3                                      tu
/home/spack/spack/etc/spack/defaults/mirrors.yaml:2                    sp
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/mirror_example/add_mi… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env create  spack_mirror                                           
spack env activate  spack_mirror                                         
spack mirror add spack_tutorial_mirror /mirror                           
                                                                         
                                                                         
######## START OF POST COMMANDS ########                                 
spack mirror list                                                        
spack config blame mirrors                                               
######## END OF POST COMMANDS   ########                                 
                                                                         
                                                                         
──────────── add_mirror/d349520a-e540-4f25-ae59-ee0b69c80082 ────────────
Executor: generic.local.bash
Description: Declare spack mirror
State: PASS
Returncode: 0
Runtime: 1.38174 sec
Starttime: 2022/06/29 17:26:57
Endtime: 2022/06/29 17:26:59
Command: bash --norc --noprofile -eo pipefail add_mirror_build.sh
Test Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirro
r/d349520a/add_mirror.sh
Build Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirr
or/d349520a/add_mirror_build.sh
Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirro
r/d349520a/add_mirror.out
Error File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror
/d349520a/add_mirror.err
Log File: /home/spack/buildtest/var/logs/buildtest_syg52kp0.log
─ Output File: /home/spack/runs/generic.local.bash/mirror_example/add_… ─
spack_tutorial_mirror    file:///mirror                                  
tutorial                 file:///mirror                                  
spack-public             https://mirror.spack.io                         
---                                                  mirrors:            
/home/spack/.spack/mirrors.yaml:2                      spack_tutorial_mir
/home/spack/.spack/mirrors.yaml:3                      tutorial: file:///
/home/spack/spack/etc/spack/defaults/mirrors.yaml:2    spack-public: http
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/mirror_example/add_mi… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack mirror add spack_tutorial_mirror /mirror                           
                                                                         
                                                                         
######## START OF POST COMMANDS ########                                 
spack mirror list                                                        
spack config blame mirrors                                               
                                                                         
######## END OF POST COMMANDS   ########

Spack Test

Note

spack test requires version 0.16.0 or higher in order to use this feature.

buildtest can run tests using spack test run that can be used for testing installed specs with tests provided by spack. In order to run tests, you need to declare the test section which is of type: object in JSON and run is a required property. The run section maps to spack test run that is responsible for running tests for a list of specs that are specified using the specs property.

Upon running the tests, we can retrieve results using spack test results which is configured using the results property. The results property expects one to specify the specs or suite or both in order to retrieve results.

The suite property is used to retrieve test results based on suite name, whereas specs property can be used to retrieve based on spec format. Both properties are a list of string types.

In example below we will test m4 package by running spack test run m4 and report the results with log file.

buildspecs:
  spack_test_m4:
    type: spack
    executor: generic.local.bash
    description: "Run spack test for m4 package and report results"
    tags: [spack]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['m4']
        results:
          suite: ['spack_test_m4']
          option: "-l"

If we look at the generated test, buildtest will install m4 followed by running the test. The spack test run –alias option is used to reference name of suitename which can be used to reference suitename when using spack test results to search for test results. buildtest will create the suite name based on name of test. If –alias is not specified, spack will generate a random text for suitename which you won’t know at time of writing test that is required by spack test results to fetch the results.

buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:27:00                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/spack_test.yml                   │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                 Discovered buildspecs                 
╔═════════════════════════════════════════════════════╗
║ buildspec                                           ║
╟─────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_test.yml ║
╚═════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_test.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ spack_t… │ generic… │ None     │ None  │ Run     │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ test    │          │         │
│          │          │          │       │ for m4  │          │         │
│          │          │          │       │ package │          │         │
│          │          │          │       │ and     │          │         │
│          │          │          │       │ report  │          │         │
│          │          │          │       │ results │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
spack_test_m4/ec74c7ba: Creating test directory: 
/home/spack/runs/generic.local.bash/spack_test/spack_test_m4/ec74c7ba
spack_test_m4/ec74c7ba: Creating the stage directory: /home/spack/runs/ge
neric.local.bash/spack_test/spack_test_m4/ec74c7ba/stage
spack_test_m4/ec74c7ba: Writing build script: /home/spack/runs/generic.lo
cal.bash/spack_test/spack_test_m4/ec74c7ba/spack_test_m4_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
spack_test_m4/ec74c7ba does not have any dependencies adding test to 
queue
spack_test_m4/ec74c7ba: Running Test via command: bash --norc --noprofile
-eo pipefail spack_test_m4_build.sh
spack_test_m4/ec74c7ba: Test completed in 1.533542 seconds
spack_test_m4/ec74c7ba: Test completed with returncode: 0
spack_test_m4/ec74c7ba: Writing output file -  /home/spack/runs/generic.l
ocal.bash/spack_test/spack_test_m4/ec74c7ba/spack_test_m4.out
spack_test_m4/ec74c7ba: Writing error file - /home/spack/runs/generic.loc
al.bash/spack_test/spack_test_m4/ec74c7ba/spack_test_m4.err
In this iteration we are going to run the following tests: [spack_test_m4/ec74c7ba]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Re ┃            ┃          ┃
┃            ┃            ┃        ┃ turnCode,  ┃            ┃          ┃
┃            ┃            ┃        ┃ Regex,     ┃            ┃          ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)   ┃ returnCode ┃ runtime  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ spack_test │ generic.l… │ PASS   │ N/A N/A    │ 0          │ 1.533542 │
│ _m4/ec74c7 │            │        │ N/A        │            │          │
│ ba         │            │        │            │            │          │
└────────────┴────────────┴────────┴────────────┴────────────┴──────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_3_bux286.log
buildtest inspect query -o -t spack_test_m4
$ buildtest inspect query -o -t spack_test_m4 
────────── spack_test_m4/ec74c7ba-419c-4496-9660-55aef5b988a9 ───────────
Executor: generic.local.bash
Description: Run spack test for m4 package and report results
State: PASS
Returncode: 0
Runtime: 1.533542 sec
Starttime: 2022/06/29 17:27:00
Endtime: 2022/06/29 17:27:02
Command: bash --norc --noprofile -eo pipefail spack_test_m4_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4
/ec74c7ba/spack_test_m4.sh
Build Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m
4/ec74c7ba/spack_test_m4_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4
/ec74c7ba/spack_test_m4.out
Error File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/
ec74c7ba/spack_test_m4.err
Log File: /home/spack/buildtest/var/logs/buildtest_3_bux286.log
─ Output File: /home/spack/runs/generic.local.bash/spack_test/spack_te… ─
==> Spack test spack_test_m4                                             
==> Testing package m4-1.4.18-mkc3u4x                                    
==> Results for test suite 'spack_test_m4':                              
==>   m4-1.4.18-mkc3u4x PASSED                                           
==> Testing package m4-1.4.18-mkc3u4x                                    
==> [2022-06-29-17:27:02.042825] test: ensuring m4 version is 1.4.18     
==> [2022-06-29-17:27:02.044072] '/home/spack/spack-workspace/software/li
m4 (GNU M4) 1.4.18                                                       
Copyright (C) 2016 Free Software Foundation, Inc.                        
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.h
This is free software: you are free to change and redistribute it.       
There is NO WARRANTY, to the extent permitted by law.                    
Written by Rene' Seindal.                                                
PASSED                                                                   
==> [2022-06-29-17:27:02.050924] test: ensuring m4 example succeeds      
==> [2022-06-29-17:27:02.051216] '/home/spack/spack-workspace/software/li
// macro is defined                                                      
Hello, World!                                                            
PASSED                                                                   
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/spack_test/spack_test… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack test run --alias spack_test_m4 m4                                  
spack test results -l spack_test_m4

We can search for test results using the spec format instead of suite name. In the results property we can use specs field instead of suite property to specify a list of spec names to run. In spack, you can retrieve the results using spack test results -- <spec>, note that double dash -- is in front of spec name. We can pass options to spack test results using the option property which is available for results and run property. Currently, spack will write test results in $HOME/.spack/tests and we can use spack test remove to clear all test results. This can be done in buildspec using the remove_tests field which is a boolean. If this is set to True buildtest will run spack test remove -y to remove all test suites before running the tests.

In this next example, we will create a spack environment to install libxml2 and libsigsegv and test the package and report log after running test.

buildspecs:
  spack_test_results_specs_format:
    type: spack
    executor: generic.local.bash
    description: "Run spack test results with spec format"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          name: spack_test_example
        activate:
          name: spack_test_example
      install:
        specs: ['libxml2', 'libsigsegv']
      test:
        remove_tests: true
        run:
          specs: ['libxml2', 'libsigsegv']
        results:
          option: '-l'
          specs: ['libxml2', 'libsigsegv']

We can build this test by running the following

buildtest build -b /home/spack/buildtest/examples/spack/spack_test_specs.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_test_specs.yml 
╭────────────────────────── buildtest summary ──────────────────────────╮
│                                                                       │
│ User:               spack                                             │
│ Hostname:           356ea13b3433                                      │
│ Platform:           Linux                                             │
│ Current Time:       2022/06/29 17:27:03                               │
│ buildtest path:     /home/spack/buildtest/bin/buildtest               │
│ buildtest version:  0.14.0                                            │
│ python path:        /home/spack/pyenv/buildtest/bin/python3           │
│ python version:     3.8.6                                             │
│ Configuration File:                                                   │
│ /home/spack/buildtest/buildtest/settings/spack_container.yml          │
│ Test Directory:     /home/spack/runs                                  │
│ Report File:        /home/spack/buildtest/var/report.json             │
│ Command:            /home/spack/buildtest/bin/buildtest build -b      │
│ /home/spack/buildtest/examples/spack/spack_test_specs.yml             │
│                                                                       │
╰───────────────────────────────────────────────────────────────────────╯
────────────────────────  Discovering Buildspecs ────────────────────────
                    Discovered buildspecs                    
╔═══════════════════════════════════════════════════════════╗
║ buildspec                                                 ║
╟───────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_test_specs.yml ║
╚═══════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────── Parsing Buildspecs ───────────────────────────
Buildtest will parse 1 buildspecs
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_test_specs.yml: VALID
Total builder objects created: 1
Total compiler builder: 0
Total script builder: 0
Total spack builder: 1
                          Spack Builder Details                          
┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ executor ┃ compiler ┃ nodes ┃ procs   ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ spack_t… │ generic… │ None     │ None  │ Run     │ /home/s… │         │
│          │          │          │       │ spack   │          │         │
│          │          │          │       │ test    │          │         │
│          │          │          │       │ results │          │         │
│          │          │          │       │ with    │          │         │
│          │          │          │       │ spec    │          │         │
│          │          │          │       │ format  │          │         │
└──────────┴──────────┴──────────┴───────┴─────────┴──────────┴─────────┘
───────────────────────────── Building Test ─────────────────────────────
spack_test_results_specs_format/ce805e1b: Creating test directory: /home/
spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_f
ormat/ce805e1b
spack_test_results_specs_format/ce805e1b: Creating the stage directory: /
home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_sp
ecs_format/ce805e1b/stage
spack_test_results_specs_format/ce805e1b: Writing build script: /home/spa
ck/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_form
at/ce805e1b/spack_test_results_specs_format_build.sh
───────────────────────────── Running Tests ─────────────────────────────
Spawning 4 processes for processing builders
────────────────────────────── Iteration 1 ──────────────────────────────
spack_test_results_specs_format/ce805e1b does not have any dependencies 
adding test to queue
spack_test_results_specs_format/ce805e1b: Running Test via command: bash 
--norc --noprofile -eo pipefail spack_test_results_specs_format_build.sh
spack_test_results_specs_format/ce805e1b: Test completed in 7.39282 
seconds
spack_test_results_specs_format/ce805e1b: Test completed with returncode:
0
spack_test_results_specs_format/ce805e1b: Writing output file -  /home/sp
ack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_for
mat/ce805e1b/spack_test_results_specs_format.out
spack_test_results_specs_format/ce805e1b: Writing error file - /home/spac
k/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_forma
t/ce805e1b/spack_test_results_specs_format.err
In this iteration we are going to run the following tests: [spack_test_results_specs_format/ce805e1b]
                              Test Summary                               
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃            ┃            ┃        ┃ checks (Ret ┃            ┃         ┃
┃            ┃            ┃        ┃ urnCode,    ┃            ┃         ┃
┃            ┃            ┃        ┃ Regex,      ┃            ┃         ┃
┃ builder    ┃ executor   ┃ status ┃ Runtime)    ┃ returnCode ┃ runtime ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_test │ generic.l… │ PASS   │ N/A N/A N/A │ 0          │ 7.39282 │
│ _results_s │            │        │             │            │         │
│ pecs_forma │            │        │             │            │         │
│ t/ce805e1b │            │        │             │            │         │
└────────────┴────────────┴────────┴─────────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to: /home/spack/buildtest/var/logs/buildtest_lujjy6x3.log

Now let’s check the generated test and output file, we see buildtest will install libxml2 and libsigsegv in spack environment followed by removing all testsuites using spack test remove -y and run the test. Note that we can query results in spec format (spack test results --l --libxml2) where spack will try to match a result file that matches the corresponding spec.

buildtest inspect query -o -t spack_test_results_specs_format
$ buildtest inspect query -o -t spack_test_results_specs_format 
─ spack_test_results_specs_format/ce805e1b-3ed5-4635-8730-209871828c6e ──
Executor: generic.local.bash
Description: Run spack test results with spec format
State: PASS
Returncode: 0
Runtime: 7.39282 sec
Starttime: 2022/06/29 17:27:03
Endtime: 2022/06/29 17:27:10
Command: bash --norc --noprofile -eo pipefail 
spack_test_results_specs_format_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_test_specs/spack_t
est_results_specs_format/ce805e1b/spack_test_results_specs_format.sh
Build Script: /home/spack/runs/generic.local.bash/spack_test_specs/spack_
test_results_specs_format/ce805e1b/spack_test_results_specs_format_build.
sh
Output File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_t
est_results_specs_format/ce805e1b/spack_test_results_specs_format.out
Error File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_te
st_results_specs_format/ce805e1b/spack_test_results_specs_format.err
Log File: /home/spack/buildtest/var/logs/buildtest_lujjy6x3.log
─ Output File: /home/spack/runs/generic.local.bash/spack_test_specs/sp… ─
==> Updating view at /home/spack/spack/var/spack/environments/spack_test_
==> Created environment 'spack_test_example' in /home/spack/spack/var/spa
==> You can activate this environment with:                              
==>   spack env activate spack_test_example                              
==> All of the packages are already installed                            
==> All of the packages are already installed                            
==> Spack test spack_test_results_specs_format                           
==> Testing package libxml2-2.9.10-yn2r3wf                               
==> Testing package libsigsegv-2.12-lbrx7ln                              
==> Results for test suite 'spack_test_results_specs_format', spec matchi
==>   libxml2-2.9.10-yn2r3wf PASSED                                      
==> Testing package libxml2-2.9.10-yn2r3wf                               
==> [2022-06-29-17:27:08.701492] test: Performing simple import test     
==> [2022-06-29-17:27:08.703887] test: xml2-config: expect command status
==> [2022-06-29-17:27:08.704384] '/home/spack/spack-workspace/software/li
2.9.10                                                                   
PASSED                                                                   
==> [2022-06-29-17:27:08.710285] test: xmllint: expect command status in 
==> [2022-06-29-17:27:08.710636] '/home/spack/spack-workspace/software/li
PASSED                                                                   
==> [2022-06-29-17:27:08.722998] test: xmllint: expect command status in 
==> [2022-06-29-17:27:08.723393] '/home/spack/spack-workspace/software/li
<?xml version="1.0"?>                                                    
<info>abc</info>                                                         
validity error : no DTD found!                                           
Document test.xml does not validate                                      
PASSED                                                                   
==> [2022-06-29-17:27:08.728919] test: xmllint: expect command status in 
==> [2022-06-29-17:27:08.729334] '/home/spack/spack-workspace/software/li
<?xml version="1.0"?>                                                    
<info>abc</info>                                                         
test.xml:2: element info: validity error : Element info content does not 
Document test.xml does not validate against /home/spack/.spack/test/d6phm
PASSED                                                                   
==> [2022-06-29-17:27:08.735692] test: xmllint: expect command status in 
==> [2022-06-29-17:27:08.736022] '/home/spack/spack-workspace/software/li
<?xml version="1.0"?>                                                    
<info>                                                                   
<data>abc</data>                                                         
</info>                                                                  
PASSED                                                                   
==> [2022-06-29-17:27:08.740868] test: xmlcatalog: expect command status 
==> [2022-06-29-17:27:08.741324] '/home/spack/spack-workspace/software/li
<?xml version="1.0"?>                                                    
<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/>           
PASSED                                                                   
==> Results for test suite 'spack_test_results_specs_format', spec matchi
==>   libsigsegv-2.12-lbrx7ln PASSED                                     
==> Testing package libsigsegv-2.12-lbrx7ln                              
==> [2022-06-29-17:27:09.187686] Multiple definitions of compiler gcc@7.5
 [gcc(/usr/bin/gcc                                                       
     /usr/bin/g++                                                        
     /usr/bin/gfortran                                                   
     /usr/bin/gfortran                                                   
     []                                                                  
     ubuntu18.04), gcc(/usr/bin/gcc                                      
     /usr/bin/g++                                                        
     /usr/bin/gfortran                                                   
     /usr/bin/gfortran                                                   
     []                                                                  
     ubuntu18.04)]                                                       
==> [2022-06-29-17:27:09.188553] test: checking ability to link to the li
==> [2022-06-29-17:27:09.188912] '/home/spack/spack/lib/spack/env/cc' '-I
PASSED                                                                   
==> [2022-06-29-17:27:09.265355] test: checking ability to use the librar
==> [2022-06-29-17:27:09.266429] './smoke_test'                          
Caught sigsegv #1                                                        
Hello World!                                                             
PASSED                                                                   
                                                                         
─ Test File: /home/spack/runs/generic.local.bash/spack_test_specs/spac… ─
#!/bin/bash                                                              
source /home/spack/spack/share/spack/setup-env.sh                        
spack env create  spack_test_example                                     
spack env activate  spack_test_example                                   
spack install  libxml2                                                   
spack install  libsigsegv                                                
spack test remove -y                                                     
spack test run --alias spack_test_results_specs_format libxml2 libsigsegv
spack test results -l -- libxml2                                         
spack test results -l -- libsigsegv

Specifying Scheduler Directives

The spack schema supports all of the scheduler scheduler directives such as sbatch, bsub, pbs, cobalt, and batch property in the buildspec.

The directives are applied at top of script. Shown below is a toy example that will define directives using sbatch property. Note, this test won’t submit job to scheduler since we are not using the a slurm executor.

buildspecs:
  spack_sbatch_example:
    type: spack
    executor: generic.local.bash
    description: "sbatch directives can be defined in spack schema"
    tags: [spack]
    sbatch: ["-N 1", "-n 8", "-t 30"]
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib@1.2.11%clang', 'zlib@1.2.8%clang']

buildtest will generate the shell script with the job directives and set the name, output and error files based on name of test. If we build this test, and inspect the generated test we see that #SBATCH directives are written based on the sbatch field.

buildtest inspect query -t spack_sbatch_example
$ buildtest inspect query -t spack_sbatch_example 
─────── spack_sbatch_example/b38cb93d-801d-4085-a22c-bfcecfbf1b7e ───────
Executor: generic.local.bash
Description: sbatch directives can be defined in spack schema
State: PASS
Returncode: 0
Runtime: 3.542731 sec
Starttime: 2022/06/29 17:27:11
Endtime: 2022/06/29 17:27:15
Command: bash --norc --noprofile -eo pipefail 
spack_sbatch_example_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatc
h_example/b38cb93d/spack_sbatch_example.sh
Build Script: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbat
ch_example/b38cb93d/spack_sbatch_example_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatc
h_example/b38cb93d/spack_sbatch_example.out
Error File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch
_example/b38cb93d/spack_sbatch_example.err
Log File: /home/spack/buildtest/var/logs/buildtest_ktoiyb7h.log
─ Test File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sb… ─
#!/bin/bash                                                              
#SBATCH -N 1                                                             
#SBATCH -n 8                                                             
#SBATCH -t 30                                                            
#SBATCH --job-name=spack_sbatch_example                                  
#SBATCH --output=spack_sbatch_example.out                                
#SBATCH --error=spack_sbatch_example.err                                 
source /home/spack/spack/share/spack/setup-env.sh                        
spack install  zlib@1.2.11%clang                                         
spack install  zlib@1.2.8%clang

You can define multiple executors in your buildspec with spack schema via executors. This can be useful if you need to specify different scheduler directives based on executor type since your executor will map to a queue.

Shown below is an example buildspec that will specify sbatch directives for generic.local.sh and generic.local.bash

buildspecs:
  spack_sbatch_multi_executors:
    type: spack
    executor: "generic.local.(sh|bash)"
    description: "sbatch directives can be defined in spack schema"
    tags: [spack]
    executors:
      generic.local.sh:
        sbatch: ["-N 1", "-t 30"]
      generic.local.bash:
        sbatch: ["-N 8", "-t 15"]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['libsigsegv']
        results:
          specs: ['libsigsegv']
          option: "-l"