Note

Please see Tutorials Setup before you proceed with this section

Buildtest Spack Integration

Note

This feature is in active development.

buildtest can use spack to build test where one can use spack to install packages followed by running any test. You must set type: spack in buildspec to use the spack schema for validating the buildspec test. Currently, we have spack.schema.json JSON schema that defines the structure of how tests are to be written in buildspec. Shown below is the schema header. The required properties are type, executor and spack.

{
  "$id": "spack.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "spack schema version",
  "description": "The spack schema is referenced using ``type: spack`` which is used for generating tests using spack package manager",
  "type": "object",
  "required": [
    "type",
    "executor",
    "spack"
  ],
  "additionalProperties": false,

Install Specs

Let’s start off with a simple example where we create a test that can spack install zlib. Shown below is a test named install_zlib. The spack keyword is a JSON object, in this test we define the root of spack using the root keyword which informs buildtest where spack is located. buildtest will automatically check the path and source the startup script. The install field is a JSON object that contains a specs property which is a list of strings types that are name of spack packages to install. Each item in the specs property will be added as a separate spack install command.

The schema is designed to mimic spack commands which will be clear with more examples.

buildspecs:
  install_specs_example:
    type: spack
    executor: generic.local.bash
    description: "Install zlib from an existing spack instance"
    tags: [spack]
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib']

Let’s build this test by running the following

buildtest build -b /home/spack/buildtest/examples/spack/install_specs.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/install_specs.yml 
Buildspec Paths: ['/home/spack/buildtest/examples']
Updating buildspec cache file: /home/spack/buildtest/var/buildspecs/cache.json
╭─────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────╮                                                                                                             
│                                                                                                                         │                                                                                                             
│ User:               spack                                                                                               │                                                                                                             
│ Hostname:           3220cd0ac8f6                                                                                        │                                                                                                             
│ Platform:           Linux                                                                                               │                                                                                                             
│ Current Time:       2024/02/08 19:00:15                                                                                 │                                                                                                             
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                 │                                                                                                             
│ buildtest version:  1.7                                                                                                 │                                                                                                             
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                             │                                                                                                             
│ python version:     3.11.6                                                                                              │                                                                                                             
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                        │                                                                                                             
│ Test Directory:     /home/spack/runs                                                                                    │                                                                                                             
│ Report File:        /home/spack/buildtest/var/report.json                                                               │                                                                                                             
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/install_specs.yml │                                                                                                             
│                                                                                                                         │                                                                                                             
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                             
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                  Discovered buildspecs                   
╔════════════════════════════════════════════════════════╗
║ buildspec                                              ║
╟────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/install_specs.yml ║
╚════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/install_specs.yml: VALID
Total builder objects created: 1
                                                                                      Builders by type=spack                                                                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                        ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                  ┃ buildspecs                                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ install_specs_example/ff766e10 │ spack │ generic.local.bash │ None     │ None  │ None  │ Install zlib from an existing spack instance │ /home/spack/buildtest/examples/spack/install_specs.yml │
└────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────────┴────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
install_specs_example/ff766e10: Creating Test Directory: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
install_specs_example/ff766e10 does not have any dependencies adding test to queue
     Builders Eligible to Run     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ install_specs_example/ff766e10 │
└────────────────────────────────┘
install_specs_example/ff766e10: Current Working Directory : /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/stage
install_specs_example/ff766e10: Running Test via command: bash install_specs_example_build.sh
install_specs_example/ff766e10: Test completed in 14.217752 seconds with returncode: 0
install_specs_example/ff766e10: Writing output file -  /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.out
install_specs_example/ff766e10: Writing error file - /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.err
                                     Test Summary                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                        ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ install_specs_example/ff766e10 │ generic.local.bash │ PASS   │ 0          │ 14.218  │
└────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_wl0yqxeh.log

Let’s inspect the generated script and output file via buildtest inspect query command. We notice that buildtest will source spack setup script and install zlib which is automatically installed from the buildcache.

buildtest inspect query -o -t install_specs_example
$ buildtest inspect query -o -t install_specs_example 
────────────────────────────────────────────────────────────────────────────────────── install_specs_example/ff766e10-9de7-4dfb-a05b-a5ed70a58bce ──────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Install zlib from an existing spack instance
State: PASS
Returncode: 0
Runtime: 14.217752 sec
Starttime: 2024/02/08 19:00:15
Endtime: 2024/02/08 19:00:29
Command: bash install_specs_example_build.sh
Test Script: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.sh
Build Script: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example_build.sh
Output File: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.out
Error File: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.err
Log File: /home/spack/buildtest/var/logs/buildtest_wl0yqxeh.log
─────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.out ────────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
[+] /home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/gmake-4.4.1-6qiak7nnzdxg4dik4mcwrwvy24bsovui                                                                                                                     
==> Installing zlib-1.3-p6c6zbrub6hv5imwnruz4uty2u7y2nfq [2/2]                                                                                                                                                                          
==> No binary for zlib-1.3-p6c6zbrub6hv5imwnruz4uty2u7y2nfq found: installing from source                                                                                                                                               
==> Fetching https://mirror.spack.io/_source-cache/archive/ff/ff0ba4c292013dbc27530b3a81e1f9a813cd39de01ca5e0f8bf355702efa593e.tar.gz                                                                                                   
==> No patches needed for zlib                                                                                                                                                                                                          
==> zlib: Executing phase: 'edit'                                                                                                                                                                                                       
==> zlib: Executing phase: 'build'                                                                                                                                                                                                      
==> zlib: Executing phase: 'install'                                                                                                                                                                                                    
==> zlib: Successfully installed zlib-1.3-p6c6zbrub6hv5imwnruz4uty2u7y2nfq                                                                                                                                                              
  Stage: 0.45s.  Edit: 0.47s.  Build: 1.95s.  Install: 0.50s.  Post-install: 0.03s.  Total: 3.44s                                                                                                                                       
[+] /home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/zlib-1.3-p6c6zbrub6hv5imwnruz4uty2u7y2nfq                                                                                                                        
                                                                                                                                                                                                                                        
───────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/install_specs/install_specs_example/ff766e10/install_specs_example.sh ─────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack install  zlib

Spack Environment

buildtest can generate scripts to make use of spack environments which can be useful if you want to install or test specs in an isolated environment.

Currently, we can create spack environment (spack env create) via name, directory and manifest file (spack.yaml, spack.lock) and pass any options to spack env create command. Furthermore, we can activate existing spack environment via name or directory using spack env activate and pass options to the command. buildtest can remove spack environments automatically before creating spack environment or one can explicitly specify by name.

Create a Spack Environment by name

In this next example, we will create a spack environment named m4_zlib that will install m4 and zlib spec. The create field is a JSON object that maps to spack env create command which can pass some arguments in the form of key/value pairs. The name property in create section is used to create a spack environment by name. The activate property maps to spack env activate command which is used to activate a spack environment. The name property is of type: string which is name of spack environment you want to activate.

The compiler_find: true is a boolean that determines if we need to find compilers in spack via spack compiler find. This can be useful if you need to find compilers so spack can install specs with a preferred compiler otherwise spack may have issues concretizing or install specs. buildtest will run spack compiler find after sourcing spack.

Note

The compiler_find option may not be useful if your compilers are already defined in one of your configuration scopes or spack.yaml that is part of your spack environment.

The option field can pass any command line arguments to spack install command and this field is available for other properties.

buildspecs:
  install_in_spack_env:
    type: spack
    executor: generic.local.bash
    description: "Install m4 and zlib in a spack environment named m4_zlib"
    tags: [spack]
    spack:
      root: $HOME/spack
      compiler_find: true
      env:
        create:
          name: 'm4_zlib'
        specs:
          - 'm4'
          - 'zlib'
        activate:
          name: m4_zlib
        concretize: true
      install:
        option: '--keep-prefix'
    post_cmds: |
      spack env deactivate
      spack env remove -y m4_zlib

If we build this test and see generated test we see that buildtest will create a spack environment m4_zlib and activate the environment, add m4 and zlib, concretize the environment and install the specs.

buildtest build -b /home/spack/buildtest/examples/spack/env_install.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_install.yml 
╭────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────╮                                                                                                               
│                                                                                                                       │                                                                                                               
│ User:               spack                                                                                             │                                                                                                               
│ Hostname:           3220cd0ac8f6                                                                                      │                                                                                                               
│ Platform:           Linux                                                                                             │                                                                                                               
│ Current Time:       2024/02/08 19:00:30                                                                               │                                                                                                               
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                               │                                                                                                               
│ buildtest version:  1.7                                                                                               │                                                                                                               
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                           │                                                                                                               
│ python version:     3.11.6                                                                                            │                                                                                                               
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                      │                                                                                                               
│ Test Directory:     /home/spack/runs                                                                                  │                                                                                                               
│ Report File:        /home/spack/buildtest/var/report.json                                                             │                                                                                                               
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/env_install.yml │                                                                                                               
│                                                                                                                       │                                                                                                               
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                               
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                 Discovered buildspecs                  
╔══════════════════════════════════════════════════════╗
║ buildspec                                            ║
╟──────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_install.yml ║
╚══════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_install.yml: VALID
Total builder objects created: 1
                                                                                          Builders by type=spack                                                                                           
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                       ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                              ┃ buildspecs                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ install_in_spack_env/5f9e9757 │ spack │ generic.local.bash │ None     │ None  │ None  │ Install m4 and zlib in a spack environment named m4_zlib │ /home/spack/buildtest/examples/spack/env_install.yml │
└───────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
install_in_spack_env/5f9e9757: Creating Test Directory: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
install_in_spack_env/5f9e9757 does not have any dependencies adding test to queue
    Builders Eligible to Run     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ install_in_spack_env/5f9e9757 │
└───────────────────────────────┘
install_in_spack_env/5f9e9757: Current Working Directory : /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/stage
install_in_spack_env/5f9e9757: Running Test via command: bash install_in_spack_env_build.sh
install_in_spack_env/5f9e9757: Test completed in 18.50853 seconds with returncode: 0
install_in_spack_env/5f9e9757: Writing output file -  /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.out
install_in_spack_env/5f9e9757: Writing error file - /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.err
                                     Test Summary                                     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                       ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ install_in_spack_env/5f9e9757 │ generic.local.bash │ PASS   │ 0          │ 18.509  │
└───────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_m7z3j_4j.log
buildtest inspect query -t install_in_spack_env
$ buildtest inspect query -t install_in_spack_env 
────────────────────────────────────────────────────────────────────────────────────── install_in_spack_env/5f9e9757-7b72-4e7c-a0f9-42ee0f3a7bf7 ───────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Install m4 and zlib in a spack environment named m4_zlib
State: PASS
Returncode: 0
Runtime: 18.50853 sec
Starttime: 2024/02/08 19:00:30
Endtime: 2024/02/08 19:00:49
Command: bash install_in_spack_env_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.sh
Build Script: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env_build.sh
Output File: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.out
Error File: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.err
Log File: /home/spack/buildtest/var/logs/buildtest_m7z3j_4j.log
─────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/env_install/install_in_spack_env/5f9e9757/install_in_spack_env.sh ───────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack compiler find                                                                                                                                                                                                                     
spack env create  m4_zlib                                                                                                                                                                                                               
spack env activate  m4_zlib                                                                                                                                                                                                             
spack add m4                                                                                                                                                                                                                            
spack add zlib                                                                                                                                                                                                                          
spack concretize -f                                                                                                                                                                                                                     
spack install --keep-prefix                                                                                                                                                                                                             
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack env deactivate                                                                                                                                                                                                                    
spack env remove -y m4_zlib                                                                                                                                                                                                             
######## END OF POST COMMANDS   ########

Creating Spack Environment in Directory

We can create spack environment from a directory using the dir property that is available as part of create and activate field. In this next example we create a spack environment in our $HOME directory and concretize m4 in the spack environment

buildspecs:
  spack_env_directory:
    type: spack
    executor: generic.local.bash
    description: "create spack environment in directory"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          dir: $HOME/spack-envs/m4
        activate:
          dir: $HOME/spack-envs/m4
        specs:
          - 'm4'
      install:
        option: ''
    post_cmds: |
      spack find
      rm -rf $HOME/spack-envs/m4

When creating spack environment using directory, buildtest will automatically add the -d option which is required when creating spack environments. However, one can also pass this using the option field. Shown below is the build and generated script after running test.

buildtest build -b /home/spack/buildtest/examples/spack/env_create_directory.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_create_directory.yml 
╭────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────╮                                                                                                      
│                                                                                                                                │                                                                                                      
│ User:               spack                                                                                                      │                                                                                                      
│ Hostname:           3220cd0ac8f6                                                                                               │                                                                                                      
│ Platform:           Linux                                                                                                      │                                                                                                      
│ Current Time:       2024/02/08 19:00:49                                                                                        │                                                                                                      
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                        │                                                                                                      
│ buildtest version:  1.7                                                                                                        │                                                                                                      
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                                    │                                                                                                      
│ python version:     3.11.6                                                                                                     │                                                                                                      
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                               │                                                                                                      
│ Test Directory:     /home/spack/runs                                                                                           │                                                                                                      
│ Report File:        /home/spack/buildtest/var/report.json                                                                      │                                                                                                      
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/env_create_directory.yml │                                                                                                      
│                                                                                                                                │                                                                                                      
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                      
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                      Discovered buildspecs                      
╔═══════════════════════════════════════════════════════════════╗
║ buildspec                                                     ║
╟───────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_create_directory.yml ║
╚═══════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_create_directory.yml: VALID
Total builder objects created: 1
                                                                                     Builders by type=spack                                                                                     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                      ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                           ┃ buildspecs                                                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_directory/07168ffb │ spack │ generic.local.bash │ None     │ None  │ None  │ create spack environment in directory │ /home/spack/buildtest/examples/spack/env_create_directory.yml │
└──────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴───────────────────────────────────────┴───────────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_directory/07168ffb: Creating Test Directory: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_directory/07168ffb does not have any dependencies adding test to queue
    Builders Eligible to Run    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_directory/07168ffb │
└──────────────────────────────┘
spack_env_directory/07168ffb: Current Working Directory : /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/stage
spack_env_directory/07168ffb: Running Test via command: bash spack_env_directory_build.sh
spack_env_directory/07168ffb: Test completed in 15.086919 seconds with returncode: 0
spack_env_directory/07168ffb: Writing output file -  /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.out
spack_env_directory/07168ffb: Writing error file - /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.err
                                    Test Summary                                     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                      ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_env_directory/07168ffb │ generic.local.bash │ PASS   │ 0          │ 15.087  │
└──────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_ph65mwi6.log
buildtest inspect query -o -t spack_env_directory
$ buildtest inspect query -o -t spack_env_directory 
─────────────────────────────────────────────────────────────────────────────────────── spack_env_directory/07168ffb-776f-440f-bf86-c1778408fc5d ───────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: create spack environment in directory
State: PASS
Returncode: 0
Runtime: 15.086919 sec
Starttime: 2024/02/08 19:00:49
Endtime: 2024/02/08 19:01:05
Command: bash spack_env_directory_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.sh
Build Script: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory_build.sh
Output File: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.out
Error File: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.err
Log File: /home/spack/buildtest/var/logs/buildtest_ph65mwi6.log
────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.out ──────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment in /home/spack/spack-envs/m4                                                                                                                                                                                    
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate /home/spack/spack-envs/m4                                                                                                                                                                                      
==> Adding m4 to environment /home/spack/spack-envs/m4                                                                                                                                                                                  
==> Concretized m4                                                                                                                                                                                                                      
[+]  oebiztm  m4@1.4.19%gcc@12.3.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
[+]  sbfcnap      ^diffutils@3.9%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                     
[+]  a34xpad          ^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
[+]  6qiak7n      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
[+]  46vv5f3      ^libsigsegv@2.14%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
==> All of the packages are already installed                                                                                                                                                                                           
==> Updating view at /home/spack/spack-envs/m4/.spack-env/view                                                                                                                                                                          
==> In environment /home/spack/spack-envs/m4                                                                                                                                                                                            
==> Root specs                                                                                                                                                                                                                          
m4                                                                                                                                                                                                                                      
==> Installed packages                                                                                                                                                                                                                  
-- linux-ubuntu22.04-x86_64_v3 / gcc@12.3.0 ---------------------                                                                                                                                                                       
diffutils@3.9                                                                                                                                                                                                                           
gmake@4.4.1                                                                                                                                                                                                                             
libiconv@1.17                                                                                                                                                                                                                           
libsigsegv@2.14                                                                                                                                                                                                                         
m4@1.4.19                                                                                                                                                                                                                               
                                                                                                                                                                                                                                        
─────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/env_create_directory/spack_env_directory/07168ffb/spack_env_directory.sh ────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env create  -d /home/spack/spack-envs/m4                                                                                                                                                                                          
spack env activate  -d /home/spack/spack-envs/m4                                                                                                                                                                                        
spack add m4                                                                                                                                                                                                                            
spack install                                                                                                                                                                                                                           
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack find                                                                                                                                                                                                                              
rm -rf $HOME/spack-envs/m4                                                                                                                                                                                                              
######## END OF POST COMMANDS   ########

Create Spack Environment from Manifest File (spack.yaml, spack.lock)

Spack can create environments from spack.yaml or spack.lock which can be used if you have a spack configuration that works for your system and want to write a buildspec. While creating a spack environment, you can use the manifest property to specify path to your spack.yaml or spack.lock.

Note

buildtest will not enforce that manifest names be spack.yaml or spack.lock since spack allows one to create spack environment from arbitrary name so long as it is a valid spack configuration.

Shown below is an example buildspec that generates a test from a manifest file. The manifest property is of type: string and this is only available as part of create property.

buildspecs:
  spack_env_create_from_manifest:
    type: spack
    executor: generic.local.bash
    description: "Create spack environment from spack.yaml"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          name: 'manifest_example'
          manifest: "$BUILDTEST_ROOT/examples/spack/example/spack.yaml"
        activate:
          name: 'manifest_example'
        concretize: true
    post_cmds: |
      spack env deactivate
      spack env remove -y manifest_example

If we build this test and inspect the generated script we see spack env create command will create an environment manifest_example using the manifest file that we provided from the spack.yaml.

buildtest build -b /home/spack/buildtest/examples/spack/env_create_manifest.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/env_create_manifest.yml 
╭────────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────╮                                                                                                       
│                                                                                                                               │                                                                                                       
│ User:               spack                                                                                                     │                                                                                                       
│ Hostname:           3220cd0ac8f6                                                                                              │                                                                                                       
│ Platform:           Linux                                                                                                     │                                                                                                       
│ Current Time:       2024/02/08 19:01:05                                                                                       │                                                                                                       
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                       │                                                                                                       
│ buildtest version:  1.7                                                                                                       │                                                                                                       
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                                   │                                                                                                       
│ python version:     3.11.6                                                                                                    │                                                                                                       
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                              │                                                                                                       
│ Test Directory:     /home/spack/runs                                                                                          │                                                                                                       
│ Report File:        /home/spack/buildtest/var/report.json                                                                     │                                                                                                       
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/env_create_manifest.yml │                                                                                                       
│                                                                                                                               │                                                                                                       
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                       
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                     Discovered buildspecs                      
╔══════════════════════════════════════════════════════════════╗
║ buildspec                                                    ║
╟──────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/env_create_manifest.yml ║
╚══════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/env_create_manifest.yml: VALID
Total builder objects created: 1
                                                                                           Builders by type=spack                                                                                            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                                 ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                              ┃ buildspecs                                                   ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_create_from_manifest/86840c2d │ spack │ generic.local.bash │ None     │ None  │ None  │ Create spack environment from spack.yaml │ /home/spack/buildtest/examples/spack/env_create_manifest.yml │
└─────────────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────┴──────────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_create_from_manifest/86840c2d: Creating Test Directory: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_create_from_manifest/86840c2d does not have any dependencies adding test to queue
         Builders Eligible to Run          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                                 ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_create_from_manifest/86840c2d │
└─────────────────────────────────────────┘
spack_env_create_from_manifest/86840c2d: Current Working Directory : /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/stage
spack_env_create_from_manifest/86840c2d: Running Test via command: bash spack_env_create_from_manifest_build.sh
spack_env_create_from_manifest/86840c2d: Test completed in 16.129629 seconds with returncode: 0
spack_env_create_from_manifest/86840c2d: Writing output file -  /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.out
spack_env_create_from_manifest/86840c2d: Writing error file - /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.err
                                          Test Summary                                          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                                 ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_env_create_from_manifest/86840c2d │ generic.local.bash │ PASS   │ 0          │ 16.130  │
└─────────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_wwt5j7y0.log
buildtest inspect query -o -t spack_env_create_from_manifest
$ buildtest inspect query -o -t spack_env_create_from_manifest 
───────────────────────────────────────────────────────────────────────────────── spack_env_create_from_manifest/86840c2d-e556-46bc-a584-8efb9a7e0eb7 ──────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Create spack environment from spack.yaml
State: PASS
Returncode: 0
Runtime: 16.129629 sec
Starttime: 2024/02/08 19:01:05
Endtime: 2024/02/08 19:01:21
Command: bash spack_env_create_from_manifest_build.sh
Test Script: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.sh
Build Script: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest_build.sh
Output File: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.out
Error File: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.err
Log File: /home/spack/buildtest/var/logs/buildtest_wwt5j7y0.log
─────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.out ────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment 'manifest_example' in /home/spack/spack/var/spack/environments/manifest_example                                                                                                                                 
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate manifest_example                                                                                                                                                                                               
==> Concretized m4                                                                                                                                                                                                                      
[+]  oebiztm  m4@1.4.19%gcc@12.3.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
[+]  sbfcnap      ^diffutils@3.9%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                     
[+]  a34xpad          ^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
[+]  6qiak7n      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
[+]  46vv5f3      ^libsigsegv@2.14%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
==> Concretized zlib                                                                                                                                                                                                                    
[+]  p6c6zbr  zlib@1.3%gcc@12.3.0+optimize+pic+shared build_system=makefile arch=linux-ubuntu22.04-x86_64_v3                                                                                                                            
[+]  6qiak7n      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
==> Concretized python                                                                                                                                                                                                                  
 -   2sxdejj  python@3.11.6%gcc@12.3.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=l
[+]  bbwtdnr      ^bzip2@1.0.8%gcc@12.3.0~debug~pic+shared build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                        
[+]  sbfcnap          ^diffutils@3.9%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                 
[+]  grrt7ig      ^expat@2.5.0%gcc@12.3.0+libbsd build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                
[+]  6qiak7n          ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                               
[+]  sygavnw          ^libbsd@0.11.7%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                 
[+]  ivvykht              ^libmd@1.0.4%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                               
[+]  w66nich      ^gdbm@1.23%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                         
 -   emdy63i      ^gettext@0.22.3%gcc@12.3.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                            
[+]  a34xpad          ^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
 -   fze4ejw          ^libxml2@2.10.3%gcc@12.3.0+pic~python+shared build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
 -   igd3u36          ^tar@1.34%gcc@12.3.0 build_system=autotools zip=pigz arch=linux-ubuntu22.04-x86_64_v3                                                                                                                             
 -   4nf2ood              ^pigz@2.7%gcc@12.3.0 build_system=makefile arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
[+]  ngvd73i              ^zstd@1.5.5%gcc@12.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                    
[+]  ecpriyn      ^libffi@3.4.4%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                      
[+]  guszc5m      ^libxcrypt@4.4.35%gcc@12.3.0~obsolete_api build_system=autotools patches=4885da3 arch=linux-ubuntu22.04-x86_64_v3                                                                                                     
[+]  hh6v2va          ^perl@5.38.0%gcc@12.3.0+cpanm+opcode+open+shared+threads build_system=generic patches=714e4d1 arch=linux-ubuntu22.04-x86_64_v3                                                                                    
[+]  krlqpve              ^berkeley-db@18.1.40%gcc@12.3.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu22.04-x86_64_v3                                                                                  
[+]  draqwfy              ^zlib-ng@2.1.4%gcc@12.3.0+compat+opt build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                  
[+]  glwymee      ^ncurses@6.4%gcc@12.3.0~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                             
 -   aawqoux      ^openssl@3.1.3%gcc@12.3.0~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu22.04-x86_64_v3                                                                                                             
[+]  l7zpjxt          ^ca-certificates-mozilla@2023-05-30%gcc@12.3.0 build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                              
 -   ku3fwtt          ^perl@5.38.0%gcc@12.3.0+cpanm+opcode+open+shared+threads build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                    
[+]  ccpwoda      ^pkgconf@1.9.5%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                     
[+]  cz4lfdu      ^readline@8.2%gcc@12.3.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu22.04-x86_64_v3                                                                                                                      
 -   k3r6rcq      ^sqlite@3.43.2%gcc@12.3.0+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                              
[+]  oagevhm      ^util-linux-uuid@2.38.1%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                            
[+]  taa3gkk      ^xz@5.4.1%gcc@12.3.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                                   
[+]  p6c6zbr      ^zlib@1.3%gcc@12.3.0+optimize+pic+shared build_system=makefile arch=linux-ubuntu22.04-x86_64_v3                                                                                                                       
==> Successfully removed environment 'manifest_example'                                                                                                                                                                                 
                                                                                                                                                                                                                                        
───────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/env_create_manifest/spack_env_create_from_manifest/86840c2d/spack_env_create_from_manifest.sh ─────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env create  manifest_example /home/spack/buildtest/examples/spack/example/spack.yaml                                                                                                                                              
spack env activate  manifest_example                                                                                                                                                                                                    
spack concretize -f                                                                                                                                                                                                                     
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack env deactivate                                                                                                                                                                                                                    
spack env remove -y manifest_example                                                                                                                                                                                                    
######## END OF POST COMMANDS   ########

Deactivate Spack Environment

When you are switching between spack environments, it can be important to deactivate your current spack environment which can be used when writing test. This can be achieved using the keyword deactivate which is a boolean type.

If deactivate: true is set, then we will deactivate the spack environment by running spack env deactivate

Shown below is an example buildspec where we create a spack environment, deactivate first and then activate the environment. Buildtest will deactivate a spack environment prior to activating an environment which is specified via activate keyword.

buildspecs:
  spack_env_deactivate_first:
    type: spack
    executor: generic.local.bash
    description: "deactivate a spack environment first prior to activating it"
    tags: [spack]
    pre_cmds: |
      rm -rf $HOME/spack-envs/m4
    spack:
      root: $HOME/spack
      env:
        create:
          dir: $HOME/spack-envs/m4
        deactivate: true
        activate:
          dir: $HOME/spack-envs/m4
        specs:
          - 'm4'
      install:
        option: ''
    post_cmds: |
      spack find
      rm -rf $HOME/spack-envs/m4

Let’s build this by running the following

buildtest build -b /home/spack/buildtest/examples/spack/spack_env_deactivate.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_env_deactivate.yml 
╭────────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────────╮                                                                                                      
│                                                                                                                                │                                                                                                      
│ User:               spack                                                                                                      │                                                                                                      
│ Hostname:           3220cd0ac8f6                                                                                               │                                                                                                      
│ Platform:           Linux                                                                                                      │                                                                                                      
│ Current Time:       2024/02/08 19:01:41                                                                                        │                                                                                                      
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                        │                                                                                                      
│ buildtest version:  1.7                                                                                                        │                                                                                                      
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                                    │                                                                                                      
│ python version:     3.11.6                                                                                                     │                                                                                                      
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                               │                                                                                                      
│ Test Directory:     /home/spack/runs                                                                                           │                                                                                                      
│ Report File:        /home/spack/buildtest/var/report.json                                                                      │                                                                                                      
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_env_deactivate.yml │                                                                                                      
│                                                                                                                                │                                                                                                      
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                      
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                      Discovered buildspecs                      
╔═══════════════════════════════════════════════════════════════╗
║ buildspec                                                     ║
╟───────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_env_deactivate.yml ║
╚═══════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_env_deactivate.yml: VALID
Total builder objects created: 1
                                                                                                   Builders by type=spack                                                                                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                             ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                                 ┃ buildspecs                                                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_deactivate_first/8be22b89 │ spack │ generic.local.bash │ None     │ None  │ None  │ deactivate a spack environment first prior to activating it │ /home/spack/buildtest/examples/spack/spack_env_deactivate.yml │
└─────────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_deactivate_first/8be22b89: Creating Test Directory: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_env_deactivate_first/8be22b89 does not have any dependencies adding test to queue
       Builders Eligible to Run        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_env_deactivate_first/8be22b89 │
└─────────────────────────────────────┘
spack_env_deactivate_first/8be22b89: Current Working Directory : /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/stage
spack_env_deactivate_first/8be22b89: Running Test via command: bash spack_env_deactivate_first_build.sh
spack_env_deactivate_first/8be22b89: Test completed in 16.525284 seconds with returncode: 0
spack_env_deactivate_first/8be22b89: Writing output file -  /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.out
spack_env_deactivate_first/8be22b89: Writing error file - /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.err
                                        Test Summary                                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                             ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_env_deactivate_first/8be22b89 │ generic.local.bash │ PASS   │ 0          │ 16.525  │
└─────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_sjwfw_5j.log

Let’s take a look at the generated test, take note of the spack env deactivate command and order of commands. We run spack env create, followed by spack env deactivate and finally we activate the environment using spack env activate.

buildtest inspect query -t spack_env_deactivate_first
$ buildtest inspect query -t spack_env_deactivate_first 
─────────────────────────────────────────────────────────────────────────────────── spack_env_deactivate_first/8be22b89-4db1-438d-aac2-7138796bdbb3 ────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: deactivate a spack environment first prior to activating it
State: PASS
Returncode: 0
Runtime: 16.525284 sec
Starttime: 2024/02/08 19:01:41
Endtime: 2024/02/08 19:01:58
Command: bash spack_env_deactivate_first_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.sh
Build Script: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.out
Error File: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.err
Log File: /home/spack/buildtest/var/logs/buildtest_sjwfw_5j.log
──────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_env_deactivate/spack_env_deactivate_first/8be22b89/spack_env_deactivate_first.sh ─────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF PRE COMMANDS ########                                                                                                                                                                                                 
rm -rf $HOME/spack-envs/m4                                                                                                                                                                                                              
                                                                                                                                                                                                                                        
######## END OF PRE COMMANDS   ########                                                                                                                                                                                                 
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env create  -d /home/spack/spack-envs/m4                                                                                                                                                                                          
spack env deactivate || true                                                                                                                                                                                                            
spack env activate  -d /home/spack/spack-envs/m4                                                                                                                                                                                        
spack add m4                                                                                                                                                                                                                            
spack install                                                                                                                                                                                                                           
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack find                                                                                                                                                                                                                              
rm -rf $HOME/spack-envs/m4                                                                                                                                                                                                              
                                                                                                                                                                                                                                        
######## END OF POST COMMANDS   ########

Removing Spack Environments

buildtest can remove spack environments which can be used if you are periodically running the same test where one is creating the same environment. buildtest can automatically remove spack environment using the property remove_environment which will remove the environment before creating it with same name. This field is part of the create field and only works if one is creating spack environments by name.

Alternately, buildtest provides the rm field which can be used for removing environment explicitly. In the rm field, the name is a required field which is the name of the spack environment to remove. The name field is of type: string Shown below are two example tests where we remove spack environment using the remove_environment and rm field.

buildspecs:
  remove_environment_automatically:
    type: spack
    executor: generic.local.bash
    description: "remove spack environment automatically before creating a new environment"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          remove_environment: true
          name: remove_environment
        activate:
          name: remove_environment
        specs:
          - 'bzip2'
        concretize: true

  remove_environment_explicit:
    type: spack
    executor: generic.local.bash
    description: "remove spack environment explicitly using the 'rm' property"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        rm:
          name: dummy
        create:
          name: dummy
        activate:
          name: dummy
        specs:
          - 'bzip2'
        concretize: true

Let’s build this by running the following

buildtest build -b /home/spack/buildtest/examples/spack/remove_environment_example.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/remove_environment_example.yml 
╭───────────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────────────╮                                                                                                
│                                                                                                                                      │                                                                                                
│ User:               spack                                                                                                            │                                                                                                
│ Hostname:           3220cd0ac8f6                                                                                                     │                                                                                                
│ Platform:           Linux                                                                                                            │                                                                                                
│ Current Time:       2024/02/08 19:01:22                                                                                              │                                                                                                
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                              │                                                                                                
│ buildtest version:  1.7                                                                                                              │                                                                                                
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                                          │                                                                                                
│ python version:     3.11.6                                                                                                           │                                                                                                
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                                     │                                                                                                
│ Test Directory:     /home/spack/runs                                                                                                 │                                                                                                
│ Report File:        /home/spack/buildtest/var/report.json                                                                            │                                                                                                
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/remove_environment_example.yml │                                                                                                
│                                                                                                                                      │                                                                                                
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                         Discovered buildspecs                         
╔═════════════════════════════════════════════════════════════════════╗
║ buildspec                                                           ║
╟─────────────────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/remove_environment_example.yml ║
╚═════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/remove_environment_example.yml: VALID
Total builder objects created: 2
                                                                                                         Builders by type=spack                                                                                                         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                                   ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                                     ┃ buildspecs                                                     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ remove_environment_automatically/5c68f81e │ spack │ generic.local.bash │ None     │ None  │ None  │ remove spack environment automatically before creating a new    │ /home/spack/buildtest/examples/spack/remove_environment_exampl │
│                                           │       │                    │          │       │       │ environment                                                     │ e.yml                                                          │
├───────────────────────────────────────────┼───────┼────────────────────┼──────────┼───────┼───────┼─────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ remove_environment_explicit/05cea701      │ spack │ generic.local.bash │ None     │ None  │ None  │ remove spack environment explicitly using the 'rm' property     │ /home/spack/buildtest/examples/spack/remove_environment_exampl │
│                                           │       │                    │          │       │       │                                                                 │ e.yml                                                          │
└───────────────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
remove_environment_automatically/5c68f81e: Creating Test Directory: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e
remove_environment_explicit/05cea701: Creating Test Directory: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
remove_environment_explicit/05cea701 does not have any dependencies adding test to queue
remove_environment_automatically/5c68f81e does not have any dependencies adding test to queue
          Builders Eligible to Run           
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                                   ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ remove_environment_explicit/05cea701      │
│ remove_environment_automatically/5c68f81e │
└───────────────────────────────────────────┘
remove_environment_explicit/05cea701: Current Working Directory : /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/stage
remove_environment_automatically/5c68f81e: Current Working Directory : /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/stage
remove_environment_explicit/05cea701: Running Test via command: bash remove_environment_explicit_build.sh
remove_environment_automatically/5c68f81e: Running Test via command: bash remove_environment_automatically_build.sh
remove_environment_explicit/05cea701: Test completed in 15.970731 seconds with returncode: 0
remove_environment_explicit/05cea701: Writing output file -  /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.out
remove_environment_explicit/05cea701: Writing error file - /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.err
remove_environment_automatically/5c68f81e: Test completed in 18.007413 seconds with returncode: 0
remove_environment_automatically/5c68f81e: Writing output file -  /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.out
remove_environment_automatically/5c68f81e: Writing error file - /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.err
                                           Test Summary                                           
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                                   ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ remove_environment_explicit/05cea701      │ generic.local.bash │ PASS   │ 0          │ 15.971  │
├───────────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ remove_environment_automatically/5c68f81e │ generic.local.bash │ PASS   │ 0          │ 18.007  │
└───────────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_mwyd1m9t.log

If we build and look at the generated te, we notice that spack will remove environments names: remove_environment, dummy.

buildtest inspect query -t remove_environment_automatically remove_environment_explicit
$ buildtest inspect query -t remove_environment_automatically remove_environment_explicit 
─────────────────────────────────────────────────────────────────────────────────── remove_environment_explicit/05cea701-6b45-4fc2-8a3e-4247cc2c5671 ───────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: remove spack environment explicitly using the 'rm' property
State: PASS
Returncode: 0
Runtime: 15.970731 sec
Starttime: 2024/02/08 19:01:22
Endtime: 2024/02/08 19:01:38
Command: bash remove_environment_explicit_build.sh
Test Script: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.sh
Build Script: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit_build.sh
Output File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.out
Error File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.err
Log File: /home/spack/buildtest/var/logs/buildtest_mwyd1m9t.log
──────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_explicit/05cea701/remove_environment_explicit.sh ─────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env rm -y dummy || true                                                                                                                                                                                                           
spack env create  dummy                                                                                                                                                                                                                 
spack env activate  dummy                                                                                                                                                                                                               
spack add bzip2                                                                                                                                                                                                                         
spack concretize -f                                                                                                                                                                                                                     
──────────────────────────────────────────────────────────────────────────────── remove_environment_automatically/5c68f81e-adb0-4cae-9555-2af1a368ba77 ─────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: remove spack environment automatically before creating a new environment
State: PASS
Returncode: 0
Runtime: 18.007413 sec
Starttime: 2024/02/08 19:01:22
Endtime: 2024/02/08 19:01:40
Command: bash remove_environment_automatically_build.sh
Test Script: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.sh
Build Script: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically_build.sh
Output File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.out
Error File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.err
Log File: /home/spack/buildtest/var/logs/buildtest_mwyd1m9t.log
─────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/remove_environment_example/remove_environment_automatically/5c68f81e/remove_environment_automatically.sh ────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env rm -y remove_environment || true                                                                                                                                                                                              
spack env create  remove_environment                                                                                                                                                                                                    
spack env activate  remove_environment                                                                                                                                                                                                  
spack add bzip2                                                                                                                                                                                                                         
spack concretize -f

Pre and Post Commands

The spack schema supports ability to write arbitrary shell script content using the pre_cmds and post_cmds field that are of type: string and buildtest will insert the content into the test exactly as it is defined by these two fields.

In this next example, we will test an installation of zlib by cloning spack from upstream and use pre_cmds field to specify where we will clone spack.

The pre_cmds are shell commands that are run before sourcing spack, whereas the post_cmds are run at the very end of the script. In the post_cmds, we will spack find that will be run after spack install. We remove spack root ($SPACK_ROOT) so that this test can be rerun again.

buildspecs:
  run_pre_post_commands:
    type: spack
    executor: generic.local.bash
    description: "Install zlib"
    tags: [spack]
    pre_cmds: |
      cat /etc/os-release
      gcc --version
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib']
    post_cmds: |
      spack location -i gcc@12.3.0
      spack --version

If we build this test and inspect the generated script we should get the following result.

buildtest build -b /home/spack/buildtest/examples/spack/pre_post_cmds.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/pre_post_cmds.yml 
╭─────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────╮                                                                                                             
│                                                                                                                         │                                                                                                             
│ User:               spack                                                                                               │                                                                                                             
│ Hostname:           3220cd0ac8f6                                                                                        │                                                                                                             
│ Platform:           Linux                                                                                               │                                                                                                             
│ Current Time:       2024/02/08 19:01:58                                                                                 │                                                                                                             
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                 │                                                                                                             
│ buildtest version:  1.7                                                                                                 │                                                                                                             
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                             │                                                                                                             
│ python version:     3.11.6                                                                                              │                                                                                                             
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                        │                                                                                                             
│ Test Directory:     /home/spack/runs                                                                                    │                                                                                                             
│ Report File:        /home/spack/buildtest/var/report.json                                                               │                                                                                                             
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/pre_post_cmds.yml │                                                                                                             
│                                                                                                                         │                                                                                                             
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                             
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                  Discovered buildspecs                   
╔════════════════════════════════════════════════════════╗
║ buildspec                                              ║
╟────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/pre_post_cmds.yml ║
╚════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/pre_post_cmds.yml: VALID
Total builder objects created: 1
                                                                      Builders by type=spack                                                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                        ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description  ┃ buildspecs                                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ run_pre_post_commands/c2936481 │ spack │ generic.local.bash │ None     │ None  │ None  │ Install zlib │ /home/spack/buildtest/examples/spack/pre_post_cmds.yml │
└────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────┴────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
run_pre_post_commands/c2936481: Creating Test Directory: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
run_pre_post_commands/c2936481 does not have any dependencies adding test to queue
     Builders Eligible to Run     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ run_pre_post_commands/c2936481 │
└────────────────────────────────┘
run_pre_post_commands/c2936481: Current Working Directory : /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/stage
run_pre_post_commands/c2936481: Running Test via command: bash run_pre_post_commands_build.sh
run_pre_post_commands/c2936481: Test completed in 12.086064 seconds with returncode: 0
run_pre_post_commands/c2936481: Writing output file -  /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.out
run_pre_post_commands/c2936481: Writing error file - /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.err
                                     Test Summary                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                        ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ run_pre_post_commands/c2936481 │ generic.local.bash │ PASS   │ 0          │ 12.086  │
└────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_f9pash9b.log
buildtest inspect query -o -t run_pre_post_commands
$ buildtest inspect query -o -t run_pre_post_commands 
────────────────────────────────────────────────────────────────────────────────────── run_pre_post_commands/c2936481-f1f4-468b-9170-fedbb8820c37 ──────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Install zlib
State: PASS
Returncode: 0
Runtime: 12.086064 sec
Starttime: 2024/02/08 19:01:59
Endtime: 2024/02/08 19:02:11
Command: bash run_pre_post_commands_build.sh
Test Script: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.sh
Build Script: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands_build.sh
Output File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.out
Error File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.err
Log File: /home/spack/buildtest/var/logs/buildtest_f9pash9b.log
─────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.out ────────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
PRETTY_NAME="Ubuntu 22.04.3 LTS"                                                                                                                                                                                                        
NAME="Ubuntu"                                                                                                                                                                                                                           
VERSION_ID="22.04"                                                                                                                                                                                                                      
VERSION="22.04.3 LTS (Jammy Jellyfish)"                                                                                                                                                                                                 
VERSION_CODENAME=jammy                                                                                                                                                                                                                  
ID=ubuntu                                                                                                                                                                                                                               
ID_LIKE=debian                                                                                                                                                                                                                          
HOME_URL="https://www.ubuntu.com/"                                                                                                                                                                                                      
SUPPORT_URL="https://help.ubuntu.com/"                                                                                                                                                                                                  
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"                                                                                                                                                                                     
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"                                                                                                                                                     
UBUNTU_CODENAME=jammy                                                                                                                                                                                                                   
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0                                                                                                                                                                                               
Copyright (C) 2021 Free Software Foundation, Inc.                                                                                                                                                                                       
This is free software; see the source for copying conditions.  There is NO                                                                                                                                                              
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.                                                                                                                                                             
[+] /home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/zlib-1.3-p6c6zbrub6hv5imwnruz4uty2u7y2nfq                                                                                                                        
/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/gcc-12.3.0-chmemdiqoycjlxz2myvdxqzt5don54uw                                                                                                                          
0.21.1 (e30fedab102f9281a220fb4fae82e3f8c43a82ac)                                                                                                                                                                                       
                                                                                                                                                                                                                                        
───────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/pre_post_cmds/run_pre_post_commands/c2936481/run_pre_post_commands.sh ─────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF PRE COMMANDS ########                                                                                                                                                                                                 
cat /etc/os-release                                                                                                                                                                                                                     
gcc --version                                                                                                                                                                                                                           
                                                                                                                                                                                                                                        
######## END OF PRE COMMANDS   ########                                                                                                                                                                                                 
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack install  zlib                                                                                                                                                                                                                     
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack location -i gcc@12.3.0                                                                                                                                                                                                            
spack --version                                                                                                                                                                                                                         
######## END OF POST COMMANDS   ########

Configuring Spack Mirrors

We can add mirrors in the spack instance or spack environment using the mirror property which is available in the spack and env section. If the mirrror property is part of the env section, the mirror will be added to spack environment. The mirror is an object that expects a Key/Value pair where the key is the name of mirror and value is location of the spack mirror.

In this next example, we will define a mirror name e4s that points to https://cache.e4s.io as the mirror location. Internally, this translates to spack mirror add e4s https://cache.e4s.io command.

buildspecs:
  add_mirror:
    type: spack
    executor: generic.local.bash
    description: Declare spack mirror
    tags: [spack]
    spack:
      root: $HOME/spack
      mirror:
        spack_tutorial_mirror: /mirror
    post_cmds: |
      spack mirror list
      spack config blame mirrors

  add_mirror_in_spack_env:
    type: spack
    executor: generic.local.bash
    description: Declare spack mirror in spack environment
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          name: spack_mirror
        activate:
          name: spack_mirror
        mirror:
          spack_tutorial_mirror: /mirror
    post_cmds: |
      spack mirror list
      spack config blame mirrors

This test can be built by running:

buildtest build -b $BUILDTEST_ROOT/examples/spack/mirror_example.yml

If we look at the generated script for both tests, we see that mirror is added for both tests. Note that one can have mirrors defined in their spack.yaml or one of the configuration scopes defined by spack.

buildtest inspect query -o -t add_mirror add_mirror_in_spack_env
$ buildtest inspect query -o  -t add_mirror add_mirror_in_spack_env 
─────────────────────────────────────────────────────────────────────────────────────────── add_mirror/09d8ced6-8324-43d4-9093-9d7011dc64f6 ────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Declare spack mirror
State: PASS
Returncode: 0
Runtime: 9.969547 sec
Starttime: 2024/02/08 19:02:11
Endtime: 2024/02/08 19:02:21
Command: bash add_mirror_build.sh
Test Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror.sh
Build Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror_build.sh
Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror.out
Error File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror.err
Log File: /home/spack/buildtest/var/logs/buildtest_vjkkqypo.log
────────────────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror.out ──────────────────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
spack_tutorial_mirror [sb] file:///mirror                                                                                                                                                                                               
tutorial              [sb] file:///mirror                                                                                                                                                                                               
spack-public          [s ] https://mirror.spack.io                                                                                                                                                                                      
---                                                  mirrors:                                                                                                                                                                           
/home/spack/.spack/mirrors.yaml:2                      spack_tutorial_mirror: /mirror                                                                                                                                                   
/home/spack/.spack/mirrors.yaml:3                      tutorial: file:///mirror                                                                                                                                                         
/home/spack/spack/etc/spack/defaults/mirrors.yaml:2    spack-public:                                                                                                                                                                    
/home/spack/spack/etc/spack/defaults/mirrors.yaml:3      binary: False                                                                                                                                                                  
/home/spack/spack/etc/spack/defaults/mirrors.yaml:4      url: https://mirror.spack.io                                                                                                                                                   
                                                                                                                                                                                                                                        
─────────────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror/09d8ced6/add_mirror.sh ────────────────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack mirror add spack_tutorial_mirror /mirror                                                                                                                                                                                          
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack mirror list                                                                                                                                                                                                                       
spack config blame mirrors                                                                                                                                                                                                              
                                                                                                                                                                                                                                        
######## END OF POST COMMANDS   ########                                                                                                                                                                                                
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
───────────────────────────────────────────────────────────────────────────────────── add_mirror_in_spack_env/8c8e7ddf-e05c-4f83-8a55-6b4014afc77a ─────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Declare spack mirror in spack environment
State: PASS
Returncode: 0
Runtime: 10.477546 sec
Starttime: 2024/02/08 19:02:11
Endtime: 2024/02/08 19:02:22
Command: bash add_mirror_in_spack_env_build.sh
Test Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env.sh
Build Script: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env_build.sh
Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env.out
Error File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env.err
Log File: /home/spack/buildtest/var/logs/buildtest_vjkkqypo.log
───────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env.out ─────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment 'spack_mirror' in /home/spack/spack/var/spack/environments/spack_mirror                                                                                                                                         
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate spack_mirror                                                                                                                                                                                                   
spack_tutorial_mirror [sb] file:///mirror                                                                                                                                                                                               
tutorial              [sb] file:///mirror                                                                                                                                                                                               
spack-public          [s ] https://mirror.spack.io                                                                                                                                                                                      
---                                                                  mirrors:                                                                                                                                                           
/home/spack/spack/var/spack/environments/spack_mirror/spack.yaml:12    spack_tutorial_mirror: /mirror                                                                                                                                   
/home/spack/.spack/mirrors.yaml:3                                      tutorial: file:///mirror                                                                                                                                         
/home/spack/spack/etc/spack/defaults/mirrors.yaml:2                    spack-public:                                                                                                                                                    
/home/spack/spack/etc/spack/defaults/mirrors.yaml:3                      binary: False                                                                                                                                                  
/home/spack/spack/etc/spack/defaults/mirrors.yaml:4                      url: https://mirror.spack.io                                                                                                                                   
                                                                                                                                                                                                                                        
────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/mirror_example/add_mirror_in_spack_env/8c8e7ddf/add_mirror_in_spack_env.sh ───────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env create  spack_mirror                                                                                                                                                                                                          
spack env activate  spack_mirror                                                                                                                                                                                                        
spack mirror add spack_tutorial_mirror /mirror                                                                                                                                                                                          
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack mirror list                                                                                                                                                                                                                       
spack config blame mirrors                                                                                                                                                                                                              
######## END OF POST COMMANDS   ########

Spack Test

Note

spack test requires version 0.16.0 or higher in order to use this feature.

buildtest can run tests via spack test run that can be used for testing installed specs which comes with builtin tests by the spack framework. In order to use this feature, you need to declare the test section which is type: object in JSON and run is a required property. The run section maps to spack test run that is responsible for running tests for a list of specs that are specified using the specs property.

Upon running the tests, we can retrieve results using spack test results which is configured using the results property. The results property can query test results one of the following ways:

  1. Spec Format: spack test results -- <spec>

  2. Suitename: spack test results <suitename>

In example below, we will test m4 package by running spack test run m4 and specify the -l option (i.e spack test results -l) which will retrieve the test log.

buildspecs:
  spack_test_m4:
    type: spack
    executor: generic.local.bash
    description: "Run spack test for m4 package and report results"
    tags: [spack]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['m4']
        results:
          option: "-l"

The spack test run –alias option is used to query results by suitename which can be used by spack test results command. buildtest will create a unique suite name for every run so you don’t have to remember the suite name when writing the buildspec.

buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml 
╭───────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────╮                                                                                                                
│                                                                                                                      │                                                                                                                
│ User:               spack                                                                                            │                                                                                                                
│ Hostname:           3220cd0ac8f6                                                                                     │                                                                                                                
│ Platform:           Linux                                                                                            │                                                                                                                
│ Current Time:       2024/02/08 19:02:33                                                                              │                                                                                                                
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                              │                                                                                                                
│ buildtest version:  1.7                                                                                              │                                                                                                                
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                          │                                                                                                                
│ python version:     3.11.6                                                                                           │                                                                                                                
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                     │                                                                                                                
│ Test Directory:     /home/spack/runs                                                                                 │                                                                                                                
│ Report File:        /home/spack/buildtest/var/report.json                                                            │                                                                                                                
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_test.yml │                                                                                                                
│                                                                                                                      │                                                                                                                
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                 Discovered buildspecs                 
╔═════════════════════════════════════════════════════╗
║ buildspec                                           ║
╟─────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_test.yml ║
╚═════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_test.yml: VALID
Total builder objects created: 1
                                                                                  Builders by type=spack                                                                                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                      ┃ buildspecs                                          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_test_m4/1849512d │ spack │ generic.local.bash │ None     │ None  │ None  │ Run spack test for m4 package and report results │ /home/spack/buildtest/examples/spack/spack_test.yml │
└────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_test_m4/1849512d: Creating Test Directory: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_test_m4/1849512d does not have any dependencies adding test to queue
 Builders Eligible to Run 
┏━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_test_m4/1849512d │
└────────────────────────┘
spack_test_m4/1849512d: Current Working Directory : /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/stage
spack_test_m4/1849512d: Running Test via command: bash spack_test_m4_build.sh
spack_test_m4/1849512d: Test completed in 7.837149 seconds with returncode: 0
spack_test_m4/1849512d: Writing output file -  /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.out
spack_test_m4/1849512d: Writing error file - /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.err
                                 Test Summary                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_test_m4/1849512d │ generic.local.bash │ PASS   │ 0          │ 7.837   │
└────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_gstxuuko.log

Take note of the generated test and the suite-name that is generated by buildtest.

buildtest inspect query -o -t spack_test_m4
$ buildtest inspect query -o -t spack_test_m4 
────────────────────────────────────────────────────────────────────────────────────────── spack_test_m4/1849512d-59b0-4e60-bc3a-cd0815f55d7d ──────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run spack test for m4 package and report results
State: PASS
Returncode: 0
Runtime: 7.837149 sec
Starttime: 2024/02/08 19:02:33
Endtime: 2024/02/08 19:02:41
Command: bash spack_test_m4_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.sh
Build Script: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.out
Error File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.err
Log File: /home/spack/buildtest/var/logs/buildtest_gstxuuko.log
───────────────────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.out ─────────────────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Spack test 2210f73d-1863-43d9-bd71-8dd5688b44b2                                                                                                                                                                                     
==> Testing package m4-1.4.19-jnv5nut                                                                                                                                                                                                   
==> Testing package m4-1.4.19-oebiztm                                                                                                                                                                                                   
============================= 2 passed of 2 specs ==============================                                                                                                                                                        
==> Results for test suite '2210f73d-1863-43d9-bd71-8dd5688b44b2':                                                                                                                                                                      
==> test specs:                                                                                                                                                                                                                         
==>   m4-1.4.19-jnv5nut PASSED                                                                                                                                                                                                          
==> Testing package m4-1.4.19-jnv5nut                                                                                                                                                                                                   
==> [2024-02-08-19:02:40.434974] test: test_hello: ensure m4 hello example runs                                                                                                                                                         
==> [2024-02-08-19:02:40.435704] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/m4-1.4.19-jnv5nut43juhm7esjn3tvxas6m67zbx4/bin/m4' '/home/spack/.spack/test/yo6e3tqenpu3iofea6nvc4l2i2sxbqw2/m4-1.4.19-jnv5nut/data
// macro is defined                                                                                                                                                                                                                     
Hello, World!                                                                                                                                                                                                                           
PASSED: M4::test_hello                                                                                                                                                                                                                  
==> [2024-02-08-19:02:40.439031] test: test_version: ensure m4 version matches installed spec                                                                                                                                           
==> [2024-02-08-19:02:40.439449] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/m4-1.4.19-jnv5nut43juhm7esjn3tvxas6m67zbx4/bin/m4' '--version'                                                                     
m4 (GNU M4) 1.4.19                                                                                                                                                                                                                      
Copyright (C) 2021 Free Software Foundation, Inc.                                                                                                                                                                                       
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.                                                                                                                                                         
This is free software: you are free to change and redistribute it.                                                                                                                                                                      
There is NO WARRANTY, to the extent permitted by law.                                                                                                                                                                                   
Written by Rene' Seindal.                                                                                                                                                                                                               
PASSED: M4::test_version                                                                                                                                                                                                                
==> [2024-02-08-19:02:40.441023] Completed testing                                                                                                                                                                                      
==> [2024-02-08-19:02:40.441121]                                                                                                                                                                                                        
========================== SUMMARY: m4-1.4.19-jnv5nut ==========================                                                                                                                                                        
M4::test_hello .. PASSED                                                                                                                                                                                                                
M4::test_version .. PASSED                                                                                                                                                                                                              
============================= 2 passed of 2 parts ==============================                                                                                                                                                        
==>   m4-1.4.19-oebiztm PASSED                                                                                                                                                                                                          
==> Testing package m4-1.4.19-oebiztm                                                                                                                                                                                                   
==> [2024-02-08-19:02:40.679533] test: test_hello: ensure m4 hello example runs                                                                                                                                                         
==> [2024-02-08-19:02:40.680284] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/m4-1.4.19-oebiztml2dkrtmp4ghswfv7yvnskbzjz/bin/m4' '/home/spack/.spack/test/yo6e3tqenpu3iofea6nvc4l2i2sxbqw2/m4-1.4.19-oebiztm/data
// macro is defined                                                                                                                                                                                                                     
Hello, World!                                                                                                                                                                                                                           
PASSED: M4::test_hello                                                                                                                                                                                                                  
==> [2024-02-08-19:02:40.682415] test: test_version: ensure m4 version matches installed spec                                                                                                                                           
==> [2024-02-08-19:02:40.682788] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/m4-1.4.19-oebiztml2dkrtmp4ghswfv7yvnskbzjz/bin/m4' '--version'                                                                     
m4 (GNU M4) 1.4.19                                                                                                                                                                                                                      
Copyright (C) 2021 Free Software Foundation, Inc.                                                                                                                                                                                       
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.                                                                                                                                                         
This is free software: you are free to change and redistribute it.                                                                                                                                                                      
There is NO WARRANTY, to the extent permitted by law.                                                                                                                                                                                   
Written by Rene' Seindal.                                                                                                                                                                                                               
PASSED: M4::test_version                                                                                                                                                                                                                
==> [2024-02-08-19:02:40.685394] Completed testing                                                                                                                                                                                      
==> [2024-02-08-19:02:40.685510]                                                                                                                                                                                                        
========================== SUMMARY: m4-1.4.19-oebiztm ==========================                                                                                                                                                        
M4::test_hello .. PASSED                                                                                                                                                                                                                
M4::test_version .. PASSED                                                                                                                                                                                                              
============================= 2 passed of 2 parts ==============================                                                                                                                                                        
============================= 2 passed of 2 specs ==============================                                                                                                                                                        
                                                                                                                                                                                                                                        
────────────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_test/spack_test_m4/1849512d/spack_test_m4.sh ───────────────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack test run --alias 2210f73d-1863-43d9-bd71-8dd5688b44b2 m4                                                                                                                                                                          
spack test results -l 2210f73d-1863-43d9-bd71-8dd5688b44b2

We can search for test results using the spec format instead of suite name. In the results property we can use specs field to specify a list of spec names to run. In spack, you can retrieve the results using spack test results -- <spec>, note that double dash -- is in front of spec name. We can pass options to spack test results using the option property which is available for results and run property. Currently, spack will write test results in $HOME/.spack/tests and we can use spack test remove to clear all test results. This can be done in buildspec using the remove_tests field which is a boolean. If this is set to True buildtest will run spack test remove -y to remove all test suites before running the tests.

In this next example, we will create a spack environment to install libxml2 and libsigsegv and test the package and report log after running test.

buildspecs:
  spack_test_results_specs_format:
    type: spack
    executor: generic.local.bash
    description: "Run spack test results with spec format"
    tags: [spack]
    spack:
      root: $HOME/spack
      env:
        create:
          remove_environment: true
          name: spack_test_example
        activate:
          name: spack_test_example
      install:
        specs: ['libxml2', 'libsigsegv']
        option: "--add"
      test:
        remove_tests: true
        run:
          specs: ['libxml2', 'libsigsegv']
        results:
          option: '-l'
          specs: ['libxml2', 'libsigsegv']
    post_cmds: |
      spack env deactivate
      spack env remove -y spack_test_example

We can build this test by running the following

buildtest build -b /home/spack/buildtest/examples/spack/spack_test_specs.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_test_specs.yml 
╭──────────────────────────────────────────────────── buildtest summary ─────────────────────────────────────────────────────╮                                                                                                          
│                                                                                                                            │                                                                                                          
│ User:               spack                                                                                                  │                                                                                                          
│ Hostname:           3220cd0ac8f6                                                                                           │                                                                                                          
│ Platform:           Linux                                                                                                  │                                                                                                          
│ Current Time:       2024/02/08 19:02:42                                                                                    │                                                                                                          
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                    │                                                                                                          
│ buildtest version:  1.7                                                                                                    │                                                                                                          
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                                │                                                                                                          
│ python version:     3.11.6                                                                                                 │                                                                                                          
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                           │                                                                                                          
│ Test Directory:     /home/spack/runs                                                                                       │                                                                                                          
│ Report File:        /home/spack/buildtest/var/report.json                                                                  │                                                                                                          
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_test_specs.yml │                                                                                                          
│                                                                                                                            │                                                                                                          
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                          
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                    Discovered buildspecs                    
╔═══════════════════════════════════════════════════════════╗
║ buildspec                                                 ║
╟───────────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_test_specs.yml ║
╚═══════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_test_specs.yml: VALID
Total builder objects created: 1
                                                                                          Builders by type=spack                                                                                          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                                  ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                             ┃ buildspecs                                                ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_test_results_specs_format/f3a03779 │ spack │ generic.local.bash │ None     │ None  │ None  │ Run spack test results with spec format │ /home/spack/buildtest/examples/spack/spack_test_specs.yml │
└──────────────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────┴───────────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_test_results_specs_format/f3a03779: Creating Test Directory: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_test_results_specs_format/f3a03779 does not have any dependencies adding test to queue
          Builders Eligible to Run          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_test_results_specs_format/f3a03779 │
└──────────────────────────────────────────┘
spack_test_results_specs_format/f3a03779: Current Working Directory : /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/stage
spack_test_results_specs_format/f3a03779: Running Test via command: bash spack_test_results_specs_format_build.sh
spack_test_results_specs_format/f3a03779: Test completed in 35.460273 seconds with returncode: 0
spack_test_results_specs_format/f3a03779: Writing output file -  /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.out
spack_test_results_specs_format/f3a03779: Writing error file - /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.err
                                          Test Summary                                           
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                                  ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_test_results_specs_format/f3a03779 │ generic.local.bash │ PASS   │ 0          │ 35.460  │
└──────────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_z1clop8d.log

Now let’s check the generated test and output file, we see buildtest will install libxml2 and libsigsegv in spack environment followed by removing all testsuites using spack test remove -y and run the test. Note that we can query results in spec format (spack test results --l --libxml2) where spack will try to match a result file that matches the corresponding spec.

buildtest inspect query -o -t spack_test_results_specs_format
$ buildtest inspect query -o -t spack_test_results_specs_format 
───────────────────────────────────────────────────────────────────────────────── spack_test_results_specs_format/f3a03779-5dfc-425b-87ba-47b58a13e500 ─────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run spack test results with spec format
State: PASS
Returncode: 0
Runtime: 35.460273 sec
Starttime: 2024/02/08 19:02:42
Endtime: 2024/02/08 19:03:17
Command: bash spack_test_results_specs_format_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.sh
Build Script: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.out
Error File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.err
Log File: /home/spack/buildtest/var/logs/buildtest_z1clop8d.log
──────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.out ────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment 'spack_test_example' in /home/spack/spack/var/spack/environments/spack_test_example                                                                                                                             
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate spack_test_example                                                                                                                                                                                             
==> Concretized libxml2                                                                                                                                                                                                                 
[+]  damyeos  libxml2@2.10.3%gcc@12.3.0+pic~python+shared build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                       
[+]  6qiak7n      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
[+]  a34xpad      ^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                                  
[+]  ccpwoda      ^pkgconf@1.9.5%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                     
[+]  taa3gkk      ^xz@5.4.1%gcc@12.3.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu22.04-x86_64_v3                                                                                                                   
[+]  draqwfy      ^zlib-ng@2.1.4%gcc@12.3.0+compat+opt build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                          
==> All of the packages are already installed                                                                                                                                                                                           
==> Updating view at /home/spack/spack/var/spack/environments/spack_test_example/.spack-env/view                                                                                                                                        
==> Concretized libsigsegv                                                                                                                                                                                                              
[+]  46vv5f3  libsigsegv@2.14%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                        
[+]  6qiak7n      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-ubuntu22.04-x86_64_v3                                                                                                                                   
==> All of the packages are already installed                                                                                                                                                                                           
==> Updating view at /home/spack/spack/var/spack/environments/spack_test_example/.spack-env/view                                                                                                                                        
==> Spack test 9c6b84e3-e1bc-44af-8925-d4003db9237c                                                                                                                                                                                     
==> Testing package libxml2-2.10.3-damyeos                                                                                                                                                                                              
==> Testing package libsigsegv-2.14-46vv5f3                                                                                                                                                                                             
============================= 2 passed of 2 specs ==============================                                                                                                                                                        
==> Results for test suite '9c6b84e3-e1bc-44af-8925-d4003db9237c', spec matching 'libxml2':                                                                                                                                             
==> test specs:                                                                                                                                                                                                                         
==>   libxml2-2.10.3-damyeos PASSED                                                                                                                                                                                                     
==> Testing package libxml2-2.10.3-damyeos                                                                                                                                                                                              
==> [2024-02-08-19:03:13.695601] test: test_import: import module test                                                                                                                                                                  
SKIPPED: Libxml2::test_import: Package must be built with +python                                                                                                                                                                       
==> [2024-02-08-19:03:13.696025] test: test_xml2_config: check version output                                                                                                                                                           
==> [2024-02-08-19:03:13.696690] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xml2-config' '--version'                                                       
2.10.3                                                                                                                                                                                                                                  
PASSED: Libxml2::test_xml2_config                                                                                                                                                                                                       
==> [2024-02-08-19:03:13.698302] test: test_xmlcatalog: check minimal creation output                                                                                                                                                   
==> [2024-02-08-19:03:13.698660] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xmlcatalog' '--create'                                                         
<?xml version="1.0"?>                                                                                                                                                                                                                   
<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">                                                                                  
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/>                                                                                                                                                                          
PASSED: Libxml2::test_xmlcatalog                                                                                                                                                                                                        
==> [2024-02-08-19:03:13.701802] test: test_xmllint: run xmllint generation and validation checks                                                                                                                                       
==> [2024-02-08-19:03:13.702347] test: test_xmllint_auto: generate test.xml                                                                                                                                                             
==> [2024-02-08-19:03:13.702770] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xmllint' '--auto' '-o' 'test.xml'                                              
PASSED: Libxml2::test_xmllint_auto                                                                                                                                                                                                      
==> [2024-02-08-19:03:13.705150] test: test_xmllint_validate_no_dtd: validate test.xml without a DTD                                                                                                                                    
==> [2024-02-08-19:03:13.705360] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xmllint' '--postvalid' 'test.xml'                                              
<?xml version="1.0"?>                                                                                                                                                                                                                   
<info>abc</info>                                                                                                                                                                                                                        
validity error : no DTD found!                                                                                                                                                                                                          
Document test.xml does not validate                                                                                                                                                                                                     
PASSED: Libxml2::test_xmllint_validate_no_dtd                                                                                                                                                                                           
==> [2024-02-08-19:03:13.707783] test: test_xmllint_validate_with_dtd: validate test.xml with a DTD                                                                                                                                     
==> [2024-02-08-19:03:13.708170] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xmllint' '--dtdvalid' '/home/spack/.spack/test/w2ar4eek6n52akgu237rjsak2d7gidoy
<?xml version="1.0"?>                                                                                                                                                                                                                   
<info>abc</info>                                                                                                                                                                                                                        
test.xml:2: element info: validity error : Element info content does not follow the DTD, expecting (data), got (CDATA)                                                                                                                  
Document test.xml does not validate against /home/spack/.spack/test/w2ar4eek6n52akgu237rjsak2d7gidoy/libxml2-2.10.3-damyeos/data/libxml2/info.dtd                                                                                       
PASSED: Libxml2::test_xmllint_validate_with_dtd                                                                                                                                                                                         
==> [2024-02-08-19:03:13.711812] test: test_xmllint_validate_works: validate /home/spack/.spack/test/w2ar4eek6n52akgu237rjsak2d7gidoy/libxml2-2.10.3-damyeos/data/libxml2/info.xml with a DTD                                           
==> [2024-02-08-19:03:13.712343] '/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libxml2-2.10.3-damyeoscgt5gdqacxr2xv2ewplkequwl/bin/xmllint' '--dtdvalid' '/home/spack/.spack/test/w2ar4eek6n52akgu237rjsak2d7gidoy
<?xml version="1.0"?>                                                                                                                                                                                                                   
<info>                                                                                                                                                                                                                                  
<data>abc</data>                                                                                                                                                                                                                        
</info>                                                                                                                                                                                                                                 
PASSED: Libxml2::test_xmllint_validate_works                                                                                                                                                                                            
PASSED: Libxml2::test_xmllint                                                                                                                                                                                                           
==> [2024-02-08-19:03:13.715409] Completed testing                                                                                                                                                                                      
==> [2024-02-08-19:03:13.715526]                                                                                                                                                                                                        
======================= SUMMARY: libxml2-2.10.3-damyeos ========================                                                                                                                                                        
Libxml2::test_import .. SKIPPED                                                                                                                                                                                                         
Libxml2::test_xml2_config .. PASSED                                                                                                                                                                                                     
Libxml2::test_xmlcatalog .. PASSED                                                                                                                                                                                                      
Libxml2::test_xmllint_auto .. PASSED                                                                                                                                                                                                    
Libxml2::test_xmllint_validate_no_dtd .. PASSED                                                                                                                                                                                         
Libxml2::test_xmllint_validate_with_dtd .. PASSED                                                                                                                                                                                       
Libxml2::test_xmllint_validate_works .. PASSED                                                                                                                                                                                          
Libxml2::test_xmllint .. PASSED                                                                                                                                                                                                         
======================== 1 skipped, 7 passed of 8 parts ========================                                                                                                                                                        
============================== 1 passed of 1 spec ==============================                                                                                                                                                        
==> Results for test suite '9c6b84e3-e1bc-44af-8925-d4003db9237c', spec matching 'libsigsegv':                                                                                                                                          
==> test specs:                                                                                                                                                                                                                         
==>   libsigsegv-2.14-46vv5f3 PASSED                                                                                                                                                                                                    
==> Testing package libsigsegv-2.14-46vv5f3                                                                                                                                                                                             
==> [2024-02-08-19:03:14.006703] Reading config from file /home/spack/spack/etc/spack/defaults/packages.yaml                                                                                                                            
==> [2024-02-08-19:03:14.042780] Reading config from file /etc/spack/packages.yaml                                                                                                                                                      
==> [2024-02-08-19:03:14.050225] Installing /home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libsigsegv-2.14-46vv5f3mkcqgishvdf7qrgtuc3vhl3gg/.spack/test to /home/spack/.spack/test/w2ar4eek6n52akgu237rjsak2d7gidoy/
==> [2024-02-08-19:03:14.052564] test: test_smoke_test: build and run smoke test                                                                                                                                                        
==> [2024-02-08-19:03:14.053350] '/home/spack/spack/lib/spack/env/gcc/gcc' '-I/home/spack/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-12.3.0/libsigsegv-2.14-46vv5f3mkcqgishvdf7qrgtuc3vhl3gg/include' '/home/spack/.spack/test/w2ar
==> [2024-02-08-19:03:14.112899] 'smoke_test'                                                                                                                                                                                           
Caught sigsegv #1                                                                                                                                                                                                                       
Hello World!                                                                                                                                                                                                                            
PASSED: Libsigsegv::test_smoke_test                                                                                                                                                                                                     
==> [2024-02-08-19:03:14.115089] Completed testing                                                                                                                                                                                      
==> [2024-02-08-19:03:14.115179]                                                                                                                                                                                                        
======================= SUMMARY: libsigsegv-2.14-46vv5f3 =======================                                                                                                                                                        
Libsigsegv::test_smoke_test .. PASSED                                                                                                                                                                                                   
============================= 1 passed of 1 parts ==============================                                                                                                                                                        
============================== 1 passed of 1 spec ==============================                                                                                                                                                        
==> Successfully removed environment 'spack_test_example'                                                                                                                                                                               
                                                                                                                                                                                                                                        
───────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_test_specs/spack_test_results_specs_format/f3a03779/spack_test_results_specs_format.sh ──────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack env rm -y spack_test_example || true                                                                                                                                                                                              
spack env create  spack_test_example                                                                                                                                                                                                    
spack env activate  spack_test_example                                                                                                                                                                                                  
spack install --add libxml2                                                                                                                                                                                                             
spack install --add libsigsegv                                                                                                                                                                                                          
spack test remove -y                                                                                                                                                                                                                    
spack test run --alias 9c6b84e3-e1bc-44af-8925-d4003db9237c libxml2 libsigsegv                                                                                                                                                          
spack test results -l -- libxml2                                                                                                                                                                                                        
spack test results -l -- libsigsegv                                                                                                                                                                                                     
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack env deactivate                                                                                                                                                                                                                    
spack env remove -y spack_test_example                                                                                                                                                                                                  
######## END OF POST COMMANDS   ########

Specifying Scheduler Directives

The spack schema supports all of the scheduler scheduler directives such as sbatch, bsub and pbs property in the buildspec.

The directives are applied at top of script. Shown below is a toy example that will define directives using sbatch property. Note, this test won’t submit job to scheduler since we are not using the a slurm executor.

buildspecs:
  spack_sbatch_example:
    type: spack
    executor: generic.local.bash
    description: "sbatch directives can be defined in spack schema"
    tags: [spack]
    sbatch: ["-N 1", "-n 8", "-t 30"]
    spack:
      root: $HOME/spack
      install:
        specs: ['zlib%clang', 'zlib%gcc']

buildtest will generate the shell script with the job directives and set the name, output and error files based on name of test. If we build this test, and inspect the generated test we see that #SBATCH directives are written based on the sbatch field.

buildtest build -b /home/spack/buildtest/examples/spack/spack_sbatch.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_sbatch.yml 
╭────────────────────────────────────────────────── buildtest summary ───────────────────────────────────────────────────╮                                                                                                              
│                                                                                                                        │                                                                                                              
│ User:               spack                                                                                              │                                                                                                              
│ Hostname:           3220cd0ac8f6                                                                                       │                                                                                                              
│ Platform:           Linux                                                                                              │                                                                                                              
│ Current Time:       2024/02/08 19:03:18                                                                                │                                                                                                              
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                                │                                                                                                              
│ buildtest version:  1.7                                                                                                │                                                                                                              
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                            │                                                                                                              
│ python version:     3.11.6                                                                                             │                                                                                                              
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                       │                                                                                                              
│ Test Directory:     /home/spack/runs                                                                                   │                                                                                                              
│ Report File:        /home/spack/buildtest/var/report.json                                                              │                                                                                                              
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_sbatch.yml │                                                                                                              
│                                                                                                                        │                                                                                                              
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                              
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                  Discovered buildspecs                  
╔═══════════════════════════════════════════════════════╗
║ buildspec                                             ║
╟───────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_sbatch.yml ║
╚═══════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_sbatch.yml: VALID
Total builder objects created: 1
                                                                                       Builders by type=spack                                                                                       
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                       ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                      ┃ buildspecs                                            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_sbatch_example/55266bd0 │ spack │ generic.local.bash │ None     │ None  │ None  │ sbatch directives can be defined in spack schema │ /home/spack/buildtest/examples/spack/spack_sbatch.yml │
└───────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴──────────────────────────────────────────────────┴───────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_sbatch_example/55266bd0: Creating Test Directory: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_sbatch_example/55266bd0 does not have any dependencies adding test to queue
    Builders Eligible to Run     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_sbatch_example/55266bd0 │
└───────────────────────────────┘
spack_sbatch_example/55266bd0: Current Working Directory : /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/stage
spack_sbatch_example/55266bd0: Running Test via command: bash spack_sbatch_example_build.sh
spack_sbatch_example/55266bd0: Test completed in 20.449731 seconds with returncode: 0
spack_sbatch_example/55266bd0: Writing output file -  /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.out
spack_sbatch_example/55266bd0: Writing error file - /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.err
                                     Test Summary                                     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                       ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_sbatch_example/55266bd0 │ generic.local.bash │ PASS   │ 0          │ 20.450  │
└───────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_ndy35mwi.log
buildtest inspect query -t spack_sbatch_example
$ buildtest inspect query -t spack_sbatch_example 
────────────────────────────────────────────────────────────────────────────────────── spack_sbatch_example/55266bd0-1e0d-4668-a3e5-c2f4548c3594 ───────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: sbatch directives can be defined in spack schema
State: PASS
Returncode: 0
Runtime: 20.449731 sec
Starttime: 2024/02/08 19:03:18
Endtime: 2024/02/08 19:03:38
Command: bash spack_sbatch_example_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.sh
Build Script: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.out
Error File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.err
Log File: /home/spack/buildtest/var/logs/buildtest_ndy35mwi.log
────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_sbatch/spack_sbatch_example/55266bd0/spack_sbatch_example.sh ───────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
#SBATCH -N 1                                                                                                                                                                                                                            
#SBATCH -n 8                                                                                                                                                                                                                            
#SBATCH -t 30                                                                                                                                                                                                                           
#SBATCH --job-name=spack_sbatch_example                                                                                                                                                                                                 
#SBATCH --output=spack_sbatch_example.out                                                                                                                                                                                               
#SBATCH --error=spack_sbatch_example.err                                                                                                                                                                                                
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack install  zlib%clang                                                                                                                                                                                                               
spack install  zlib%gcc

You can define multiple executors in your buildspec with spack schema via executors. This can be useful if you need to specify different scheduler directives based on executor type since your executor will map to a queue.

Shown below is an example buildspec that will specify sbatch directives for generic.local.sh and generic.local.bash

buildspecs:
  spack_sbatch_multi_executors:
    type: spack
    executor: "generic.local.(sh|bash)"
    description: "sbatch directives can be defined in spack schema"
    tags: [spack]
    executors:
      generic.local.sh:
        sbatch: ["-N 1", "-t 30"]
      generic.local.bash:
        sbatch: ["-N 8", "-t 15"]
    spack:
      root: $HOME/spack
      test:
        run:
          specs: ['libsigsegv']
        results:
          specs: ['libsigsegv']
          option: "-l"

Cloning Spack

buildtest will automatically clone spack if root is not specified in the buildspec, which will be performed in the test directory where the test is executed. In example below have two tests, first one will clone spack automatically and second test will clone explicitly in /tmp where we specify root property.

buildspecs:
  clone_spack_automatically:
    type: spack
    executor: generic.local.bash
    description: "Clone spack automatically"
    tags: [ spack ]
    spack:
      env:
        create:
          name: 'spack-develop'
    post_cmds: |
      spack env list
      which spack

  clone_spack_and_specify_root:
    type: spack
    executor: generic.local.bash
    description: Clone spack explicitly and specify root
    tags: [ spack ]
    pre_cmds: git clone -b e4s-23.05 https://github.com/spack/spack.git /tmp/spack-demo
    spack:
      root: /tmp/spack-demo
      env:
        create:
          name: 'e4s'
    post_cmds: |
      spack env list
      which spack

Let’s build the following test

buildtest build -b /home/spack/buildtest/examples/spack/spack_clone.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/clone_spack.yml 
╭────────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────╮                                                                                                               
│                                                                                                                       │                                                                                                               
│ User:               spack                                                                                             │                                                                                                               
│ Hostname:           3220cd0ac8f6                                                                                      │                                                                                                               
│ Platform:           Linux                                                                                             │                                                                                                               
│ Current Time:       2024/02/08 19:05:16                                                                               │                                                                                                               
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                               │                                                                                                               
│ buildtest version:  1.7                                                                                               │                                                                                                               
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                           │                                                                                                               
│ python version:     3.11.6                                                                                            │                                                                                                               
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                      │                                                                                                               
│ Test Directory:     /home/spack/runs                                                                                  │                                                                                                               
│ Report File:        /home/spack/buildtest/var/report.json                                                             │                                                                                                               
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/clone_spack.yml │                                                                                                               
│                                                                                                                       │                                                                                                               
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                               
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                 Discovered buildspecs                  
╔══════════════════════════════════════════════════════╗
║ buildspec                                            ║
╟──────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/clone_spack.yml ║
╚══════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/clone_spack.yml: VALID
Total builder objects created: 2
                                                                                      Builders by type=spack                                                                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                               ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                             ┃ buildspecs                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ clone_spack_automatically/5f062c7c    │ spack │ generic.local.bash │ None     │ None  │ None  │ Clone spack automatically               │ /home/spack/buildtest/examples/spack/clone_spack.yml │
├───────────────────────────────────────┼───────┼────────────────────┼──────────┼───────┼───────┼─────────────────────────────────────────┼──────────────────────────────────────────────────────┤
│ clone_spack_and_specify_root/cbc530a3 │ spack │ generic.local.bash │ None     │ None  │ None  │ Clone spack explicitly and specify root │ /home/spack/buildtest/examples/spack/clone_spack.yml │
└───────────────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────┴──────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
clone_spack_automatically/5f062c7c: Creating Test Directory: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c
clone_spack_and_specify_root/cbc530a3: Creating Test Directory: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
clone_spack_automatically/5f062c7c does not have any dependencies adding test to queue
clone_spack_and_specify_root/cbc530a3 does not have any dependencies adding test to queue
        Builders Eligible to Run         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                               ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ clone_spack_automatically/5f062c7c    │
│ clone_spack_and_specify_root/cbc530a3 │
└───────────────────────────────────────┘
clone_spack_automatically/5f062c7c: Current Working Directory : /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/stage
clone_spack_and_specify_root/cbc530a3: Current Working Directory : /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/stage
clone_spack_and_specify_root/cbc530a3: Running Test via command: bash clone_spack_and_specify_root_build.sh
clone_spack_automatically/5f062c7c: Running Test via command: bash clone_spack_automatically_build.sh
clone_spack_and_specify_root/cbc530a3: Test completed in 62.922582 seconds with returncode: 0
clone_spack_and_specify_root/cbc530a3: Writing output file -  /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.out
clone_spack_and_specify_root/cbc530a3: Writing error file - /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.err
clone_spack_automatically/5f062c7c: Test completed in 63.326133 seconds with returncode: 0
clone_spack_automatically/5f062c7c: Writing output file -  /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.out
clone_spack_automatically/5f062c7c: Writing error file - /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.err
                                         Test Summary                                         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                               ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ clone_spack_automatically/5f062c7c    │ generic.local.bash │ PASS   │ 0          │ 63.326  │
├───────────────────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ clone_spack_and_specify_root/cbc530a3 │ generic.local.bash │ PASS   │ 0          │ 62.923  │
└───────────────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%


Adding 2 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_idqnwz4r.log

Let’s check the generated output, take note in the output the full path to where spack binary is present in each test. You must clone spack in pre_cmds in second test in order for buildtest to find the spack binary since you need to specify this the root property in-order for buildtest to install spack in the environment.

buildtest inspect query -t clone_spack_automatically clone_spack_and_specify_root
$ buildtest inspect query -o -t clone_spack_automatically clone_spack_and_specify_root 
──────────────────────────────────────────────────────────────────────────────────── clone_spack_automatically/5f062c7c-fb7a-4376-b386-f9fe474d3b13 ────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Clone spack automatically
State: PASS
Returncode: 0
Runtime: 63.326133 sec
Starttime: 2024/02/08 19:05:16
Endtime: 2024/02/08 19:06:19
Command: bash clone_spack_automatically_build.sh
Test Script: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.sh
Build Script: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically_build.sh
Output File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.out
Error File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.err
Log File: /home/spack/buildtest/var/logs/buildtest_idqnwz4r.log
──────────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.out ─────────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment 'spack-develop' in /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/stage/spack/var/spack/environments/spack-develop                                                          
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate spack-develop                                                                                                                                                                                                  
    spack-develop                                                                                                                                                                                                                       
/home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/stage/spack/bin/spack                                                                                                                                
                                                                                                                                                                                                                                        
────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_automatically/5f062c7c/clone_spack_automatically.sh ──────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
git clone https://github.com/spack/spack                                                                                                                                                                                                
source ./spack/share/spack/setup-env.sh                                                                                                                                                                                                 
spack env create  spack-develop                                                                                                                                                                                                         
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack env list                                                                                                                                                                                                                          
which spack                                                                                                                                                                                                                             
                                                                                                                                                                                                                                        
######## END OF POST COMMANDS   ########                                                                                                                                                                                                
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
────────────────────────────────────────────────────────────────────────────────── clone_spack_and_specify_root/cbc530a3-df8c-4c29-97d8-01af9c238ddd ───────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Clone spack explicitly and specify root
State: PASS
Returncode: 0
Runtime: 62.922582 sec
Starttime: 2024/02/08 19:05:16
Endtime: 2024/02/08 19:06:19
Command: bash clone_spack_and_specify_root_build.sh
Test Script: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.sh
Build Script: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root_build.sh
Output File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.out
Error File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.err
Log File: /home/spack/buildtest/var/logs/buildtest_idqnwz4r.log
───────────────────────────────────────────────── Output File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.out ──────────────────────────────────────────────────
==> Regenerating tcl module files                                                                                                                                                                                                       
==> Created environment 'e4s' in /tmp/spack-demo/var/spack/environments/e4s                                                                                                                                                             
==> You can activate this environment with:                                                                                                                                                                                             
==>   spack env activate e4s                                                                                                                                                                                                            
    e4s                                                                                                                                                                                                                                 
/tmp/spack-demo/bin/spack                                                                                                                                                                                                               
                                                                                                                                                                                                                                        
─────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/clone_spack/clone_spack_and_specify_root/cbc530a3/clone_spack_and_specify_root.sh ───────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF PRE COMMANDS ########                                                                                                                                                                                                 
git clone -b e4s-23.05 https://github.com/spack/spack.git /tmp/spack-demo                                                                                                                                                               
######## END OF PRE COMMANDS   ########                                                                                                                                                                                                 
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
source /tmp/spack-demo/share/spack/setup-env.sh                                                                                                                                                                                         
spack env create  e4s                                                                                                                                                                                                                   
                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                        
######## START OF POST COMMANDS ########                                                                                                                                                                                                
spack env list                                                                                                                                                                                                                          
which spack                                                                                                                                                                                                                             
######## END OF POST COMMANDS   ########

Loading Specs

We can load specs into our user environment via spack load which can be used for running tests or simply loading the package once it is installed. In this next example, we will test m4 package, where we load the package first prior to testing it. The load refers to spack load and options refers to command options passed to spack load command with a list of specs to load defined by specs property.

In this test, the command would translate to spack load --only package m4

buildspecs:
  spack_load_example:
    type: spack
    executor: generic.local.bash
    description: "Run spack load for m4 package and run tests"
    tags: [spack]
    spack:
      root: $HOME/spack
      load:
        options: '--only package'
        specs: ['m4%gcc@12']
      test:
        run:
          specs: ['m4%gcc@12']
        results:
          option: "-l"

Let’s build this by running the following

buildtest build -b /home/spack/buildtest/examples/spack/spack_load.yml
$ buildtest build -b /home/spack/buildtest/examples/spack/spack_load.yml 
╭───────────────────────────────────────────────── buildtest summary ──────────────────────────────────────────────────╮                                                                                                                
│                                                                                                                      │                                                                                                                
│ User:               spack                                                                                            │                                                                                                                
│ Hostname:           3220cd0ac8f6                                                                                     │                                                                                                                
│ Platform:           Linux                                                                                            │                                                                                                                
│ Current Time:       2024/02/08 19:02:23                                                                              │                                                                                                                
│ buildtest path:     /home/spack/buildtest/bin/buildtest                                                              │                                                                                                                
│ buildtest version:  1.7                                                                                              │                                                                                                                
│ python path:        /home/spack/pyenv/buildtest/bin/python3                                                          │                                                                                                                
│ python version:     3.11.6                                                                                           │                                                                                                                
│ Configuration File: /home/spack/buildtest/buildtest/settings/spack_container.yml                                     │                                                                                                                
│ Test Directory:     /home/spack/runs                                                                                 │                                                                                                                
│ Report File:        /home/spack/buildtest/var/report.json                                                            │                                                                                                                
│ Command:            /home/spack/buildtest/bin/buildtest build -b /home/spack/buildtest/examples/spack/spack_load.yml │                                                                                                                
│                                                                                                                      │                                                                                                                
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                
───────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────
                 Discovered buildspecs                 
╔═════════════════════════════════════════════════════╗
║ buildspec                                           ║
╟─────────────────────────────────────────────────────╢
║ /home/spack/buildtest/examples/spack/spack_load.yml ║
╚═════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ──────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/spack/buildtest/examples/spack/spack_load.yml: VALID
Total builder objects created: 1
                                                                                  Builders by type=spack                                                                                   
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder                     ┃ type  ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                 ┃ buildspecs                                          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_load_example/e72ddfe0 │ spack │ generic.local.bash │ None     │ None  │ None  │ Run spack load for m4 package and run tests │ /home/spack/buildtest/examples/spack/spack_load.yml │
└─────────────────────────────┴───────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────┴─────────────────────────────────────────────────────┘
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_load_example/e72ddfe0: Creating Test Directory: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0
──────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 8 processes for processing builders
───────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
spack_load_example/e72ddfe0 does not have any dependencies adding test to queue
   Builders Eligible to Run    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder                     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ spack_load_example/e72ddfe0 │
└─────────────────────────────┘
spack_load_example/e72ddfe0: Current Working Directory : /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/stage
spack_load_example/e72ddfe0: Running Test via command: bash spack_load_example_build.sh
spack_load_example/e72ddfe0: Test completed in 9.780865 seconds with returncode: 0
spack_load_example/e72ddfe0: Writing output file -  /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.out
spack_load_example/e72ddfe0: Writing error file - /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.err
                                    Test Summary                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder                     ┃ executor           ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ spack_load_example/e72ddfe0 │ generic.local.bash │ PASS   │ 0          │ 9.781   │
└─────────────────────────────┴────────────────────┴────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /home/spack/buildtest/var/report.json
Writing Logfile to /home/spack/buildtest/var/logs/buildtest_17hscxmj.log

Let’s take a look at the generated test, take note of the spack load command

buildtest inspect query -o -t spack_load_example
$ buildtest inspect query -t spack_load_example 
─────────────────────────────────────────────────────────────────────────────────────── spack_load_example/e72ddfe0-e0e7-454e-a121-1ea7535a5baa ────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run spack load for m4 package and run tests
State: PASS
Returncode: 0
Runtime: 9.780865 sec
Starttime: 2024/02/08 19:02:23
Endtime: 2024/02/08 19:02:33
Command: bash spack_load_example_build.sh
Test Script: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.sh
Build Script: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example_build.sh
Output File: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.out
Error File: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.err
Log File: /home/spack/buildtest/var/logs/buildtest_17hscxmj.log
───────────────────────────────────────────────────────────── Test File: /home/spack/runs/generic.local.bash/spack_load/spack_load_example/e72ddfe0/spack_load_example.sh ──────────────────────────────────────────────────────────────
#!/bin/bash                                                                                                                                                                                                                             
set -eo pipefail                                                                                                                                                                                                                        
source /home/spack/spack/share/spack/setup-env.sh                                                                                                                                                                                       
spack load  m4%gcc@12                                                                                                                                                                                                                   
spack test run --alias 1a5e3a2a-d606-4c94-be9c-8a69c6cf5b33 m4%gcc@12                                                                                                                                                                   
spack test results -l 1a5e3a2a-d606-4c94-be9c-8a69c6cf5b33