Running Test Across Multiple Executors

The executor property can support regular expression to search for compatible executors, this can be used if you want to run a test across multiple executors. In buildtest, we use re.fullmatch with the input pattern defined by executor property against a list of available executors defined in configuration file. You can retrieve a list of executors by running buildtest config executors.

In example below we will run this test on generic.local.bash and generic.local.sh executor based on the regular expression.

buildspecs:
  multiple_executors:
    type: script
    executor: 'generic.local.(bash|sh)'
    description: run test with executor generic.local.bash and generic.local.sh executor
    tags: [tutorials]
    run: date

If we build this test, notice that there are two tests, one for each executor.

buildtest build -b tutorials/multi_executors/executor_regex_script.yml
$ buildtest build -b tutorials/multi_executors/executor_regex_script.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-19398495-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2023/02/06 21:51:49                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.1                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.7.15                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/t… ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/tutorials/multi_executors/executor_regex_script.yml: VALID
Total builder objects created: 2
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ multipl… │ script │ generic… │ None     │ None  │ None  │ run test │ /home/… │
│          │        │          │          │       │       │ with     │         │
│          │        │          │          │       │       │ executor │         │
│          │        │          │          │       │       │ generic… │         │
│          │        │          │          │       │       │ and      │         │
│          │        │          │          │       │       │ generic… │         │
│          │        │          │          │       │       │ executor │         │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ multipl… │ script │ generic… │ None     │ None  │ None  │ run test │ /home/… │
│          │        │          │          │       │       │ with     │         │
│          │        │          │          │       │       │ executor │         │
│          │        │          │          │       │       │ generic… │         │
│          │        │          │          │       │       │ and      │         │
│          │        │          │          │       │       │ generic… │         │
│          │        │          │          │       │       │ executor │         │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
multiple_executors/3efc552f: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f
multiple_executors/3efc552f: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f/stage
multiple_executors/3efc552f: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f/multiple_executors_build.sh
multiple_executors/37502c23: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23
multiple_executors/37502c23: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23/stage
multiple_executors/37502c23: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23/multiple_executors_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
multiple_executors/3efc552f does not have any dependencies adding test to queue
multiple_executors/37502c23 does not have any dependencies adding test to queue
multiple_executors/3efc552f: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f/stage
multiple_executors/3efc552f: Running Test via command: bash --norc --noprofile -eo pipefail multiple_executors_build.sh
multiple_executors/3efc552f: Test completed in 0.007421 seconds
multiple_executors/3efc552f: Test completed with returncode: 0
multiple_executors/3efc552f: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f/multiple_executors.out
multiple_executors/3efc552f: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_regex_script/multiple_executors/3efc552f/multiple_executors.err
multiple_executors/37502c23: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23/stage
multiple_executors/37502c23: Running Test via command: sh --norc --noprofile -eo pipefail multiple_executors_build.sh
multiple_executors/37502c23: Test completed in 0.002958 seconds
multiple_executors/37502c23: Test completed with returncode: 2
multiple_executors/37502c23: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23/multiple_executors.out
multiple_executors/37502c23: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_regex_script/multiple_executors/37502c23/multiple_executors.err
In this iteration we are going to run the following tests: [multiple_executors/3efc552f, multiple_executors/37502c23]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ builder      ┃ executor     ┃ status ┃ Runtime)      ┃ returncode ┃ runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ multiple_exe │ generic.loc… │ FAIL   │ N/A N/A N/A   │ 2          │ 0.002958 │
│ cutors/37502 │              │        │               │            │          │
│ c23          │              │        │               │            │          │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ multiple_exe │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.007421 │
│ cutors/3efc5 │              │        │               │            │          │
│ 52f          │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/report.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_cpzd429z.log

Multiple Executors

Note

This feature is in active development

Note

This feature is compatible with type: script and type: spack.

The executors property can be used to define executor specific configuration for each test, currently this field can be used with vars, env , scheduler directives: sbatch, bsub, pbs, cobalt and cray burst buffer/data warp. The executors field is a JSON object that expects name of executor followed by property set per executor. In this next example, we define variables X, Y and environment SHELL based on executors generic.local.sh and generic.local.bash.

buildspecs:
  executors_vars_env_declaration:
    type: script
    executor: 'generic.local.(bash|sh)'
    description: Declaring env and vars by executors section
    tags: [tutorials]
    run: |
      echo "X:" $X
      echo "Y:" $Y
      echo $SHELL
    executors:
      generic.local.bash:
        vars:
          X: 1
          Y: 3
        env:
          SHELL: bash
      generic.local.sh:
        vars:
          X: 2
          Y: 4
        env:
          SHELL: sh

Let’s build this test.

buildtest build -b tutorials/multi_executors/executors_var_env_declaration.yml
$ buildtest build -b tutorials/multi_executors/executors_var_env_declaration.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-19398495-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2023/02/06 21:51:49                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.1                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.7.15                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/t… ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/tutorials/multi_executors/executors_var_env_declaration.yml: VALID
Total builder objects created: 2
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ executo… │ script │ generic… │ None     │ None  │ None  │ Declari… │ /home/… │
│          │        │          │          │       │       │ env and  │         │
│          │        │          │          │       │       │ vars by  │         │
│          │        │          │          │       │       │ executo… │         │
│          │        │          │          │       │       │ section  │         │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ executo… │ script │ generic… │ None     │ None  │ None  │ Declari… │ /home/… │
│          │        │          │          │       │       │ env and  │         │
│          │        │          │          │       │       │ vars by  │         │
│          │        │          │          │       │       │ executo… │         │
│          │        │          │          │       │       │ section  │         │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
executors_vars_env_declaration/42956b6a: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a
executors_vars_env_declaration/42956b6a: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/stage
executors_vars_env_declaration/42956b6a: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration_build.sh
executors_vars_env_declaration/f5852a0a: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a
executors_vars_env_declaration/f5852a0a: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/stage
executors_vars_env_declaration/f5852a0a: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
executors_vars_env_declaration/f5852a0a does not have any dependencies adding test to queue
executors_vars_env_declaration/42956b6a does not have any dependencies adding test to queue
executors_vars_env_declaration/f5852a0a: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/stage
executors_vars_env_declaration/f5852a0a: Running Test via command: sh --norc --noprofile -eo pipefail executors_vars_env_declaration_build.sh
executors_vars_env_declaration/f5852a0a: Test completed in 0.003144 seconds
executors_vars_env_declaration/f5852a0a: Test completed with returncode: 2
executors_vars_env_declaration/f5852a0a: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration.out
executors_vars_env_declaration/f5852a0a: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration.err
executors_vars_env_declaration/42956b6a: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/stage
executors_vars_env_declaration/42956b6a: Running Test via command: bash --norc --noprofile -eo pipefail executors_vars_env_declaration_build.sh
executors_vars_env_declaration/42956b6a: Test completed in 0.005383 seconds
executors_vars_env_declaration/42956b6a: Test completed with returncode: 0
executors_vars_env_declaration/42956b6a: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration.out
executors_vars_env_declaration/42956b6a: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration.err
In this iteration we are going to run the following tests: [executors_vars_env_declaration/f5852a0a, executors_vars_env_declaration/42956b6a]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ builder      ┃ executor     ┃ status ┃ Runtime)      ┃ returncode ┃ runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ executors_va │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.005383 │
│ rs_env_decla │              │        │               │            │          │
│ ration/42956 │              │        │               │            │          │
│ b6a          │              │        │               │            │          │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ executors_va │ generic.loc… │ FAIL   │ N/A N/A N/A   │ 2          │ 0.003144 │
│ rs_env_decla │              │        │               │            │          │
│ ration/f5852 │              │        │               │            │          │
│ a0a          │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/report.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_ajsxcra_.log

Now let’s look at the generated content of the test as follows. We will see that buildtest will set X=1, Y=3 and SHELL=bash for generic.local.bash and X=2, Y=4 and SHELL=sh for generic.local.sh

buildtest inspect query -t executors_vars_env_declaration/
$ buildtest inspect query -t executors_vars_env_declaration/
───── executors_vars_env_declaration/f5852a0a-8d39-47d2-8b1d-8b319130f468 ──────
Executor: generic.local.sh
Description: Declaring env and vars by executors section
State: FAIL
Returncode: 2
Runtime: 0.003144 sec
Starttime: 2023/02/06 21:51:50
Endtime: 2023/02/06 21:51:50
Command: sh --norc --noprofile -eo pipefail executors_vars_env_declaration_build.sh
Test Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration.sh
Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration_build.sh
Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration.out
Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executors_var_env_declaration/executors_vars_env_declaration/f5852a0a/executors_vars_env_declaration.err
Log File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_ajsxcra_.log
─ Test File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/check… ─
#!/usr/bin/sh                                                                   
export SHELL="sh"                                                               
X="2"                                                                           
Y="4"                                                                           
# Content of run section                                                        
echo "X:" $X                                                                    
echo "Y:" $Y                                                                    
echo $SHELL                                                                     
                                                                                
───── executors_vars_env_declaration/42956b6a-4781-4543-b1b2-db705557fa5c ──────
Executor: generic.local.bash
Description: Declaring env and vars by executors section
State: PASS
Returncode: 0
Runtime: 0.005383 sec
Starttime: 2023/02/06 21:51:50
Endtime: 2023/02/06 21:51:50
Command: bash --norc --noprofile -eo pipefail executors_vars_env_declaration_build.sh
Test Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration.sh
Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration_build.sh
Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration.out
Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executors_var_env_declaration/executors_vars_env_declaration/42956b6a/executors_vars_env_declaration.err
Log File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_ajsxcra_.log
─ Test File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/check… ─
#!/usr/bin/bash                                                                 
export SHELL="bash"                                                             
X="1"                                                                           
Y="3"                                                                           
# Content of run section                                                        
echo "X:" $X                                                                    
echo "Y:" $Y                                                                    
echo $SHELL

Scheduler Directives

We can also define scheduler directives based on executor type, in this example we define sbatch property per executor type. Note that sbatch property in the executors section will override the sbatch property defined in the top-level file otherwise it will use the default.

buildspecs:
  executors_sbatch_declaration:
    type: script
    executor: 'generic.local.(bash|sh)'
    description: Declaring env and vars by executors section
    tags: [tutorials]
    run: hostname
    sbatch: ["-N 4"]
    executors:
      generic.local.bash:
        sbatch: ["-n 4", "-N 1", "-t 30"]
      generic.local.sh:
        sbatch: ["-n 8", "-N 1", "-t 60"]
buildtest build -b tutorials/multi_executors/executor_scheduler.yml
$ buildtest build -b tutorials/multi_executors/executor_scheduler.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-19398495-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2023/02/06 21:51:51                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.1                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.7.15                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/t… ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/tutorials/multi_executors/executor_scheduler.yml: VALID
Total builder objects created: 2
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ executo… │ script │ generic… │ None     │ None  │ None  │ Declari… │ /home/… │
│          │        │          │          │       │       │ env and  │         │
│          │        │          │          │       │       │ vars by  │         │
│          │        │          │          │       │       │ executo… │         │
│          │        │          │          │       │       │ section  │         │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ executo… │ script │ generic… │ None     │ None  │ None  │ Declari… │ /home/… │
│          │        │          │          │       │       │ env and  │         │
│          │        │          │          │       │       │ vars by  │         │
│          │        │          │          │       │       │ executo… │         │
│          │        │          │          │       │       │ section  │         │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
executors_sbatch_declaration/7661f515: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515
executors_sbatch_declaration/7661f515: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/stage
executors_sbatch_declaration/7661f515: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration_build.sh
executors_sbatch_declaration/994046f8: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8
executors_sbatch_declaration/994046f8: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/stage
executors_sbatch_declaration/994046f8: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
executors_sbatch_declaration/994046f8 does not have any dependencies adding test to queue
executors_sbatch_declaration/7661f515 does not have any dependencies adding test to queue
executors_sbatch_declaration/994046f8: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/stage
executors_sbatch_declaration/994046f8: Running Test via command: sh --norc --noprofile -eo pipefail executors_sbatch_declaration_build.sh
executors_sbatch_declaration/994046f8: Test completed in 0.003131 seconds
executors_sbatch_declaration/994046f8: Test completed with returncode: 2
executors_sbatch_declaration/994046f8: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration.out
executors_sbatch_declaration/994046f8: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration.err
executors_sbatch_declaration/7661f515: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/stage
executors_sbatch_declaration/7661f515: Running Test via command: bash --norc --noprofile -eo pipefail executors_sbatch_declaration_build.sh
executors_sbatch_declaration/7661f515: Test completed in 0.005908 seconds
executors_sbatch_declaration/7661f515: Test completed with returncode: 0
executors_sbatch_declaration/7661f515: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration.out
executors_sbatch_declaration/7661f515: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration.err
In this iteration we are going to run the following tests: [executors_sbatch_declaration/994046f8, executors_sbatch_declaration/7661f515]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ builder      ┃ executor     ┃ status ┃ Runtime)      ┃ returncode ┃ runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ executors_sb │ generic.loc… │ FAIL   │ N/A N/A N/A   │ 2          │ 0.003131 │
│ atch_declara │              │        │               │            │          │
│ tion/994046f │              │        │               │            │          │
│ 8            │              │        │               │            │          │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ executors_sb │ generic.loc… │ PASS   │ N/A N/A N/A   │ 0          │ 0.005908 │
│ atch_declara │              │        │               │            │          │
│ tion/7661f51 │              │        │               │            │          │
│ 5            │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/report.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_h7vy8ggk.log

If we inspect this test, we will see each each test have different #SBATCH directives for each test based on the sbatch property defined in the executors field.

buildtest inspect query -t executors_sbatch_declaration/
$ buildtest inspect query -t executors_sbatch_declaration/
────── executors_sbatch_declaration/7661f515-3de2-4032-a0ff-7621ddb019d9 ───────
Executor: generic.local.bash
Description: Declaring env and vars by executors section
State: PASS
Returncode: 0
Runtime: 0.005908 sec
Starttime: 2023/02/06 21:51:51
Endtime: 2023/02/06 21:51:51
Command: bash --norc --noprofile -eo pipefail executors_sbatch_declaration_build.sh
Test Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration.sh
Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration_build.sh
Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration.out
Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/executor_scheduler/executors_sbatch_declaration/7661f515/executors_sbatch_declaration.err
Log File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_h7vy8ggk.log
─ Test File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/check… ─
#!/usr/bin/bash                                                                 
#SBATCH -n 4                                                                    
#SBATCH -N 1                                                                    
#SBATCH -t 30                                                                   
#SBATCH --job-name=executors_sbatch_declaration                                 
#SBATCH --output=executors_sbatch_declaration.out                               
#SBATCH --error=executors_sbatch_declaration.err                                
# Content of run section                                                        
hostname                                                                        
────── executors_sbatch_declaration/994046f8-0dc3-4dbb-834c-90bc973a8a1d ───────
Executor: generic.local.sh
Description: Declaring env and vars by executors section
State: FAIL
Returncode: 2
Runtime: 0.003131 sec
Starttime: 2023/02/06 21:51:51
Endtime: 2023/02/06 21:51:51
Command: sh --norc --noprofile -eo pipefail executors_sbatch_declaration_build.sh
Test Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration.sh
Build Script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration_build.sh
Output File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration.out
Error File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/executor_scheduler/executors_sbatch_declaration/994046f8/executors_sbatch_declaration.err
Log File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest_h7vy8ggk.log
─ Test File: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/check… ─
#!/usr/bin/sh                                                                   
#SBATCH -n 8                                                                    
#SBATCH -N 1                                                                    
#SBATCH -t 60                                                                   
#SBATCH --job-name=executors_sbatch_declaration                                 
#SBATCH --output=executors_sbatch_declaration.out                               
#SBATCH --error=executors_sbatch_declaration.err                                
# Content of run section                                                        
hostname

Cray Burst Buffer and Data Warp

You can also define BB and DW directives in the executors field to override cray burst buffer and data warp settings per executor. buildtest will use the fields BB and DW and insert the #BB and #DW directives in the job script. For more details see Cray Burst Buffer & Data Warp.

buildspecs:
  create_burst_buffer_executors:
    type: script
    executor: "generic.local.(sh|bash)"
    sbatch: ["-N 1", "-t 10", "-C knl"]
    description: Create a burst buffer for multiple executors
    tags: [jobs]
    executors:
      generic.local.sh:
        BB:
          - create_persistent name=buffer1 capacity=10GB access_mode=striped type=scratch
        DW:
          - persistentdw name=buffer1
      generic.local.bash:
        BB:
        - create_persistent name=buffer2 capacity=10GB access_mode=striped type=scratch
        DW:
          - persistentdw name=buffer2
    run: hostname

Custom Status by Executor

The status and metrics field are supported in executors which can be defined within the named executor. In this next example, we will define executor generic.local.bash to match for returncode 0 or 2 while second test will use executor generic.local.sh to match returncode of 1.

buildspecs:
  status_returncode_by_executors:
    type: script
    executor: "generic.local.(bash|sh)"
    description: define status per executor type.
    tags: [tutorials]
    run: exit 0
    executors:
      generic.local.bash:
        status:
          returncode: [0, 2]
      generic.local.sh:
        status:
          returncode: 1

Now let’s run this test and we will see the test using executor generic.local.sh will fail because we have a returncode mismatch even though both tests got a 0 returncode as its actual value.

buildtest build -b tutorials/multi_executors/status_by_executors.yml
$ buildtest build -b tutorials/multi_executors/status_by_executors.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│                                                                              │
│ User:               docs                                                     │
│ Hostname:           build-19398495-project-280831-buildtest                  │
│ Platform:           Linux                                                    │
│ Current Time:       2023/02/06 21:51:52                                      │
│ buildtest path:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version:  1.1                                                      │
│ python path:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version:     3.7.15                                                   │
│ Configuration File: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Test Directory:     /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Report File:        /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ Command:            /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
───────────────────────────  Discovering Buildspecs ────────────────────────────
                             Discovered buildspecs                              
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec                                                                    ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/t… ║
╚══════════════════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/tutorials/multi_executors/status_by_executors.yml: VALID
Total builder objects created: 2
                            Builders by type=script                             
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ builder  ┃ type   ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ descrip… ┃ builds… ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ status_… │ script │ generic… │ None     │ None  │ None  │ define   │ /home/… │
│          │        │          │          │       │       │ status   │         │
│          │        │          │          │       │       │ per      │         │
│          │        │          │          │       │       │ executor │         │
│          │        │          │          │       │       │ type.    │         │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ status_… │ script │ generic… │ None     │ None  │ None  │ define   │ /home/… │
│          │        │          │          │       │       │ status   │         │
│          │        │          │          │       │       │ per      │         │
│          │        │          │          │       │       │ executor │         │
│          │        │          │          │       │       │ type.    │         │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
status_returncode_by_executors/705d7b80: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80
status_returncode_by_executors/705d7b80: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80/stage
status_returncode_by_executors/705d7b80: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80/status_returncode_by_executors_build.sh
status_returncode_by_executors/090067ee: Creating test directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee
status_returncode_by_executors/090067ee: Creating the stage directory: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee/stage
status_returncode_by_executors/090067ee: Writing build script: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee/status_returncode_by_executors_build.sh
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
status_returncode_by_executors/090067ee does not have any dependencies adding test to queue
status_returncode_by_executors/705d7b80 does not have any dependencies adding test to queue
status_returncode_by_executors/090067ee: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee/stage
status_returncode_by_executors/090067ee: Running Test via command: sh --norc --noprofile -eo pipefail status_returncode_by_executors_build.sh
status_returncode_by_executors/090067ee: Test completed in 0.003222 seconds
status_returncode_by_executors/090067ee: Test completed with returncode: 2
status_returncode_by_executors/090067ee: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee/status_returncode_by_executors.out
status_returncode_by_executors/090067ee: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.sh/status_by_executors/status_returncode_by_executors/090067ee/status_returncode_by_executors.err
status_returncode_by_executors/090067ee: Checking returncode - 2 is matched in list [1]
status_returncode_by_executors/705d7b80: Current Working Directory : /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80/stage
status_returncode_by_executors/705d7b80: Running Test via command: bash --norc --noprofile -eo pipefail status_returncode_by_executors_build.sh
status_returncode_by_executors/705d7b80: Test completed in 0.005446 seconds
status_returncode_by_executors/705d7b80: Test completed with returncode: 0
status_returncode_by_executors/705d7b80: Writing output file -  /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80/status_returncode_by_executors.out
status_returncode_by_executors/705d7b80: Writing error file - /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/tests/generic.local.bash/status_by_executors/status_returncode_by_executors/705d7b80/status_returncode_by_executors.err
status_returncode_by_executors/705d7b80: Checking returncode - 0 is matched in list [0, 2]
In this iteration we are going to run the following tests: [status_returncode_by_executors/090067ee, status_returncode_by_executors/705d7b80]
                                  Test Summary                                  
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓
┃              ┃              ┃        ┃ checks        ┃            ┃          ┃
┃              ┃              ┃        ┃ (ReturnCode,  ┃            ┃          ┃
┃              ┃              ┃        ┃ Regex,        ┃            ┃          ┃
┃ builder      ┃ executor     ┃ status ┃ Runtime)      ┃ returncode ┃ runtime  ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩
│ status_retur │ generic.loc… │ PASS   │ True False    │ 0          │ 0.005446 │
│ ncode_by_exe │              │        │ False         │            │          │
│ cutors/705d7 │              │        │               │            │          │
│ b80          │              │        │               │            │          │
├──────────────┼──────────────┼────────┼───────────────┼────────────┼──────────┤
│ status_retur │ generic.loc… │ FAIL   │ False False   │ 2          │ 0.003222 │
│ ncode_by_exe │              │        │ False         │            │          │
│ cutors/09006 │              │        │               │            │          │
│ 7ee          │              │        │               │            │          │
└──────────────┴──────────────┴────────┴───────────────┴────────────┴──────────┘



Passed Tests: 1/2 Percentage: 50.000%
Failed Tests: 1/2 Percentage: 50.000%


Adding 2 test results to /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/report.json
Writing Logfile to: /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/v1.2/var/logs/buildtest__ze4gsn6.log