Buildspecs Interface¶
Now that we learned how to build tests, in this section we will discuss how one can query a buildspec cache. In buildtest, one can load all buildspecs which is equivalent to validating all buildspecs with the appropriate schema. Buildtest will ignore all invalid buildspecs and store them in a separate file.
The buildtest buildspec find
command is used for finding buildspecs from buildspec
cache. This command is also used for generating the buildspec cache. Shown below is a list of options for
buildtest buildspec find
.
$ buildtest buildspec find --help
usage: buildtest [options] [COMMANDS] buildspec find [-h] [--root ROOT] [-r] [-t] [-b] [-e] [-p] [--group-by-tags]
[--group-by-executor] [-m] [-mb] [--filter FILTER]
[--format FORMAT] [--helpfilter] [--helpformat]
optional arguments:
-h, --help show this help message and exit
--root ROOT Specify root buildspecs (directory) path to load buildspecs into buildspec cache.
-r, --rebuild Rebuild buildspec cache and find all buildspecs again
-t, --tags List all available tags
-b, --buildspec Get all buildspec files from cache
-e, --executors get all unique executors from buildspecs
-p, --paths print all root buildspec paths
--group-by-tags Group tests by tag name
--group-by-executor Group tests by executor name
-m, --maintainers Get all maintainers for all buildspecs
-mb, --maintainers-by-buildspecs
Show maintainers breakdown by buildspecs
--filter FILTER Filter buildspec cache with filter fields in format --filter key1=val1,key2=val2
--format FORMAT Format buildspec cache with format fields in format --format field1,field2,...
--helpfilter Show Filter fields for --filter option for filtering buildspec cache output
--helpformat Show Format fields for --format option for formatting buildspec cache output
Finding Buildspecs¶
To find all buildspecs run buildtest buildspec find
which will discover
all buildspecs in all repos by recursively finding all .yml extensions.
$ buildtest buildspec find
Buildspec Paths: ['/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials', '/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests']
Discovered Buildspecs
________________________________________________________________________________
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_buildspec_section.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/executor_regex_script.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/run_only_distro.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shell_examples.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/environment.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/python-hello.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/vars.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/selinux.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shebang.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/csh_shell_examples.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_tags.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/hello_world.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_executor.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/root_user.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/tags_example.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/run_only_platform.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/python-shell.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/skip_tests.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/sleep.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/openmp_hello.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/vecadd.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/envvar_override.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_exclude.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_c.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/pre_post_build_run.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/custom_run.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/gnu_hello_fortran.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/compilers/compiler_status_regex.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/disk_usage.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/kernel_state.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/ssh_localhost.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/systemd-default-target.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/ulimits.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/slurm/sacctmgr.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/slurm/scontrol.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/slurm/squeue.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/slurm/sinfo.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/cobalt/commands.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/lsf/bmgroups.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/lsf/lsinfo.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/lsf/bugroup.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/lsf/bqueues.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/lsf/bhosts.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/pbs/batch.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/pbs/hostname.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/containers/singularity/pull.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/containers/singularity/run.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/containers/singularity/inspect.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/containers/singularity/build.yml
Found 51 buildspecs
________________________________________________________________________________
Validated 5/51 buildspecs
Validated 10/51 buildspecs
Validated 15/51 buildspecs
Validated 20/51 buildspecs
Validated 25/51 buildspecs
Validated 30/51 buildspecs
Validated 35/51 buildspecs
Validated 40/51 buildspecs
Validated 45/51 buildspecs
Validated 50/51 buildspecs
Validated 51/51 buildspecs
Invalid buildspecs
________________________________________________________________________________
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_buildspec_section.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_tags.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/invalid_executor.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/pbs/batch.yml
/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/sched/pbs/hostname.yml
Found 5 invalid buildspecs
________________________________________________________________________________
Adding 46 buildspec files to cache
There are 100 tests in buildspec cache
Updating buildspec cache file: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/buildspecs/cache.json
Writing invalid buildspecs to file: /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/buildspecs/error.txt
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| name | type | executor | tags | description |
+==============================+==========+=========================+===============================+================================================================================+
| executor_regex_script_schema | script | generic.local.(bash|sh) | ['tutorials'] | regular expression test with executor using script schema |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_macos_distro | script | generic.local.bash | ['mac'] | Run test only if distro is darwin. |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_linux_distro | script | generic.local.bash | ['mac'] | Run test only if distro is CentOS. |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| _bin_sh_shell | script | generic.local.sh | ['tutorials'] | /bin/sh shell example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| _bin_bash_shell | script | generic.local.bash | ['tutorials'] | /bin/bash shell example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bash_shell | script | generic.local.bash | ['tutorials'] | bash shell example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| sh_shell | script | generic.local.sh | ['tutorials'] | sh shell example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| shell_options | script | generic.local.sh | ['tutorials'] | shell options |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bash_env_variables | script | generic.local.bash | ['tutorials'] | Declare environment variables in default shell (bash) |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| csh_env_declaration | script | generic.local.csh | ['tutorials'] | csh shell example to declare environment variables |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| tcsh_env_declaration | script | generic.local.csh | ['tutorials'] | tcsh shell example to declare environment variables |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| python_hello | script | generic.local.bash | python | Hello World python |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| variables_bash | script | generic.local.bash | ['tutorials'] | Declare shell variables in bash |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| selinux_disable | script | generic.local.bash | ['tutorials'] | Check if SELinux is Disabled |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bash_login_shebang | script | generic.local.bash | tutorials | customize shebang line with bash login shell |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bash_nonlogin_shebang | script | generic.local.bash | tutorials | customize shebang line with default bash (nonlogin) shell |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| exit1_fail | script | generic.local.sh | ['tutorials', 'fail'] | exit 1 by default is FAIL |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| exit1_pass | script | generic.local.sh | ['tutorials', 'pass'] | report exit 1 as PASS |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| returncode_list_mismatch | script | generic.local.sh | ['tutorials', 'fail'] | exit 2 failed since it failed to match returncode 1 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| returncode_int_match | script | generic.local.sh | ['tutorials', 'pass'] | exit 128 matches returncode 128 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| csh_shell | script | generic.local.csh | ['tutorials'] | csh shell example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| hello_world | script | generic.local.bash | tutorials | hello world example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_as_root | script | generic.local.bash | ['tutorials'] | This test will only run if current user is root |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| string_tag | script | generic.local.bash | network | tags can be a string |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| list_of_strings_tags | script | generic.local.bash | ['network', 'ping'] | tags can be a list of strings |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_platform_darwin | script | generic.local.python | ['tutorials'] | This test will only run if target platform is Darwin |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_platform_linux | script | generic.local.python | ['tutorials'] | This test will only run if target platform is Linux |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| circle_area | script | generic.local.python | ['tutorials', 'python'] | Calculate circle of area given a radius |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| skip | script | generic.local.bash | ['tutorials'] | This test is skipped |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| unskipped | script | generic.local.bash | ['tutorials'] | This test is not skipped |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| timelimit_min_max | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if its within 1.0-4.0sec |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| timelimit_min | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if its exceeds min time of 1.0 sec |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| timelimit_max | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if it's within max time: 5.0 sec |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| timelimit_min_fail | script | generic.local.sh | ['tutorials'] | This test fails because it runs less than mintime of 1.0 second |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| timelimit_max_fail | script | generic.local.sh | ['tutorials'] | This test fails because it exceeds maxtime of 1.0 second |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| sleep | script | generic.local.bash | ['tutorials'] | sleep 2 seconds |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| openmp_hello_c_example | compiler | generic.local.bash | ['tutorials', 'compile'] | OpenMP Hello World C example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| vecadd_gnu | compiler | generic.local.bash | ['tutorials', 'compile'] | Vector Addition example with GNU compiler |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| override_environmentvars | compiler | generic.local.bash | ['tutorials', 'compile'] | override default environment variables |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| vecadd_gnu_exclude | compiler | generic.local.bash | ['tutorials', 'compile'] | Vector Addition example with GNU compilers but exclude gcc@10.2.0 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| hello_c | compiler | generic.local.bash | ['tutorials', 'compile'] | Hello World C Compilation |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| pre_post_build_run | compiler | generic.local.bash | ['tutorials', 'compile'] | example using pre_build, post_build, pre_run, post_run example |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| custom_run_by_compilers | compiler | generic.local.bash | ['tutorials', 'compile'] | Customize binary launch based on compiler |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| hello_f | compiler | generic.local.bash | ['tutorials', 'compile'] | Hello World Fortran Compilation |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| default_status_regex | compiler | generic.local.bash | ['tutorials', 'compile'] | Regular expression check in stdout for gcc group |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| override_status_regex | compiler | generic.local.bash | ['tutorials', 'compile'] | Override regular expression for compiler gcc/10.2.0-37fmsw7 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| root_disk_usage | script | generic.local.bash | ['filesystem', 'storage'] | Check root disk usage and report if it exceeds threshold |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| kernel_swapusage | script | generic.local.bash | ['configuration'] | Retrieve Kernel Swap Usage |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ssh_localhost_remotecommand | script | generic.local.bash | ['ssh'] | Test if ssh on localhost works and if we can run remote command. |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| systemd_default_target | script | generic.local.bash | ['system'] | check if default target is multi-user.target |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_filelock_unlimited | script | generic.local.bash | ['system'] | Check if file lock is set to unlimited in ulimits |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_cputime_unlimited | script | generic.local.bash | ['system'] | Check if cputime is set to unlimited in ulimits |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_stacksize_unlimited | script | generic.local.bash | ['system'] | Check if stack size is set to unlimited in ulimits |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_vmsize_unlimited | script | generic.local.bash | ['system'] | Check virtual memory size and check if its set to unlimited |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_filedescriptor_4096 | script | generic.local.bash | ['system'] | Check if open file descriptors limit is set to 4096 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| ulimit_max_user_process_2048 | script | generic.local.bash | ['system'] | Check max number of user process limit is set to 2048 |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_accounts | script | generic.local.bash | ['slurm'] | run sacctmgr list accounts |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_users | script | generic.local.bash | ['slurm'] | run sacctmgr list users |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_qos | script | generic.local.bash | ['slurm'] | run sacctmgr list qos |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_tres | script | generic.local.bash | ['slurm'] | run sacctmgr list tres |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| slurm_config | script | generic.local.bash | ['slurm'] | run scontrol show config |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_partition | script | generic.local.bash | ['slurm'] | run scontrol show partition |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| current_user_queue | script | generic.local.bash | ['slurm'] | show all current pending jobs for current user (squeue -u $USER) |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_all_jobs | script | generic.local.bash | ['slurm'] | show all pending + running jobs (squeue -a) |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| nodes_state_down | script | generic.local.bash | ['slurm'] | Show nodes in DOWN state |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| nodes_state_reboot | script | generic.local.bash | ['slurm'] | Show nodes in REBOOT state |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| nodes_state_allocated | script | generic.local.bash | ['slurm'] | Show nodes in ALLOCATED state |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| nodes_state_completing | script | generic.local.bash | ['slurm'] | Show nodes in COMPLETING state |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| nodes_state_idle | script | generic.local.bash | ['slurm'] | Show nodes in IDLE state |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| node_down_fail_list_reason | script | generic.local.bash | ['slurm'] | Show nodes DOWN, DRAINED, FAIL or FAILING and list reason |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| dead_nodes | script | generic.local.bash | ['slurm'] | Show non-responding nodes |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| get_partitions | script | generic.local.bash | ['slurm'] | Get all slurm partitions |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| sinfo_version | script | generic.local.bash | ['slurm'] | get slurm version |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| qsub_version | script | generic.local.sh | ['cobalt'] | print version for qsub command |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| qselect_version | script | generic.local.sh | ['cobalt'] | print version for qselect |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| cqsub_version | script | generic.local.sh | ['cobalt'] | print version for cqsub command |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| qdel_version | script | generic.local.sh | ['cobalt'] | print version for qdel command |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| qmove_version | script | generic.local.sh | ['cobalt'] | print version for qmove command |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_jobs | script | generic.local.sh | ['cobalt'] | Show all jobs in queue |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_queues | script | generic.local.sh | ['cobalt'] | Show all queues |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_host_groups | script | generic.local.bash | lsf | Show information about host groups using bmgroup |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_configuration | script | generic.local.bash | lsf | Show LSF configuration using lsinfo |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_models | script | generic.local.bash | lsf | Show information about host models in LSF cluster |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_resources | script | generic.local.bash | lsf | Show information about LSF resources |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| lsf_version | script | generic.local.bash | lsf | Display lsf version using lsinfo |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_user_groups | script | generic.local.bash | lsf | Show information about all LSF user groups |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_queues | script | generic.local.bash | lsf | Show LSF queues |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_queues_formatted | script | generic.local.bash | lsf | Show LSF queues with formatted columns |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| show_lsf_queues_current_user | script | generic.local.bash | lsf | Show LSF queues available for current user |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| display_lsf_hosts | script | generic.local.bash | lsf | Show all hosts in LSF cluster |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| display_hosts_format | script | generic.local.bash | lsf | Show all hosts with column hostname and status |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bhosts_version | script | generic.local.bash | lsf | display version from bhosts command |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| pullImage_dockerhub | script | generic.local.bash | ['containers', 'singularity'] | Pull image docker://godlovedc/lolcow from DockerHub |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| pullImage_sylabscloud | script | generic.local.bash | ['containers', 'singularity'] | Pull image library://alpine:latest from Sylabs Cloud |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| pullImage_shub | script | generic.local.bash | ['containers', 'singularity'] | Pull image shub://vsoch/singularity-images from SingularityHub |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| runImage | script | generic.local.bash | ['containers', 'singularity'] | run container docker://godlovedc/lolcow |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| inspect_image | script | generic.local.bash | ['containers', 'singularity'] | Inspect image via 'singularity inspect' |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| build_sif_from_dockerimage | script | generic.local.bash | ['containers', 'singularity'] | build sif image from docker image docker://godlovedc/lolcow |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| build_sandbox_image | script | generic.local.bash | ['containers', 'singularity'] | build sandbox image from docker image docker://godlovedc/lolcow |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| build_remoteimages | script | generic.local.bash | ['containers', 'singularity'] | build remote hosted image from AWS |
+------------------------------+----------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
buildtest will validate each buildspec file with the appropriate schema type. buildspecs that pass validation will be displayed on screen. buildtest will report all invalid buildspecs in a text file for you to review.
buildtest will cache the results in var/buildspec-cache.json so subsequent
runs to buildtest buildspec find
will be much faster because it is read from cache.
If you make changes to buildspec you may want to rebuild the buildspec cache then
run:
$ buildtest buildspec find --rebuild
If you want to find all buildspec files in cache run buildtest buildspec find --buildspec
$ buildtest buildspec find --buildspec
+----------------------------------------------------------------------------------------------------------+
| buildspecs |
+==========================================================================================================+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/executor_regex_script.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/run_only_distro.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shell_examples.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/environment.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/python-hello.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/vars.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/selinux.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shebang.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/csh_shell_examples.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/hello_world.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/root_user.yml |
+----------------------------------------------------------------------------------------------------------+
| /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/tags_example.yml |
+----------------------------------------------------------------------------------------------------------+
...
If you want to find root directories of buildspecs loaded in buildspec cache use the
buildtest buildspec find --paths
option.
$ buildtest buildspec find --paths
/Users/siddiq90/Documents/buildtest/tutorials
/Users/siddiq90/Documents/buildtest/general_tests
buildtest will search buildspecs in buildspecs root defined in your configuration,
which is a list of directory paths to search for buildspecs.
If you want to load buildspecs from a directory path, one can run specify a directory
path via --root
such as buildtest buildspec find --root <path> --rebuild
.
buildtest will load all valid buildspecs into cache and ignore
the rest. It’s important to add --rebuild
if you want to regenerate buildspec cache.
Filtering buildspec¶
Once you have a buildspec cache, we can query the buildspec cache for certain attributes.
When you run buildtest buildspec find it will report all buildspecs from cache which can
be difficult to process. Therefore, we have a filter option (--filter
) to restrict our search.
Let’s take a look at the available filter fields that are acceptable with filter option.
$ buildtest buildspec find --helpfilter
Field Description Type
-------- ----------------------- ------
executor Filter by executor name STRING
tags Filter by tag name STRING
type Filter by schema type STRING
The --filter
option expects arguments in key=value format as follows:
buildtest buildspec find --filter key1=value1,key2=value2,key3=value3
We can filter buildspec cache by tags=fail
which will query all tests with
associated tag field in test.
$ buildtest buildspec find --filter tags=fail
+--------------------------+--------+------------------+-----------------------+-----------------------------------------------------+
| name | type | executor | tags | description |
+==========================+========+==================+=======================+=====================================================+
| exit1_fail | script | generic.local.sh | ['tutorials', 'fail'] | exit 1 by default is FAIL |
+--------------------------+--------+------------------+-----------------------+-----------------------------------------------------+
| returncode_list_mismatch | script | generic.local.sh | ['tutorials', 'fail'] | exit 2 failed since it failed to match returncode 1 |
+--------------------------+--------+------------------+-----------------------+-----------------------------------------------------+
In addition, we can query buildspecs by schema type using type
property. In this
example we query all tests by type property
$ buildtest buildspec find --filter type=script
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| name | type | executor | tags | description |
+==============================+========+=========================+===============================+================================================================================+
| executor_regex_script_schema | script | generic.local.(bash|sh) | ['tutorials'] | regular expression test with executor using script schema |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_macos_distro | script | generic.local.bash | ['mac'] | Run test only if distro is darwin. |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| run_only_linux_distro | script | generic.local.bash | ['mac'] | Run test only if distro is CentOS. |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| _bin_sh_shell | script | generic.local.sh | ['tutorials'] | /bin/sh shell example |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| _bin_bash_shell | script | generic.local.bash | ['tutorials'] | /bin/bash shell example |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| bash_shell | script | generic.local.bash | ['tutorials'] | bash shell example |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| sh_shell | script | generic.local.sh | ['tutorials'] | sh shell example |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
| shell_options | script | generic.local.sh | ['tutorials'] | shell options |
+------------------------------+--------+-------------------------+-------------------------------+--------------------------------------------------------------------------------+
...
Finally, we can combine multiple filter fields separated by comma, in the next example
we query all buildspecs with tags=tutorials
, executor=generic.local.sh
, and type=script
$ buildtest buildspec find --filter tags=tutorials,executor=generic.local.sh,type=script
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| name | type | executor | tags | description |
+==========================+========+==================+=======================+================================================================================+
| _bin_sh_shell | script | generic.local.sh | ['tutorials'] | /bin/sh shell example |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| sh_shell | script | generic.local.sh | ['tutorials'] | sh shell example |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| shell_options | script | generic.local.sh | ['tutorials'] | shell options |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| exit1_fail | script | generic.local.sh | ['tutorials', 'fail'] | exit 1 by default is FAIL |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| exit1_pass | script | generic.local.sh | ['tutorials', 'pass'] | report exit 1 as PASS |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| returncode_list_mismatch | script | generic.local.sh | ['tutorials', 'fail'] | exit 2 failed since it failed to match returncode 1 |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| returncode_int_match | script | generic.local.sh | ['tutorials', 'pass'] | exit 128 matches returncode 128 |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| timelimit_min_max | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if its within 1.0-4.0sec |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| timelimit_min | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if its exceeds min time of 1.0 sec |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| timelimit_max | script | generic.local.sh | ['tutorials'] | Run a sleep job for 2 seconds and test pass if it's within max time: 5.0 sec |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| timelimit_min_fail | script | generic.local.sh | ['tutorials'] | This test fails because it runs less than mintime of 1.0 second |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
| timelimit_max_fail | script | generic.local.sh | ['tutorials'] | This test fails because it exceeds maxtime of 1.0 second |
+--------------------------+--------+------------------+-----------------------+--------------------------------------------------------------------------------+
Format buildspec cache¶
We have seen how one can filter buildspecs, but we can also configure which columns to display in the output of buildtest buildspec find. By default, we show few format fields in the output, however there are more format fields hidden from the default output.
The format fields are specified comma separated using format: --format <field1>,<field2>,...
.
You can see a list of all format fields by --helpformat
option as shown below
$ buildtest buildspec find --helpformat
Field Description
----------- -----------------------
description Format by description
executor Format by executor type
file Format by file
name Format by test name
tags Format by tag name
type Format by schema type
In the next example, we utilize --format
field with --filter
option to show
how format fields affect table columns. buildtest will display the table in order of
format fields specified in command line.
$ buildtest buildspec find --format name,description,file --filter tags=tutorials,executor=generic.local.sh
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| name | description | file |
+==========================+================================================================================+=====================================================================================+
| _bin_sh_shell | /bin/sh shell example | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shell_examples.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| sh_shell | sh shell example | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shell_examples.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| shell_options | shell options | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/shell_examples.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| exit1_fail | exit 1 by default is FAIL | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| exit1_pass | report exit 1 as PASS | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| returncode_list_mismatch | exit 2 failed since it failed to match returncode 1 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| returncode_int_match | exit 128 matches returncode 128 | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/pass_returncode.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| timelimit_min_max | Run a sleep job for 2 seconds and test pass if its within 1.0-4.0sec | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| timelimit_min | Run a sleep job for 2 seconds and test pass if its exceeds min time of 1.0 sec | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| timelimit_max | Run a sleep job for 2 seconds and test pass if it's within max time: 5.0 sec | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| timelimit_min_fail | This test fails because it runs less than mintime of 1.0 second | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
| timelimit_max_fail | This test fails because it exceeds maxtime of 1.0 second | /Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/runtime_status_test.yml |
+--------------------------+--------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+
buildtest makes use of python library named tabulate
to generate these tables which are found in commands line like buildtest buildspec find
and buildtest report
.
Querying buildspec executor¶
If you want to know all executors in your buildspec cache use the
buildtest buildspec find --list-executors
command. This can be useful when
you want to build by executors (buildtest build --executor
).
$ buildtest buildspec find --executors
+-------------------------+
| executors |
+=========================+
| generic.local.python |
+-------------------------+
| generic.local.sh |
+-------------------------+
| generic.local.(bash|sh) |
+-------------------------+
| generic.local.csh |
+-------------------------+
| generic.local.bash |
+-------------------------+
Similar to --group-by-tags
, buildtest has an option to group tests by executor
using --group-by-executor
option. This will show tests grouped by executor,
name of test and test description. Shown below is an example output.
$ buildtest buildspec find --group-by-executor
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| executor | name | description |
+=========================+==============================+================================================================================+
| generic.local.(bash|sh) | executor_regex_script_schema | regular expression test with executor using script schema |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | run_only_macos_distro | Run test only if distro is darwin. |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | run_only_linux_distro | Run test only if distro is CentOS. |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | _bin_bash_shell | /bin/bash shell example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | bash_shell | bash shell example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | bash_env_variables | Declare environment variables in default shell (bash) |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | python_hello | Hello World python |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | variables_bash | Declare shell variables in bash |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | selinux_disable | Check if SELinux is Disabled |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | bash_login_shebang | customize shebang line with bash login shell |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | bash_nonlogin_shebang | customize shebang line with default bash (nonlogin) shell |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | hello_world | hello world example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | run_only_as_root | This test will only run if current user is root |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | string_tag | tags can be a string |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | list_of_strings_tags | tags can be a list of strings |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | skip | This test is skipped |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | unskipped | This test is not skipped |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | sleep | sleep 2 seconds |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | openmp_hello_c_example | OpenMP Hello World C example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | vecadd_gnu | Vector Addition example with GNU compiler |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | override_environmentvars | override default environment variables |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | vecadd_gnu_exclude | Vector Addition example with GNU compilers but exclude gcc@10.2.0 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | hello_c | Hello World C Compilation |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | pre_post_build_run | example using pre_build, post_build, pre_run, post_run example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | custom_run_by_compilers | Customize binary launch based on compiler |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | hello_f | Hello World Fortran Compilation |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | default_status_regex | Regular expression check in stdout for gcc group |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | override_status_regex | Override regular expression for compiler gcc/10.2.0-37fmsw7 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | root_disk_usage | Check root disk usage and report if it exceeds threshold |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | kernel_swapusage | Retrieve Kernel Swap Usage |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ssh_localhost_remotecommand | Test if ssh on localhost works and if we can run remote command. |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | systemd_default_target | check if default target is multi-user.target |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_filelock_unlimited | Check if file lock is set to unlimited in ulimits |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_cputime_unlimited | Check if cputime is set to unlimited in ulimits |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_stacksize_unlimited | Check if stack size is set to unlimited in ulimits |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_vmsize_unlimited | Check virtual memory size and check if its set to unlimited |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_filedescriptor_4096 | Check if open file descriptors limit is set to 4096 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | ulimit_max_user_process_2048 | Check max number of user process limit is set to 2048 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_accounts | run sacctmgr list accounts |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_users | run sacctmgr list users |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_qos | run sacctmgr list qos |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_tres | run sacctmgr list tres |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | slurm_config | run scontrol show config |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_partition | run scontrol show partition |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | current_user_queue | show all current pending jobs for current user (squeue -u $USER) |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_all_jobs | show all pending + running jobs (squeue -a) |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | nodes_state_down | Show nodes in DOWN state |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | nodes_state_reboot | Show nodes in REBOOT state |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | nodes_state_allocated | Show nodes in ALLOCATED state |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | nodes_state_completing | Show nodes in COMPLETING state |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | nodes_state_idle | Show nodes in IDLE state |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | node_down_fail_list_reason | Show nodes DOWN, DRAINED, FAIL or FAILING and list reason |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | dead_nodes | Show non-responding nodes |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | get_partitions | Get all slurm partitions |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | sinfo_version | get slurm version |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_host_groups | Show information about host groups using bmgroup |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_configuration | Show LSF configuration using lsinfo |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_models | Show information about host models in LSF cluster |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_resources | Show information about LSF resources |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | lsf_version | Display lsf version using lsinfo |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_user_groups | Show information about all LSF user groups |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_queues | Show LSF queues |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_queues_formatted | Show LSF queues with formatted columns |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | show_lsf_queues_current_user | Show LSF queues available for current user |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | display_lsf_hosts | Show all hosts in LSF cluster |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | display_hosts_format | Show all hosts with column hostname and status |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | bhosts_version | display version from bhosts command |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | pullImage_dockerhub | Pull image docker://godlovedc/lolcow from DockerHub |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | pullImage_sylabscloud | Pull image library://alpine:latest from Sylabs Cloud |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | pullImage_shub | Pull image shub://vsoch/singularity-images from SingularityHub |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | runImage | run container docker://godlovedc/lolcow |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | inspect_image | Inspect image via 'singularity inspect' |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | build_sif_from_dockerimage | build sif image from docker image docker://godlovedc/lolcow |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | build_sandbox_image | build sandbox image from docker image docker://godlovedc/lolcow |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.bash | build_remoteimages | build remote hosted image from AWS |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | _bin_sh_shell | /bin/sh shell example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | sh_shell | sh shell example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | shell_options | shell options |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | exit1_fail | exit 1 by default is FAIL |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | exit1_pass | report exit 1 as PASS |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | returncode_list_mismatch | exit 2 failed since it failed to match returncode 1 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | returncode_int_match | exit 128 matches returncode 128 |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | timelimit_min_max | Run a sleep job for 2 seconds and test pass if its within 1.0-4.0sec |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | timelimit_min | Run a sleep job for 2 seconds and test pass if its exceeds min time of 1.0 sec |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | timelimit_max | Run a sleep job for 2 seconds and test pass if it's within max time: 5.0 sec |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | timelimit_min_fail | This test fails because it runs less than mintime of 1.0 second |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | timelimit_max_fail | This test fails because it exceeds maxtime of 1.0 second |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | qsub_version | print version for qsub command |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | qselect_version | print version for qselect |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | cqsub_version | print version for cqsub command |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | qdel_version | print version for qdel command |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | qmove_version | print version for qmove command |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | show_jobs | Show all jobs in queue |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.sh | show_queues | Show all queues |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.csh | csh_env_declaration | csh shell example to declare environment variables |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.csh | tcsh_env_declaration | tcsh shell example to declare environment variables |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.csh | csh_shell | csh shell example |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.python | run_only_platform_darwin | This test will only run if target platform is Darwin |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.python | run_only_platform_linux | This test will only run if target platform is Linux |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
| generic.local.python | circle_area | Calculate circle of area given a radius |
+-------------------------+------------------------------+--------------------------------------------------------------------------------+
Query Maintainers in buildspecs¶
The maintainers
field can be used for identifying author for buildspec
file which can be useful if you want to find out who is responsible for the test.
You can retrieve all buildspec maintainers using --maintainers
option or -m
short option. The command below will show all maintainers for buildspecs in buildspec
cache
$ buildtest buildspec find --maintainers
+------------------+
| maintainers |
+==================+
| @shahzebsiddiqui |
+------------------+
If you want to see a breakdown of maintainers by buildspec file you can use --maintainers-by-buildspecs
or -mb
short option. This can be useful when tracking maintainers by buildspec files.
$ buildtest buildspec find --maintainers-by-buildspecs
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| maintainers | buildspec |
+==================+==============================================================================================================================================================================+
| @shahzebsiddiqui | ['/Users/siddiq90/Documents/GitHubDesktop/buildtest/tutorials/hello_world.yml', '/Users/siddiq90/Documents/GitHubDesktop/buildtest/general_tests/configuration/ulimits.yml'] |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+