Building Documentation

The buildtest documentation is written in reStructuredText using sphinx. You should be familiar with rst if you want to contribute to user documentation.

ReadTheDocs

buildtest documentation is hosted by ReadTheDocs at https://readthedocs.org which is a documentation platform for building and hosting your docs.

buildtest project can be found at https://readthedocs.org/projects/buildtest/ which will show the recent builds and project setting. The file .readthedocs.yaml is a special configuration file used to configure the build process for ReadTheDocs. Please refer to this file to tweak changes to ReadTheDocs.

If you are interested in becoming a maintainer, please contact Shahzeb Siddiqui (shahzebmsiddiqui@gmail.com) to grant access to this project.

Setup

buildtest documentation is located in top-level docs directory. If you want to build the documentation you will need to make sure your python environment has all the packages defined in docs/requirements.txt. If your environment setup, then you can skip this step.

To install python dependencies for documentation build, you can run the following:

# install from requirements file
pip install -r docs/requirements.txt

# install from pyproject.toml
pip install '.[docs]'

Building docs locally

To build your documentation, navigate to the docs directory and run the following:

cd docs
make clean
make html

It’s best practice to run make clean to ensure sphinx will remove old html content from previous builds, but it is ok to skip this step if you are making minor changes.

Running make html will build the sphinx project and generate all the html files in docs/_build/html. Once this process is complete you can view the html pages by running the following:

open _build/html/index.html

Please refer to the Makefile to see list of tags or run make for additional help.

Sphinx

The documentation is built via Sphinx using reStructuredText (rST) as its markup language. When you run make you are running sphinx-build command which will generate the documentation.

Sphinx will read the configuration file conf.py used for building the project. We have enabled a couple sphinx extensions in our project to customize our documentation

API Generation

We make use of Sphinx AutoAPI to generate buildtest API documentation that is hosted on https://buildtest.readthedocs.io/en/devel/api/index.html. The Sphinx AutoAPI configuration is configured in sphinx configuration file conf.py. For more details on configuration options see https://sphinx-autoapi.readthedocs.io/en/latest/reference/config.html

Command Line Documentation

We make use of sphinx-argparse to generate documentation for buildtest command line that is hosted at https://buildtest.readthedocs.io/en/devel/command.html. In order to use this tool one must install this package and enable the extension in sphinx configuration.

DocStrings

We have enabled napolean extension to support Google style docstring. Please follow this format when you are writting docstring for buildtest codebase. For more details on google style see: https://google.github.io/styleguide/pyguide.html

Generating Documentation Examples for Buildtest Tutorial

The buildtest tutorial-examples command is used for auto-generating examples for the buildtest tutorial. This command will serve the purpose of semi-automating the test creation where examples can’t be run on readthedocs platform. Shown below is the command usage for buildtest tutorial-examples.

$ buildtest tutorial-examples --help
usage: buildtest [options] [COMMANDS] tutorial-examples [-h] [-d] [-w]
                                                        [--failfast]
                                                        {aws,spack}

positional arguments:
  {aws,spack}   Select which tutorial examples to build

options:
  -h, --help    show this help message and exit
  -d, --dryrun  Just print commands that will be generated without running
                them
  -w, --write   Write the content of each command to file
  --failfast    Stop on first failure

The documentation examples for the Buildtest Spack Integration are run inside the container image ghcr.io/buildtesters/buildtest_spack:latest which means that some of the example output needs to be generated manually.

To get into the container along with the buildtest codebase you will need to run the following commands

Note

You may need to source /etc/profile in your container if you see module command is not found.

Note

You will need to volume mount $BUILDTEST_ROOT into /home/spack/buildtest in-order to get buildtest code-base accessible inside the container.

docker run -it -v  $BUILDTEST_ROOT:/home/spack/buildtest ghcr.io/buildtesters/buildtest_spack:latest

Once you are in the container run the following commands

cd /home/spack/buildtest
source scripts/spack_container/setup.sh

Once your setup is complete, you can run the spack generated examples in dryrun mode by running:

buildtest tutorial-examples spack --dryrun

This will print a list of buildtest commands that will be run without actually executing them. If you want to test the examples, you can run the following command:

buildtest tutorial-examples spack

If you want buildtest to write the changes to the documentation, you will need to use the --write flag which will run the example tests and write the output to file.

If you want to generate the examples for Buildtest Tutorial on AWS, you will need to access the E4S Pro container image and clone buildtest, checkout to your branch and then run the command:

buildtest tutorial-examples aws

Please verify all the auto-generated examples that will be used in the documentation. Once you are content with all the changes please add all the changes via git add.