Developers Contributing Guide

This guide will walk through the code contribution guide, we expect you have a github account and experience using git and familiarity with GitHub interface.

GitHub Account

If you don’t have a GitHub account please register your account.

Fork the repo

First, you’ll need to fork the repo

You might need to setup your SSH keys in your git profile if you are using ssh option for cloning. For more details on setting up SSH keys in your profile, follow instruction found in

SSH key will help you pull and push to repository without requesting for password for every commit. Once you have forked the repo, clone your local repo:

git clone\_GITHUB\_LOGIN/buildtest.git

Adding Upstream Remote

First you need to add the upstream repo, to do this you can issue the following:

git remote add upstream

The upstream tag is used to sync changes from upstream repo to keep your repo in sync before you contribute back.

Make sure you have set your user name and email set properly in git configuration. We don’t want commits from unknown users. This can be done by setting the following:

git config "First Last"
git config ""

For more details see First Time Git Setup

Sync your branch from upstream

The devel from upstream will get Pull Requests from other contributors, in-order to sync your forked repo with upstream, run the commands below:

git checkout devel
git fetch upstream devel
git pull upstream devel

Once the changes are pulled locally you can sync devel branch with your fork as follows:

git checkout devel
git push origin devel

Repeat this same operation with master branch if you want to sync it with upstream repo

Contribution Workflow

If you want to contribute back, you should create a feature branch from devel and add your files, commit and push them to your fork. The workflow can be summarized as follows:

git checkout devel
git checkout -b featureX
git add <file1> <file2> ...
git commit -m "commit message"
git push origin featureX

Once the branch is created in your fork, you can create a Pull Request with the destination branch devel at and base branch which is your feature branch pushed at your fork.


Do not push to master or devel branch on your fork or upstream.

Best Practices When Creating Pull Request

  • It’s good practice to link PR to an issue during commit message. Such as stating Fix #132 for fixing issue 132.

  • Please create a meaningful title and PR description to help outline your proposed changes.

  • Assign PR to yourself when creating the issue. You should @ mention (@shahzebsiddiqui) the project maintainers to get their attention.

  • If your PR is not ready for review, please add WIP: to your PR title to indicate that it’s a work in progress and make it a draft PR. This will prevent maintainers from reviewing your PR until it’s ready.

  • Check the CI checks corresponding to your PR to ensure all checks are passed. If you see any failures, please fix them especially regression test failures.

Pull Request Review

Once you have submitted a Pull Request, please check the automated checks that are run for your PR to ensure checks are passed. Most common failures in CI checks are black and pyflakes issue, this can be done by configuring black and running pyflakes. Once all checks have passed, maintainer will review your PR and provide feedback so please be patient. Please coordinate with maintainer through PR or Slack.

Resolving PR Merge Conflicts

Often times, you may start a feature branch and your PR get’s out of sync with devel branch which may lead to conflicts, this is a result of merging incoming PRs that may cause upstream HEAD to change over time which can cause merge conflicts. This may be confusing at first, but don’t worry we are here to help. For more details about merge conflicts click here.

Syncing your feature branch with devel is out of scope for this documentation, however you can use the steps below as a guide when you run into this issue.

You may want to take the steps to first sync devel branch and then selectively rebase or merge devel into your feature branch.

First go to devel branch and fetch changes from upstream:

git checkout devel
git fetch upstream devel

Note you shouldn’t be making any changes to your local devel branch, if git fetch was successful you can merge your devel with upstream as follows:

git merge upstream/devel

Next, navigate to your feature branch and sync feature changes with devel:

git checkout <feature-branch>
git merge devel


Running above command will sync your feature branch with devel but you may have some file conflicts depending on files changed during PR. You will need to resolve them manually before pushing your changes

Instead of merge from devel you can rebase your commits interactively when syncing with devel. This can be done by running:

git rebase -i devel

Once you have synced your branch push your changes and check if file conflicts are resolved in your Pull Request:

git push origin <feature-branch>

General Tips

  • If you have an issue, ask your question in slack before reporting the issue. If your issue is not resolved check any open issues for resolution before creating a new issue.

  • For new features or significant code refactors, please notify maintainers and open an issue before working on task to keep everyone informed.

  • If you open an issue, please respond back during the discussion, if there is no activity the issue will be closed.

  • Please refrain from opening a duplicate issue, check if there is an existing issue addressing similar problems. You can ask questions in slack to report your issue or contact project maintainers.

  • There should not be any branches other than master or devel. Feature branches should be pushed to your fork and not to origin.

Configuring Black Pre-Commit Hook

To configure pre-commit hook, make sure you install pre-commit via pip install pre-commit. The pre-commit utility should be available if you install extra dependencies from buildtest (pip install '.[dev]').

The pre-commit hook configuration can be found in .pre-commit-config.yaml

To install the pre-commit hook run:

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

This will invoke hook .git/hooks/pre-commit prior to git commit. Shown below we attempt to commit which resulted in pre commit hook and caused black to format code.

$ git commit -m "test black commit with precommit"
- hook id: black
- files were modified by this hook

reformatted buildtest/
All done! ✨ 🍰 ✨
1 file reformatted.

If you are interested in running black locally to see diff result from black without auto-formatting code, you can do the following:

$ black --check --diff .
--- tests/       2020-02-25 18:58:58.360360 +0000
+++ tests/       2020-02-25 18:59:07.336414 +0000
@@ -18,11 +18,11 @@
 def test_distro_short():
     assert "rhel" == distro_short("Red Hat Enterprise Linux Server")
     assert "centos" == distro_short("CentOS")
     assert "suse" == distro_short("SUSE Linux Enterprise Server")
-    x=0+1*3
+    x = 0 + 1 * 3

The changes will be shown with lines removed or added via - and +. For more details refer to black documentation.


isort is a python utility that will sort python imports alphabetically. We use isort as part of the CI checks, this is configured in pyproject.toml that defines the isort configuration that is compatible with black utility. We have setup a pre-commit hook that can be used to automatically run isort as part of your git commit process. This is defined in pre-commit configuration file .pre-commit-config.yaml that can be installed by running pre-commit install. Once this is setup, you will see isort and black checks are run during the commit process.

$ git commit
[sphinx_fix 85d9d42c] fix issue with rendering bullet points in sphinx. This is solved by downgrading docutils to version 0.16.
 2 files changed, 5 insertions(+)

If you want to run isort, you can use the -c and –diff option to check and see diff between files. For instance in example below we see isort reports changes to import statement

$ isort -c --diff profile black  buildtest/
ERROR: /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest/ Imports are incorrectly sorted and/or formatted.
--- /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest/      2021-07-13 16:53:42.722718
+++ /Users/siddiq90/Documents/GitHubDesktop/buildtest/buildtest/       2021-07-13 16:54:12.135986
@@ -1,8 +1,7 @@
 """Entry point for buildtest"""

+import os
 import webbrowser
-import os

 from buildtest.cli import get_parser
 from import BuildTest
Broken 2 paths

If you want to apply the changes you can get rid of -c and --diff option and isort will apply the changes. Please see and for documentation regarding black and isort compatibility.


pyflakes is a program that checks for python source code for errors such as unused imports. We have configured an automated check to test your incoming PR using pyflakes. pyflakes should be available in your python environment if you installed the dev dependencies (pip install '.[dev]').

You can run pyflakes against any file or directory the ones of importance is running pyflakes against buildtest source code and regression test. You can do that by running:

pyflakes buildtest tests

Running yamllint

We are using yamllint, which is a linter for YAML files. We have a configuration file .yamllint.yml used for configuring yamllint.

You can run yamllint against any file or and it will show the lint errors such as this example below

 yamllint .github/workflows/style.yml
  18:81     warning  line too long (103 > 80 characters)  (line-length)
  36:81     warning  line too long (107 > 80 characters)  (line-length)

You don’t need to specify path to configuration file (i.e yamllint -c /path/to/.yamllint.yml) because .yamllint.yml is a default configuration file by the linter. Please refer to for more details on configuration options for the linter.

The Style Check workflow is responsible for running the yamllinter check on the buildtest codebase. Please refer to the CI check, when debugging linter errors.

Shell Check

We are using shellcheck a static analysis tool for checking shell scripts. This package can be installed in your system using package manager of your choice. Please refer to README for more details on installation.

The shellcheck binary can be used to check bash or sh scripts. A typical output will consist of list of error codes with line number where error appears such as one below

  shellcheck bin/buildtest

In bin/buildtest line 14:
        export BUILDTEST_PYTHON="$(command -v "$cmd")"
               ^--------------^ SC2155 (warning): Declare and assign separately to avoid masking return values.

In bin/buildtest line 21:
^---^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

In bin/buildtest line 23:
import os
^-------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

In bin/buildtest line 24:
import sys
^--------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

In bin/buildtest line 26:
^------------^ SC2034 (warning): buildtest_file appears unused. Verify use (or export if used externally).
^-----------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                               ^-- SC1036 (error): '(' is invalid here. Did you forget to escape it?
                               ^-- SC1088 (error): Parsing stopped here. Invalid use of parentheses?

For more information: -- buildtest_file appears unused. Ve... -- Declare and assign separately to ... -- Command appears to be unreachable...

We have configured shellcheck with a configuration file .shellcheckrc that can be used to disable certain error codes from checks. This is equivalent to running shellcheck -e <CODE1>,<CODE2> but we included this in configuration file to make it the default setting.

We have a shellcheck workflow that will perform check on shell scripts, please refer to the CI results when troubleshooting errors.

Running stylechecks via buildtest stylecheck

The buildtest stylecheck command can run the stylechecks such as black, isort, pyflakes which can should be used before you commit your changes. Shown below are the available options for buildtest stylecheck

$ buildtest stylecheck --help
usage: buildtest [options] [COMMANDS] stylecheck [-h] [--no-black]
                                                 [--no-isort] [--no-pyflakes]

optional arguments:
  -h, --help     show this help message and exit
  --no-black     Don't run black style check
  --no-isort     Don't run isort style check
  --no-pyflakes  Don't run pyflakes check
  -a, --apply    Apply style checks to codebase.


buildtest style is an alias for buildtest stylecheck

By default, all the checks are run when no options are specified however if you want to disable a particular style check you can specify on command line such as --no-black will disable black style check.

Shown below is an example output of what style check will report. By default, black and isort will report changes that will need to be fixed, if you want to apply those changes to buildtest codebase you can pass the --apply option.

buildtest stylecheck
$ buildtest stylecheck
Running black check: black --check --diff /home/docs/checkouts/ /home/docs/checkouts/ /home/docs/checkouts/
Running black check FAILED
────────────────────── Running black check output message ──────────────────────
--- /home/docs/checkouts/    2024-02-23 20:52:21.231713+00:00
+++ /home/docs/checkouts/    2024-02-23 20:54:02.441243+00:00
@@ -259,12 +259,10 @@
 # A list of files that should not be packed into the epub file.
 epub_exclude_files = ["search.html"]
 #          auto-created specific configuration            #
@@ -287,169 +285,162 @@
 from sphinx import version_info
 # Get suffix for proper linking to GitHub
 # This is deprecated in Sphinx 1.3+,
 # as each page can have its own suffix
-if globals().get('source_suffix', False):
+if globals().get("source_suffix", False):
     if isinstance(source_suffix, string_types):
         SUFFIX = source_suffix
     elif isinstance(source_suffix, (list, tuple)):
         # Sphinx >= 1.3 supports list/tuple to define multiple suffixes
         SUFFIX = source_suffix[0]
     elif isinstance(source_suffix, dict):
         # Sphinx >= 1.8 supports a mapping dictionary for multiple suffixes
-        SUFFIX = list(source_suffix.keys())[0]  # make a ``list()`` for py2/py3 compatibility
+        SUFFIX = list(source_suffix.keys())[
+            0
+        ]  # make a ``list()`` for py2/py3 compatibility
         # default to .rst
-        SUFFIX = '.rst'
+        SUFFIX = ".rst"
-    SUFFIX = '.rst'
+    SUFFIX = ".rst"
 # Add RTD Static Path. Add to the end because it overwrites previous files.
-if not 'html_static_path' in globals():
+if not "html_static_path" in globals():
     html_static_path = []
-if os.path.exists('_static'):
-    html_static_path.append('_static')
+if os.path.exists("_static"):
+    html_static_path.append("_static")
 # Define this variable in case it's not defined by the user.
 # It defaults to `alabaster` which is the default from Sphinx.
-html_theme = globals().get('html_theme', 'alabaster')
-#Add project information to the template context.
+html_theme = globals().get("html_theme", "alabaster")
+# Add project information to the template context.
 context = {
-    'html_theme': html_theme,
-    'current_version': "latest",
-    'version_slug': "latest",
-    'MEDIA_URL': "",
-    'STATIC_URL': "",
-    'proxied_static_path': "/_/static/",
-    'versions': [
-    ("latest", "/en/latest/"),
-    ("stable", "/en/stable/"),
-    ("v1.8", "/en/v1.8/"),
-    ("v1.7", "/en/v1.7/"),
-    ("v1.6", "/en/v1.6/"),
-    ("v1.5", "/en/v1.5/"),
-    ("v1.4_a", "/en/v1.4_a/"),
-    ("v1.3", "/en/v1.3/"),
-    ("v1.2", "/en/v1.2/"),
-    ("v1.1", "/en/v1.1/"),
-    ("v1.0", "/en/v1.0/"),
-    ("v0.15.0", "/en/v0.15.0/"),
-    ("v0.14.0", "/en/v0.14.0/"),
-    ("v0.13.0", "/en/v0.13.0/"),
-    ("v0.12.0", "/en/v0.12.0/"),
-    ("v0.11.0", "/en/v0.11.0/"),
-    ("v0.10.2", "/en/v0.10.2/"),
-    ("v0.10.0", "/en/v0.10.0/"),
-    ("v0.9.6", "/en/v0.9.6/"),
-    ("v0.9.0", "/en/v0.9.0/"),
-    ("v0.8.0", "/en/v0.8.0/"),
-    ("devel", "/en/devel/"),
+    "html_theme": html_theme,
+    "current_version": "latest",
+    "version_slug": "latest",
+    "MEDIA_URL": "",
+    "STATIC_URL": "",
+    "proxied_static_path": "/_/static/",
+    "versions": [
+        ("latest", "/en/latest/"),
+        ("stable", "/en/stable/"),
+        ("v1.8", "/en/v1.8/"),
+        ("v1.7", "/en/v1.7/"),
+        ("v1.6", "/en/v1.6/"),
+        ("v1.5", "/en/v1.5/"),
+        ("v1.4_a", "/en/v1.4_a/"),
+        ("v1.3", "/en/v1.3/"),
+        ("v1.2", "/en/v1.2/"),
+        ("v1.1", "/en/v1.1/"),
+        ("v1.0", "/en/v1.0/"),
+        ("v0.15.0", "/en/v0.15.0/"),
+        ("v0.14.0", "/en/v0.14.0/"),
+        ("v0.13.0", "/en/v0.13.0/"),
+        ("v0.12.0", "/en/v0.12.0/"),
+        ("v0.11.0", "/en/v0.11.0/"),
+        ("v0.10.2", "/en/v0.10.2/"),
+        ("v0.10.0", "/en/v0.10.0/"),
+        ("v0.9.6", "/en/v0.9.6/"),
+        ("v0.9.0", "/en/v0.9.0/"),
+        ("v0.8.0", "/en/v0.8.0/"),
+        ("devel", "/en/devel/"),
-    'downloads': [ 
-    ],
-    'subprojects': [ 
-    ],
-    'slug': 'buildtest',
-    'name': u'buildtest',
-    'rtd_language': u'en',
-    'programming_language': u'py',
-    'canonical_url': '',
-    'analytics_code': 'UA-132939465-1',
-    'single_version': False,
-    'conf_py_path': '/docs/',
-    'api_host': '',
-    'github_user': 'buildtesters',
-    'proxied_api_host': '/_',
-    'github_repo': 'buildtest',
-    'github_version': 'master',
-    'display_github': True,
-    'bitbucket_user': 'None',
-    'bitbucket_repo': 'None',
-    'bitbucket_version': 'master',
-    'display_bitbucket': False,
-    'gitlab_user': 'None',
-    'gitlab_repo': 'None',
-    'gitlab_version': 'master',
-    'display_gitlab': False,
-    'READTHEDOCS': True,
-    'using_theme': (html_theme == "default"),
-    'new_theme': (html_theme == "sphinx_rtd_theme"),
-    'source_suffix': SUFFIX,
-    'ad_free': False,
-    'docsearch_disabled': False,
-    'user_analytics_code': 'UA-132939465-1',
-    'global_analytics_code': 'UA-17997319-1',
-    'commit': 'a74e2f74',
+    "downloads": [],
+    "subprojects": [],
+    "slug": "buildtest",
+    "name": "buildtest",
+    "rtd_language": "en",
+    "programming_language": "py",
+    "canonical_url": "",
+    "analytics_code": "UA-132939465-1",
+    "single_version": False,
+    "conf_py_path": "/docs/",
+    "api_host": "",
+    "github_user": "buildtesters",
+    "proxied_api_host": "/_",
+    "github_repo": "buildtest",
+    "github_version": "master",
+    "display_github": True,
+    "bitbucket_user": "None",
+    "bitbucket_repo": "None",
+    "bitbucket_version": "master",
+    "display_bitbucket": False,
+    "gitlab_user": "None",
+    "gitlab_repo": "None",
+    "gitlab_version": "master",
+    "display_gitlab": False,
+    "READTHEDOCS": True,
+    "using_theme": (html_theme == "default"),
+    "new_theme": (html_theme == "sphinx_rtd_theme"),
+    "source_suffix": SUFFIX,
+    "ad_free": False,
+    "docsearch_disabled": False,
+    "user_analytics_code": "UA-132939465-1",
+    "global_analytics_code": "UA-17997319-1",
+    "commit": "a74e2f74",
 # For sphinx >=1.8 we can use html_baseurl to set the canonical URL.
 if version_info >= (1, 8):
-    if not globals().get('html_baseurl'):
-        html_baseurl = context['canonical_url']
-    context['canonical_url'] = None
-if 'html_context' in globals():
+    if not globals().get("html_baseurl"):
+        html_baseurl = context["canonical_url"]
+    context["canonical_url"] = None
+if "html_context" in globals():
     for key in context:
         if key not in html_context:
             html_context = context
     html_context = context
 # Add custom RTD extension
-if 'extensions' in globals():
+if "extensions" in globals():
     # Insert at the beginning because it can interfere
     # with other extensions.
     # See
     extensions.insert(0, "readthedocs_ext.readthedocs")
     extensions = ["readthedocs_ext.readthedocs"]
 # Add External version warning banner to the external version documentation
-if 'branch' == 'external':
+if "branch" == "external":
     extensions.insert(1, "readthedocs_ext.external_version_warning")
-    readthedocs_vcs_url = 'None'
-    readthedocs_build_url = ''
-project_language = 'en'
+    readthedocs_vcs_url = "None"
+    readthedocs_build_url = (
+        ""
+    )
+project_language = "en"
 # User's Sphinx configurations
-language_user = globals().get('language', None)
-latex_engine_user = globals().get('latex_engine', None)
-latex_elements_user = globals().get('latex_elements', None)
+language_user = globals().get("language", None)
+latex_engine_user = globals().get("latex_engine", None)
+latex_elements_user = globals().get("latex_elements", None)
 # Remove this once xindy gets installed in Docker image and XINDYOPS
 # env variable is supported
 latex_use_xindy = False
-chinese = any([
-    language_user in ('zh_CN', 'zh_TW'),
-    project_language in ('zh_CN', 'zh_TW'),
-japanese = any([
-    language_user == 'ja',
-    project_language == 'ja',
+chinese = any(
+japanese = any()
 if chinese:
-    latex_engine = latex_engine_user or 'xelatex'
-    latex_elements_rtd = {
-        'preamble': '\\usepackage[UTF8]{ctex}\n',
-    }
+    latex_engine = latex_engine_user or "xelatex"
+    latex_elements_rtd = {"preamble": "\\usepackage[UTF8]{ctex}\n"}
     latex_elements = latex_elements_user or latex_elements_rtd
 elif japanese:
-    latex_engine = latex_engine_user or 'platex'
+    latex_engine = latex_engine_user or "platex"
 # Make sure our build directory is always excluded
-exclude_patterns = globals().get('exclude_patterns', [])
+exclude_patterns = globals().get("exclude_patterns", [])

────────────────────── Running black check error message ───────────────────────
would reformat /home/docs/checkouts/
Oh no! 💥 💔 💥
1 file would be reformatted, 158 files would be left unchanged.

Running pyflakes check: pyflakes /home/docs/checkouts/ /home/docs/checkouts/ /home/docs/checkouts/
Running pyflakes check FAILED
──────────────────── Running pyflakes check output message ─────────────────────
/home/docs/checkouts/ 'importlib' imported but unused
/home/docs/checkouts/ undefined name 'basestring'

───────────────────── Running pyflakes check error message ─────────────────────