GitHub Integrations¶
buildtest has several CI checks that are run when you create a Pull Request, it is your responsibility to review the CI checks and make sure all checks are passing. Each pull request will show the CI checks, you can see the github actions that are also typically linked as part of the pull request.
Coverage¶
We use coverage to measure code coverage of buildtest when running regression test. We use CodeCov to display coverage reports through web interface. The coverage configuration is managed by .coveragerc file found in the root of the repo.
Whenever you add new feature to buildtest, please add regression test with test coverage to help maintainers review new feature request. For more details on running coverage tests see Running test via coverage.
CodeCov¶
Codecov report coverage details in web-browser. CodeCov can perform pull request comments after coverage report is uploaded to Codecov which is useful for reviewer and assignee to see status of coverage report during PR review process. The codecov file .codecov.yml is used for configuration codecov. For more details on codecov yaml file see https://docs.codecov.io/docs/codecov-yaml.
Gitlab CI checks¶
buildtest has automated CI checks on gitlab servers: https://software.nersc.gov and https://code.ornl.gov. The gitlab pipelines are stored in .gitlab directory found in root of repository.
The mirror.yml github workflow is responsible for mirroring and trigger CI check and return result back to github PR. Currently, we are using github action stenongithub/gitlab-mirror-and-ci-action to perform pull mirroring and triggering CI job.
The gitlab server https://software.nersc.gov is hosted at NERSC. The following steps were taken to setup pipeline
Create a Personal Access token with read_api, read_repository, write_repository scope at https://software.nersc.gov/-/profile/personal_access_tokens
Define a secret CORI_GITLAB_PASSWORD at https://github.com/buildtesters/buildtest/settings/secrets/actions with token value generated in step 1
Import buildtest project from github at https://software.nersc.gov/siddiq90/buildtest
Add variable SECRET_CODECOV_TOKEN in https://software.nersc.gov/siddiq90/buildtest/-/settings/ci_cd that contains codecov token found at https://app.codecov.io/gh/buildtesters/buildtest/settings
Change gitlab CI configuration file to .gitlab/cori.yml under Settings > CI/CD > General pipelines. For more details see https://docs.gitlab.com/ee/ci/pipelines/settings.html#custom-cicd-configuration-path
The gitlab server https://code.ornl.gov is hosted at OLCF which has access to systems like Summit and Ascent. We performed similar steps at as shown above with slight modification
Create a Personal access token with same scope at https://code.ornl.gov/-/profile/personal_access_tokens
Define a secret OLCF_GITLAB_PASSWORD at https://github.com/buildtesters/buildtest/settings/secrets/actions
Import buildtest project at https://code.ornl.gov/ecpcitest/buildtest. Currently, all projects in
ecpcitest
project group has access to gitlab runners.Add variable SECRET_CODECOV_TOKEN in https://code.ornl.gov/ecpcitest/buildtest/-/settings/ci_cd that contains codecov token found at https://app.codecov.io/gh/buildtesters/buildtest/settings
Change gitlab CI configuration file to .gitlab/olcf.yml
Currently, the gitlab pipelines are triggered manually which requires a user to have access to the gitlab project to run the pipeline. The pipelines can be run manually at https://software.nersc.gov/siddiq90/buildtest/-/pipelines and https://code.ornl.gov/ecpcitest/buildtest/-/pipelines
The github workflow mirror.yml defines gitlab configuration for each mirror. Any changes to mirror path must be addressed in this workflow to ensure pull mirroring is done properly.
GitHub Bots¶
buildtest has a few bots to do various operations that are described below.
Stale - stale bot is used to close outdated issues. This is configured in
.github/stale.yml
. If there is no activity on a issue after certain time period, probot-stale will mark the issue and project maintainers can close it manually. For more details on Stale refer to the documentationCodeCov - The codecov bot will report codecov report from the issued pull request once coverage report is complete. The configuration for codecov is defined in
.codecov.yml
found in root of repo.Pull Request Size - is a bot that labels Pull Request by number of changed lines of code.