:py:mod:`buildtest.cli.build` ============================= .. py:module:: buildtest.cli.build .. autoapi-nested-parse:: This module contains all the methods related to "buildtest build" which is used for building test scripts from a Buildspec Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: buildtest.cli.build.Tee buildtest.cli.build.BuildTest Functions ~~~~~~~~~ .. autoapisummary:: buildtest.cli.build.resolve_testdirectory buildtest.cli.build.discover_buildspecs buildtest.cli.build.print_discovered_buildspecs buildtest.cli.build.discover_buildspecs_by_tags buildtest.cli.build.discover_buildspecs_by_name buildtest.cli.build.discover_buildspecs_by_executor buildtest.cli.build.discover_by_buildspecs buildtest.cli.build.print_filters buildtest.cli.build.update_report Attributes ~~~~~~~~~~ .. autoapisummary:: buildtest.cli.build.logger .. py:data:: logger .. py:class:: Tee(filename) Bases: :py:obj:`object` .. py:method:: __enter__() .. py:method:: __exit__(exc_type, exc_value, tb) .. py:method:: write(data) .. py:method:: flush() .. py:function:: resolve_testdirectory(configuration: buildtest.config.SiteConfiguration, testdir: str = None) -> str This method resolves which test directory to select. For example, one can specify test directory via command line ``buildtest build --testdir `` or path in configuration file. The default is $HOME/.buildtest/var/tests :param configuration: An instance of SiteConfiguration class which contains content of buildtest configuration file :type configuration: buildtest.config.SiteConfiguration :param testdir: Path to test directory specified via command line ``buildtest build --testdir`` :type testdir: str, optional :returns: Path to test directory :rtype: str .. py:function:: discover_buildspecs(buildspecs: Optional[List[str]] = None, exclude_buildspecs: Optional[List[str]] = None, name: Optional[List[str]] = None, executors: Optional[List[str]] = None, tags: Optional[List[str]] = None, verbose: Optional[bool] = False) -> Dict[str, List[str]] This method discovers all buildspecs based on --buildspecs, --tags, --executor and excluding buildspecs (--exclude). :param buildspecs: List of input buildspecs passed by argument ``buildtest build --buildspec`` :type buildspecs: list, optional :param exclude_buildspecs: List of excluded buildspecs by argument ``buildtest build --exclude`` :type exclude_buildspecs: list, optional :param name: List of test names to discover buildspecs that are specified by ``buildtest build --name`` :type name: list, optional :param tags: List of input tags for discovering buildspecs by argument ``buildtest build --tags`` :type tags: list, optional :param executors: List of input executors for discovering buildspecs by argument ``buildtest build --executor`` :type executors: list, optional :param verbose: Enable verbose output for buildtest that is specified by ``buildtest --verbose`` :type verbose: bool, optional :returns: A dictionary containing a list of included, excluded, detected buildspecs and buildspecs detected based on tags and executors :rtype: dict .. py:function:: print_discovered_buildspecs(buildspec_dict) This method will print the discovered buildspecs in table format :param buildspec_dict: A dictionary containing a list of included and excluded buildspecs and breakdown of buildspecs by tags and executors :type buildspec_dict: dict .. py:function:: discover_buildspecs_by_tags(buildspec_cache, tagnames) This method discovers buildspecs by tags, using ``buildtest build --tags`` option. This method will read BUILDSPEC_CACHE_FILE and search for ``tags`` key in buildspec recipe and match with input tag. The input ``tags`` are a list of tagnames to search in buildspec with the ``tags`` property in buildspec. The return is a list of buildspec files to process. :param buildspec_cache: Loaded buildspec cache as a dictionary :type buildspec_cache: dict :param tagnames: List of input tags from command line argument ``buildtest build --tags `` :type tagnames: list :returns: first argument is a list of buildspecs discovered for all tag names. The second argument is dictionary breakdown of buildspecs by each tag name :rtype: list, dict .. py:function:: discover_buildspecs_by_name(buildspec_cache, names) This method will discover buildspecs given a list of test names. Args: buildspec_cache (dict): Loaded buildspec cache as a dictionary names (list): List of test names to search for buildspecs from cache :returns: first argument is a list of buildspecs discovered for all test names. The second argument is dictionary breakdown of buildspecs by each name :rtype: list, dict .. py:function:: discover_buildspecs_by_executor(buildspec_cache, executors) This method discovers buildspecs by executor name, using ``buildtest build --executor`` command. This method will read BUILDSPEC_CACHE_FILE and search for ``executor`` property in buildspec and match with input executor name. The return is a list of matching buildspec with executor name to process. :param buildspec_cache: Loaded buildspec cache as a dictionary :type buildspec_cache: dict :param executors: List of input executor name from command line argument ``buildtest build --executor `` :type executors: list :returns: first argument is a list of buildspecs discovered for all executors. The second argument is dictionary breakdown of buildspecs by each executor name :rtype: list, dict .. py:function:: discover_by_buildspecs(buildspec: str) -> list Given a buildspec file specified by the user with ``buildtest build --buildspec``, discover one or more files and return a list for buildtest to process. This method is called once per argument of ``--buildspec`` or ``--exclude`` option. If its a directory path we recursively find all buildspecs with with **.yml** extension. If filepath doesn't exist or file extension is not **.yml** we return None and capture error in log. .. code-block:: console # file path buildtest build --buildspec tutorials/hello.sh.yml # directory path buildtest build --buildspec tutorials # invalid file path returns None buildtest build -b /xyz.yml # invalid file extension buildtest build -b README.md :param buildspec: Full path to buildspec based on argument ``buildtest build --buildspec`` :type buildspec: str :returns: List of resolved buildspecs. :rtype: list .. py:function:: print_filters() This method will print list of filters fields used by ``buildtest build --filter``. This method is invoked by running ``buildtest build --helpfilter``. .. py:class:: BuildTest(account=None, buildspecs=None, configuration=None, display=None, dry_run=None, exclude_buildspecs=None, exclude_tags=None, executors=None, executor_type=None, filter_buildspecs=None, helpfilter=None, limit=None, max_jobs=None, maxpendtime=None, modulepurge=None, modules=None, name=None, numnodes=None, numprocs=None, poll_interval=None, profile=None, rebuild=None, remove_stagedir=None, report_file=None, rerun=None, retry=None, save_profile=None, strict=None, tags=None, testdir=None, timeout=None, unload_modules=None, validate=None, verbose=None, write_config_file=None) This class is an interface to building tests via ``buildtest build`` command. The initializer method is responsible for checking input arguments for type check, if any argument fails type check we raise an error. If all arguments pass we assign the values and proceed with building the test. :param account: Project account to charge jobs. This takes input argument ``buildtest build --account`` :type account: str, optional :param buildspecs: list of buildspecs from command line ``buildtest build --buildspec`` :type buildspecs: list, optional :param configuration: Loaded configuration content which is an instance of SiteConfiguration :type configuration: buildtest.config.SiteConfiguration, optional :param display: Display content of output or test. This is specified via ``buildtest build --display`` :type display: list, optional :param dry_run: Show a list of tests that will potentially be run without actually running them via ``buildtest build --dry-run`` :type dry_run: bool, optional :param exclude_buildspecs: list of excluded buildspecs from command line ``buildtest build --exclude`` :type exclude_buildspecs: list, optional :param exclude_tags: list if tags to exclude specified via command line ``buildtest build --exclude-tags`` :type exclude_tags: list, optional :param executor_type: Filter test by executor type. This option will filter test after discovery by local or batch executors. This can be specified via ``buildtest build --exec-type`` :type executor_type: bool, optional :param executors: list of executors passed from command line ``buildtest build --executors`` :type executors: list, optional :param filter_buildspecs: filters buildspecs and tests based on ``buildtest build --filter`` argument which is a key/value dictionary that can filter tests based on **tags**, **type**, and **maintainers** :type filter_buildspecs: dict, optional :param helpfilter: Display available filter fields for ``buildtest build --filter`` command. This argument is set to ``True`` if one specifies ``buildtest build --helpfilter`` :type helpfilter: bool, optional :param limit: Limit number of tests that can be run. This option is specified by ``buildtest build --limit`` :type limit: int, optional :param max_jobs: Maximum number of jobs to run concurrently. This option is specified by ``buildtest build --max-jobs`` :type max_jobs: int, optional :param maxpendtime: Specify maximum pending time in seconds for batch job until job is cancelled :type maxpendtime: int, optional :param modulepurge: Determine whether to run 'module purge' before running test. This is specified via ``buildtest build --modulepurge``. :type modulepurge: bool, optional :param modules: List of modules to load for every test specified via ``buildtest build --modules``. :type modules: str, optional :param name: list of test names to run specified via command line ``buildtest build --name`` :type name: list, optional :param numnodes: List of comma separated nodes values to run batch jobs specified via ``buildtest build --nodes`` :type numnodes: list, optional :param numprocs: List of comma separated process values to run batch jobs specified via ``buildtest build --procs`` :type numprocs: list, optional :param poll_interval: Specify poll interval in seconds for polling batch jobs. :type poll_interval: int, optional :param profile: Profile to load from buildtest configuration specified by ``buildtest build --profile`` :type profile: str, optional :param rebuild: Rebuild tests X times based on ``buildtest build --rebuild`` option. :type rebuild: int, optional :param remove_stagedir: remove stage directory after job completion :type remove_stagedir: bool, optional :param report_file: Location to report file where test data will be written upon completion. This can be specified via ``buildtest build --report`` command :type report_file: str, optional :param rerun: Rerun last successful **buildtest build** command. This is specified via ``buildtest build --rerun``. All other options will be ignored and buildtest will read buildtest options from file **BUILDTEST_RERUN_FILE**. :type rerun: bool, optional :param retry: Number of retry for failed jobs :type retry: int, optional :param save_profile: Save profile to buildtest configuration specified by ``buildtest build --save-profile`` :type save_profile: str, optional :param strict: Enable strict mode for buildtest. This option is specified by ``buildtest build --strict`` :type strict: bool, optional :param tags: list if tags to discover tests specified via command line ``buildtest build --tags`` :type tags: list, optional :param testdir: Path to test directory where tests are written. This argument can be passed from command line ``buildtest build --testdir`` :type testdir: str :param timeout: Test timeout in seconds specified by ``buildtest build --timeout`` :type timeout: int, optional :param unload_modules: List of modules to unload for every test specified via ``buildtest build --unload-modules``. :type unload_modules: str, optional :param validate: Validate given buildspecs and buildtest will stop after parse stage which can be configured via ``buildtest build --validate`` option :type validate: bool, optional :param verbose: Enable verbose output for buildtest that is specified by ``buildtest --verbose`` :type verbose: bool, optional :param write_config_file: Write configuration file to specified location. This is specified by ``buildtest build --write-config-file`` :type write_config_file: str, optional .. py:attribute:: buildexecutor self.system = buildtest_system if not isinstance(self.system, BuildTestSystem): self.system = BuildTestSystem() .. py:method:: load_rerun_file() This will load content of file BUILDTEST_RERUN_FILE that contains a dictionary of key/value pair that keeps track of last ``buildtest build`` command. This is used with ``buildtest build --rerun``. Upon loading file we reinitalize all class variables that store argument for ``buildtest build`` options .. py:method:: save_rerun_file() Record buildtest command options and save them into rerun file which is read when invoking ``buildtest build --rerun``. .. py:method:: save_profile_to_configuration() This method will save profile to configuration file. This method is called when ``buildtest build --save-profile`` is invoked. We will open the configuration file and update the profile section, if profile already exist we will override it, otherwise we will insert into the configuration file. .. py:method:: load_profile() This method will load profile from configuration file and update class variables used for ``buildtest build`` command. This method is called when ``buildtest build --profile`` is invoked. .. py:method:: _validate_filters() Check filter fields provided by ``buildtest build --filter`` are valid types and supported. Currently supported filter fields are ``tags``, ``type``, ``maintainers`` :raises BuildTestError: if input filter field is not valid we raise exception. For ``type`` filter we check for value and make sure the schema type is supported .. py:method:: build() This method is responsible for discovering buildspecs based on input argument. Then we parse the buildspecs and retrieve builder objects for each test. Each builder object will invoke :func:`buildtest.buildsystem.base.BuilderBase.build` which will build the test script, and then we run the test and update report. .. py:method:: parse_buildspecs() Parse all buildspecs by passing buildspec file to :class:`buildtest.buildsystem.parser.BuildspecParser` class. If buildspec fails validation we skip the buildspec and print all skipped buildspecs. If buildspec passes validation we get all builders by invoking :class:`buildtest.buildsystem.builders.Builder` class that is responsible for creating builder objects for each test. :raises SystemExit: If no builders are created after parsing buildspecs .. py:method:: build_phase() This method will build all tests by invoking class method ``build`` for each builder that generates testscript in the test directory. If no builders are present upon building test we raise exception and terminate immediately :raises BuildTestError: If no builders are present in build phase .. py:method:: run_phase() This method will run all builders with the appropriate executor. The :class:`buildtest.executors.setup.BuildExecutor` class is responsible for orchestrating builder execution to the appropriate executor class. The BuildExecutor contains a list of executors picked up from buildtest configuration. For tests running locally, we get the test metadata and count PASS/FAIL test state which is printed at end in Test Summary. For tests that need batch submission via scheduler, the first stage of run will dispatch job, and state will be unknown. After dispatching all jobs, we will poll jobs until they are complete. The poll section is skipped if all tests are run locally. In poll section we regenerate table with all valid builders and updated test state and returncode and recalculate total pass/fail tests. Any test that failed to run or be dispatched will be skipped during run stage and they will not be recorded in the test report :returns: A list of valid builders after running tests .. py:method:: build_success() Returns status of build whether it was successful or not. The return value is a boolean where ``True`` indicates build was successful and ``False`` indicates build failed. .. py:method:: print_test_summary(builders) Print a summary of total pass and fail test with percentage breakdown. :param builders: List of builders that ran to completion :type builders: list .. py:method:: _update_build_history(builders) Write a build history file that is stored in ``$BUILDTEST_ROOT/var/.history`` directory summarizing output of build. The history file is a json file named `build.json` which contains a copy of the build log for troubleshooting. buildtest will create a sub-directory that is incremented such as 0, 1, 2 in **$BUILDTEST_ROOT/var/.history** which is used to differentiate builds. Shown below is content of the top-level directory for the history directory. There is one subdirectory for each build ID starting with 0 .. code-block:: console bash-3.2$ ls -l $BUILDTEST_ROOT/var/.history total 0 drwxr-xr-x 4 siddiq90 92503 128 Sep 8 13:50 0 drwxr-xr-x 4 siddiq90 92503 128 Sep 8 13:50 1 For every build ID we have a ``build.json`` and log file for each build. .. code-block:: console bash-3.2$ ls $BUILDTEST_ROOT/var/.history/{0,1} /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/.history/0: build.json buildtest_y3gh46j_.log /Users/siddiq90/Documents/GitHubDesktop/buildtest/var/.history/1: build.json buildtest_a1rjdy59.log .. py:method:: get_build_history_dir() Return root of build history directory .. py:method:: print_builders_by_type(builders, builder_type) This method will print the builders in table format. The builders are printed by their builder type :param builders: A list of builder objects that were run that need to be printed :type builders: list :param builder_type: The builder type corresponding to the list of ``builders``. This type corresponds to the ``type`` field in the test :type builder_type: str .. py:method:: print_batch_builders(builders) This method will print the builders that were run in batch mode. The builders will be displayed in table format along with builders by numnodes and numprocs. .. py:method:: print_builders(spack_builder, script_builder, batch_builder) Print detected builders during build phase .. py:function:: update_report(valid_builders, report_file, verbose=None) This method will update BUILD_REPORT after every test run performed by ``buildtest build``. If BUILD_REPORT is not created, we will create file and update json file by extracting contents from builder metadata :param valid_builders: List of valid builders that ran to completion :type valid_builders: list :param report_file: Specify location to report file. :type report_file: str