:py:mod:`buildtest.builders.base` ================================= .. py:module:: buildtest.builders.base .. autoapi-nested-parse:: BuilderBase class is an abstract class that defines common functions for any types of builders. Each type schema (script, compiler, spack) is implemented as separate Builder which extends BuilderBase class. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: buildtest.builders.base.BuilderBase .. py:class:: BuilderBase(name, recipe, buildspec, executor, buildexecutor, testdir, numprocs=None, numnodes=None, compiler=None, display=None) Bases: :py:obj:`abc.ABC` The BuilderBase is an abstract class that implements common functions used for building and running test. buildtest will create a builder object which resembles a test. The builder will contains metadata that is unique to each builder that is captured in the report file upon completion of test. The BuilderBase provides common functions for any builder. The builder is an instance of BuilderBase. The initializer method will setup the builder attributes based on input test by ``name`` parameter. :param name: Name of test in buildspec recipe :type name: str :param recipe: The loaded test section from the buildspec file :type recipe: str :param buildspec: Full path to buildspec file :type buildspec: str :param buildexecutor: An instance of BuildExecutor class used for accessing executors :type buildexecutor: :obj:`buildtest.executors.setup.BuildExecutor` :param testdir: Test directory where tests are written. Must be full path on filesystem. :type testdir: str :param display: Display content of output/error or test. :type display: list, optional .. py:property:: dependency .. py:method:: shell_detection() Detect shell and shebang used for test script .. py:method:: _set_metadata_values() This method sets ``self.metadata`` that contains metadata for each builder object. .. py:method:: get_test_extension() Return the test extension, which depends on the shell type. By default we return `sh` file extension for all shells except for `csh` which will return "csh" extension. Returns str: Returns test extension name for generated test. .. py:method:: is_local_executor() Return True if current builder executor type is LocalExecutor otherwise returns False. :returns: returns True if builder is using executor type LocalExecutor otherwise returns False :rtype: bool .. py:method:: is_container_executor() .. py:method:: is_batch_job() Return True/False if builder.job attribute is of type Job instance if not returns False. This method indicates if builder has a job submitted to queue .. py:method:: start() Keep internal timer for test using class :class:`buildtest.utils.timer.Timer`. This method will start the timer for builder which is invoked upon running test. .. py:method:: stop() Stop internal timer for builder. .. py:method:: retry(retry) .. py:method:: build(modules=None, modulepurge=None, unload_modules=None) This method is responsible for invoking setup, creating test directory and writing test. This method is called from an instance object of this class that does ``builder.build()``. :param modules: Specify a list of modules to load in the build script :type modules: str, optional :param modulepurge: A boolean to control whether 'module purge' is run before running test :type modulepurge: bool, optional :param unload_modules: Specify a list of modules to unload in the build script :type unload_modules: str, optional .. py:method:: run(cmd, timeout=None) This is the entry point for running the test. This method will prepare test to be run, then run the test. Once test is complete, we also handle test results by capturing output and error. :returns: If success, the return type is an object of type :class:`buildtest.utils.command.BuildTestCommand` If their is a failure (non-zero) returncode we retry test and if it doesn't pass we raise exception of :class:`buildtest.exceptions.RuntimeFailure` .. py:method:: prepare_run(cmd) This method prepares the test to be run by recording starttime, setting state to running and starting the timer. In additional we will write build environment into build-env.txt which is used for debugging purposes. .. py:method:: execute_run(cmd, timeout) This method will execute the test and return the instance object of type BuildTestCommand which is used by Executors for processing output and error .. py:method:: execute_post_run_script() This method will execute the post run script which is invoked after test is complete. This is called if ``post_run`` is defined in buildspec. .. py:method:: _display_output_content(output, title, show_last_lines=10) This method will display content of output or error results. The ``output`` is content of file to display. A title is displayed at top which can be customized via ``title``. We can configure number of lines to display from end of file via ```show_last_lines``. :param output: Output content to display :type output: str :param title: Title to display before content :type title: str :param show_last_lines: Number of lines to display from end of file. Default is 10 :type show_last_lines: int, optional .. py:method:: _display_test_content(filepath, title) -> None Display content of test file, given a path to file and title to display. :param filepath: Path to file to display content :type filepath: str :param title: Title to display before content :type title: str .. py:method:: handle_run_result(command_result, timeout) This method will handle the result of running test. If the test is successful we will record endtime, copy output and error file to test directory and set state to complete. If the test fails we will retry the test based on retry count. If the test fails after retry we will mark test as failed. :param command_result: An instance object of BuildTestCommand :type command_result: BuildTestCommand :param timeout: Timeout value for test run :type timeout: int .. py:method:: record_starttime() This method will record the starttime when job starts execution by using ``datetime.datetime.now()`` .. py:method:: record_endtime() This method is called upon termination of job, we get current time using ``datetime.datetime.now()`` and calculate runtime of job .. py:method:: runtime() Calculate runtime of job by calculating delta between endtime and starttime. The unit of measure is seconds. .. py:method:: get_runtime() Return runtime of test .. py:method:: state() .. py:method:: failed() Mark test as failure by updating the ``self._state``. A fail test will not be reported in test report .. py:method:: complete() Mark test as complete by updating the ``self._state``. A complete test assumes test ran to completion .. py:method:: running() .. py:method:: is_pending() .. py:method:: is_complete() If builder completes execution of test this method will return ``True`` otherwise returns ``False``. A builder could fail due to job cancellation, failure to submit job or raise exception during the run phase. In those case, this method will return ``False``. .. py:method:: is_failed() Return True if builder fails to run test. .. py:method:: is_running() Return True if builder fails to run test. .. py:method:: _build_setup() -> None Setup operation to get ready to build test. .. py:method:: _create_directories() -> None Create necessary directories for the build. .. py:method:: _resolve_paths() -> None Resolve full paths to generated test script and build script. .. py:method:: _copy_files_to_stage() -> None Copy all files from buildspec directory to stage directory. .. py:method:: _write_build_script(modules: List[str] = None, modulepurge: bool = None, unload_modules: List[str] = None) -> None Write the content of build script. .. py:method:: _generate_build_script_lines(modules: List[str], modulepurge: bool, unload_modules: List[str]) -> List[str] Generate lines for the build script. .. py:method:: _generate_trap_message() -> str Generate trap message for the build script. .. py:method:: _get_execution_command() -> List[str] Get the command to execute the script. .. py:method:: _copy_build_script_to_test_root() -> None Copy build script to test root directory. .. py:method:: _write_post_run_script() This method will write the content of post run script that is run after the test is complete. The post run script is used to perform cleanup operations after test is complete. Upon creating file we set permission of builder script to 755 so test can be run. .. py:method:: _write_test() This method is responsible for invoking ``generate_script`` that formulates content of testscript which is implemented in each subclass. Next we write content to file and apply 755 permission on script so it has executable permission. .. py:method:: get_container_invocation() This method returns a list of lines containing the container invocation .. py:method:: _emit_command() This method will return a shell command used to invoke the script that is used for tests that use local executors :returns: a list to show generated command used to run test. Test can be run without any argument with path to script: ``/path/to/script.sh`` Test can be run with shell name followed by path to script: ``bash /path/to/script.sh`` Test can be run with shell name, shell options and path to script: ``bash -x /path/to/script.sh`` :rtype: list .. py:method:: _emit_set_command() This method will emit the set command for strict mode that will exit immediately. In bash, zsh the command is ``set -eo pipefail``. For csh, tcsh and sh there is no such command so we return empty string. .. py:method:: _set_default_test_variables() Return a list of lines inserted in build script that define buildtest specific variables that can be referenced when writing tests. The buildtest variables all start with BUILDTEST_* .. py:method:: set_scheduler_settings() This method will resolve scheduler fields: 'sbatch', 'pbs', 'bsub' .. py:method:: get_job_directives() This method returns a list of lines containing the scheduler directives .. py:method:: _get_burst_buffer(burstbuffer) Get Burst Buffer directives (**#BB**) lines specified by ``BB`` property :param burstbuffer: Burst Buffer configuration specified by ``BB`` property in buildspec :type burstbuffer: str :returns: List of string values containing containing ``#BB`` directives written in test :rtype: list .. py:method:: _get_data_warp(datawarp) Get Cray Data Warp directives (**#DW**) lines specified by ``DW`` property. :param datawarp: Data Warp configuration specified by ``DW`` property in buildspec :type datawarp: str :returns: List of string values containing ``#DW`` directives written in test :rtype: list .. py:method:: _set_execute_perm(fname) Set permission to 755 for a given file. The filepath must be an absolute path to file .. py:method:: _get_environment(env) Retrieve a list of environment variables defined in buildspec and return them as list with the shell equivalent command :param env: list of environment variables defined by ``env`` property in buildspec :type env: dict .. py:method:: _get_variables(variables) Retrieve a list of variables defined in buildspec and return them as list with the shell equivalent command. :param variables: list of variable defined by ``vars`` property in buildspec :type variables: dict .. py:method:: _extract_line(linenum, content) Extract content based on the line number and return it as a string. :param linenum: line number :type linenum: int :param content: content to be extracted from :type content: str .. py:method:: add_metrics() This method will update the metrics field stored in ``self.metadata['metrics']``. The ``metrics`` property can be defined in the buildspec to assign value to a metrics name based on regular expression, environment or variable assignment. .. py:method:: handle_regex_metric(key, regex) Handle metrics based on regular expressions. .. py:method:: handle_file_regex_metric(key, file_regex) Handle metrics based on file regular expressions. .. py:method:: get_match(regex, content) Get the match based on the regular expression. .. py:method:: output() Return output content .. py:method:: error() Return error content .. py:method:: generate_script() :abstractmethod: Build the testscript content implemented in each subclass .. py:method:: post_run_steps() This method is called after test is complete. This method will copy files from stage directory such as output, error and test script. We will check state of test and mark job is complete. .. py:method:: is_valid_metric(name) .. py:method:: check_test_state() This method is responsible for detecting state of test (PASS/FAIL) based on returncode or regular expression. .. py:method:: _process_compiler_config() This method is responsible for setting cc, fc, cxx class variables based on compiler selection. The order of precedence is ``config``, ``default``, then buildtest setting. Compiler settings in 'config' takes highest precedence, this overrides any configuration in 'default'. Finally we resort to compiler configuration in buildtest setting if none defined. This method is responsible for setting cc, fc, cxx, cflags, cxxflags, fflags, ldflags, and cppflags. .. py:method:: __str__() Return str(self). .. py:method:: __repr__() Return repr(self).