:py:mod:`buildtest.cli.buildspec`
=================================

.. py:module:: buildtest.cli.buildspec


Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   buildtest.cli.buildspec.BuildspecCache



Functions
~~~~~~~~~

.. autoapisummary::

   buildtest.cli.buildspec.edit_buildspec_test
   buildtest.cli.buildspec.edit_buildspec_file
   buildtest.cli.buildspec.is_test_name_in_cache
   buildtest.cli.buildspec.show_buildspecs
   buildtest.cli.buildspec.show_tests_by_status
   buildtest.cli.buildspec.show_failed_buildspecs
   buildtest.cli.buildspec.handle_exception
   buildtest.cli.buildspec.buildspec_validate_command
   buildtest.cli.buildspec.summarize_buildspec_cache
   buildtest.cli.buildspec.summary_print
   buildtest.cli.buildspec.buildspec_maintainers
   buildtest.cli.buildspec.buildspec_find
   buildtest.cli.buildspec.open_buildspec_in_editor
   buildtest.cli.buildspec.validate_buildspec



Attributes
~~~~~~~~~~

.. autoapisummary::

   buildtest.cli.buildspec.logger


.. py:data:: logger

   

.. py:class:: BuildspecCache(configuration, rebuild=False, filterfields=None, formatfields=None, header=None, terse=None, pager=None, color=None, count=None, row_count=None, search_buildspecs=None)


   
   The initializer method for BuildspecCache class is responsible for loading and finding buildspecs into buildspec cache. First we
   resolve paths to directory where buildspecs will be searched. This can be specified via ``--directory`` option on command line or one can
   specify directory paths in the configuration file. Next we build the cache that contains metadata for each buildspec that will be
   written to file. If any filter or format options are specified we check if they are valid and finally display a content of the cache
   depending on the argument.

   This method is called when using ``buildtest buildspec find`` command.

   :param configuration: Instance of SiteConfiguration class that is loaded buildtest configuration.
   :type configuration: buildtest.config.SiteConfiguration
   :param rebuild: rebuild the buildspec cache by validating all buildspecs when using ``buildtest buildspec find --rebuild``. Defaults to ``False`` if ``--rebuild`` is not specified
   :type rebuild: bool, optional
   :param filterfields: The filter options specified via ``buildtest buildspec find --filter`` that contains list of key value pairs for filtering buildspecs
   :type filterfields: str, optional
   :param formatfields: The format options used for formating table. The format option is a comma separated list of format fields specified via ``buildtest buildspec find --format``
   :type formatfields: str, optional
   :param headers: Option to control whether header are printed in terse output. This argument contains value of ``buildtest buildspec find --no-header``
   :type headers: bool, optional
   :param terse: Enable terse mode when printing output. In this mode we don't print output in table format instead output is printed in parseable format. This option can be specified via ``buildtest buildspec find --terse``
   :type terse: bool, optional
   :param color: An instance of a string class that selects the color to use when printing table output
   :type color: str, optional
   :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
   :type count: int, optional
   :param row_count: Print total number of records from the table
   :type row_count: bool, optional
   :param search_buildspecs: List of buildspecs to search and add into cache. This can be file or directory and this argument contains value of ``buildtest buildspec find --search``
   :type search_buildspecs: list, optional

   .. py:attribute:: table

      

   .. py:attribute:: filter_fields
      :value: ['type', 'executor', 'tags', 'buildspec']

      

   .. py:attribute:: default_format_fields
      :value: ['name', 'type', 'executor', 'tags', 'description']

      

   .. py:attribute:: format_fields

      

   .. py:method:: get_cache()

      Returns cache file as loaded dictionary


   .. py:method:: load_paths()

      Add all paths to search for buildspecs. We read configuration file
      and check whether we need to load buildspecs from list of directories.
      We check if directories exist, if any fail we don't add them to path.
      If no root directories are specified we load buildspecs in
      `tutorials <https://github.com/buildtesters/buildtest/tree/devel/tutorials>`_
      and `general_tests <https://github.com/buildtesters/buildtest/tree/devel/general_tests>`_ directory.


   .. py:method:: build()

      This method will build buildspec cache file. If user requests to
      rebuild cache we remove the file and recreate cache. If cache file
      exists, we simply load from cache


   .. py:method:: _discover_buildspecs()

      This method retrieves buildspecs based on ``self.paths`` which is a
      list of directory paths to search. If ``--directory`` is specified
      we process each argument and recursively find all .yml files


   .. py:method:: _write_buildspec_cache()

      This method is responsible for writing buildspec cache to file


   .. py:method:: _validate_buildspecs(buildspecs)

      Given a list of buildspec files, validate each buildspec using :class:`buildtest.buildsystem.parser.BuildspecParser`
      class and return a list of valid buildspecs. Any invalid buildspecs are added to separate list

      :param buildspecs: A list of buildspec to validate


   .. py:method:: get_names()

      Return a list of test names found in buildspec cache. We only return test names for valid buildspecs


   .. py:method:: get_random_tests(num_items=1)

      Returns a list of random test names from the list of available test. The test are picked
      using `random.sample <https://docs.python.org/3/library/random.html#random.sample>`_

      :param num_items: Number of test items to retrieve
      :type num_items: int, optional


   .. py:method:: lookup_buildspec_by_name(name)

      Given an input test name, return corresponding buildspec file found in the cache.

      :param name: Name of test to query in cache
      :type name: str

      :returns: Return path to buildspec that contains name of test


   .. py:method:: build_cache()

      This method will rebuild the buildspec cache file by recursively searching
      all .yml files specified by input argument ``paths`` which is a list of directory paths.
      The buildspecs are validated and cache file is updated


   .. py:method:: _check_filter_fields()

      This method checks filter fields are valid. The filter fields are specified
      as ``buildtest buildspec find --filter <KEY1>=<VAL1>,<KEY2>=<VAL2>,...``

      :raises BuildTestError: If there is an invalid filter field


   .. py:method:: _check_format_fields()

      This method will check if all format fields are valid. Format fields
      are passed as comma separated fields: ``--format field1,field2,field3,...``

      :raises BuildTestError: If there is an invalid format field


   .. py:method:: _filter_buildspecs(executor, tags, schema_type)

      This method will return a boolean True/False that determines if
      buildspec test entry is skipped as part of filter process. The filter
      are done based on executor, tags, type field. ``True`` indicates test
      needs to be skipped.

      :param executor: ``executor`` property in buildspec
      :type executor: str
      :param tags: `List of tagnames specified via `tags`` property in buildspec
      :type tags: list
      :param schema_type: ``type`` property in buildspec
      :type schema_type: str

      :returns: Return True if there is **no** match otherwise returns False
      :rtype: bool


   .. py:method:: find_buildspecs()

      This method will find buildspecs based on cache content. We skip any
      tests based on executor filter, tag filter or type filter and build
      a table of tests that will be printed using ``print_buildspecs`` method.

      :raises BuildTestError: Raises exception if input buildspec for ``buildtest buildspec find --filter buildspec`` is invalid path or directory or buildspec not found in cache.


   .. py:method:: get_valid_buildspecs()

      Return a list of valid buildspecs


   .. py:method:: get_invalid_buildspecs()

      Return a list of invalid buildspecs


   .. py:method:: get_unique_tags()

      Return a list of unique tags.


   .. py:method:: get_unique_executors()

      Return a list of unique executors.


   .. py:method:: get_maintainers()

      Return a list of maintainers.


   .. py:method:: get_paths()

      Return a list of search paths


   .. py:method:: tag_breakdown()

      This method will return a breakdown of tags by test names.


   .. py:method:: executor_breakdown()

      This method will return a dictionary with breakdown of executors by test names.


   .. py:method:: test_breakdown_by_buildspec()

      This method will return a dictionary with breakdown of buildspecs by test names.


   .. py:method:: print_buildspecfiles(terse=None, header=None, row_count=None, count=None)

      This method implements ``buildtest buildspec find --buildspec`` which reports all buildspec files in cache.

      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional
      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional


   .. py:method:: print_tags(row_count=None, count=None, terse=None, header=None)

      This method implements ``buildtest buildspec find --tags`` which
      reports a list of unique tags from all buildspecs in cache file.

      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional
      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional


   .. py:method:: print_executors(row_count=None, count=None, terse=None, header=None)

      This method implements ``buildtest buildspec find --executors`` which reports all executors from cache.

      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional
      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional


   .. py:method:: print_by_executors(row_count=None, count=None, terse=None, header=None)

      This method prints executors by tests and implements ``buildtest buildspec find --group-by-executor`` command

      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional
      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional


   .. py:method:: print_by_tags(count=None, row_count=None, terse=None, header=None)

      This method prints tags by tests and implements ``buildtest buildspec find --group-by-tags`` command
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional
      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional


   .. py:method:: print_buildspecs(terse=None, header=None, quiet=None, row_count=None, count=None)

      Print buildspec table. This method is typically called when running ``buildtest buildspec find`` or options
      with ``--filter`` and ``--format``.

      :param terse: This argument will print output in terse format if ``--terse`` option is specified otherwise will print output in table format
      :type terse: bool, optional
      :param header: This argument controls whether header will be printed in terse format. If ``--terse`` option is not specified this argument has no effect. This argument holds the value of ``--no-header`` option
      :type header: bool, optional
      :param quiet: If this option is set we return immediately and don't anything. This is specified via ``buildtest buildspec find --quiet`` which can be useful when rebuilding cache without displaying output
      :type quiet: bool, optional
      :param row_count: Print total number of records from the table
      :type row_count: bool, optional
      :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec find --count``
      :type count: int, optional


   .. py:method:: list_maintainers()

      Return a list of maintainers


   .. py:method:: print_maintainer(row_count=None, terse=None, pager=None, count=None)

      This method prints maintainers from buildspec cache file which implements ``buildtest buildspec maintainers`` command.


   .. py:method:: print_maintainers_find(name)

      Display a list of buildspec files associated to a given maintainer. This command is used when running
      ``buildtest buildspec maintainers find``

      :param name: Name of maintainer specified via ``buildtest buildspec maintainers find <name>``
      :type name: str


   .. py:method:: print_maintainers_by_buildspecs()

      This method prints maintainers breakdown by buildspecs. This method implements ``buildtest buildspec maintainers --breakdown``.


   .. py:method:: print_invalid_buildspecs(error=None, terse=None, row_count=None)

      Print invalid buildspecs from cache file. This method implements command ``buildtest buildspec find invalid``

      :param error: Display error messages for invalid buildspecs. Default is ``False`` where we only print list of invalid buildspecs
      :type error: bool, optional
      :param terse: Display output in machine readable format.
      :type terse: bool, optional
      :param row_count: Display row count of invalid buildspces table
      :type row_count: bool, optional


   .. py:method:: print_filter_fields()

      This method prints filter fields available for buildspec cache. This
      method implements command ``buildtest buildspec find --helpfilter``


   .. py:method:: print_format_fields()

      This method prints format fields available for buildspec cache. This
      method implements command ``buildtest buildspec find --helpformat``


   .. py:method:: print_raw_filter_fields()

      This method prints the raw filter fields available for buildspec cache. This
      method implements command ``buildtest buildspec find --filterfields``


   .. py:method:: print_raw_format_fields()

      This method prints the raw format fields available for buildspec cache. This
      method implements command ``buildtest buildspec find --formatfields``


   .. py:method:: print_paths()

      This method print buildspec paths, this implements command ``buildtest buildspec find --paths``



.. py:function:: edit_buildspec_test(test_names, configuration, editor)

   Open a list of test names in editor mode defined by ``EDITOR`` environment otherwise resort to ``vim``.
   This method will search for buildspec cache and find path to buildspec file corresponding to test name and open
   file in editor. If multiple test are specified via ``buildtest buildspec edit-test`` then each file will be open and
   upon closing file, the next file will be open for edit until all files are written.

   :param test_names: A list of test names to open in editor
   :type test_names: list
   :param configuration: An instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration
   :param editor: Path to editor to use when opening file
   :type editor: str


.. py:function:: edit_buildspec_file(buildspecs, configuration, editor)

   Open buildspec in editor and validate buildspec with parser. This method is invoked by command ``buildtest buildspec edit-file``.

   :param buildspec: Path to buildspec file to edit
   :type buildspec: str
   :param configuration: An instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration
   :param editor: Path to editor to use when opening file
   :type editor: str


.. py:function:: is_test_name_in_cache(test_name, cache)

   Check if a test name is in the cache.

   :param test_name: The test name to check.
   :type test_name: str
   :param cache: An instance of BuildspecCache used for storing the buildspec cache
   :type cache: BuildspecCache

   :returns: True if the test name is in the cache, False otherwise.
   :rtype: bool


.. py:function:: show_buildspecs(test_names, configuration, theme=None)

   This is the entry point for ``buildtest buildspec show`` command which will print content of
   buildspec based on name of test.

   :param test_names: List of test names to show content of file
   :type test_names: list
   :param configuration: Instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration
   :param theme: Color theme to choose. This is the Pygments style (https://pygments.org/docs/styles/#getting-a-list-of-available-styles) which is specified by ``--theme`` option
   :type theme: str, optional


.. py:function:: show_tests_by_status(configuration, status, test_names=None, report_file=None, theme=None)

   This method which will print content of
   buildspec given test names for a desired status. The ``status`` argument can be **FAIL** or **PASS**
   which will be used to search in report file and extract test names based on status and then determine
   the corresponding buildspec that generated the test.

   :param configuration: Instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration
   :param status: Status of the tests to show ('FAIL' or 'PASS').
   :type status: str
   :param test_names: List of test names to show content of file
   :type test_names: list, optional
   :param report_file: Full path to report file to read
   :type report_file: str, optional
   :param theme: Color theme to choose. This is the Pygments style (https://pygments.org/docs/styles/#getting-a-list-of-available-styles) which is specified by ``--theme`` option
   :type theme: str, optional


.. py:function:: show_failed_buildspecs(configuration, test_names=None, report_file=None, theme=None)

   This is the entry point for ``buildtest buildspec show-fail`` command which will print content of
   buildspec on name of all failed tests if a list of test names are not specified

   :param configuration: Instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration
   :param test_names: List of test names to show content of file
   :type test_names: list, optional
   :param report_file: Full path to report file to read
   :type report_file: str, optional
   :param theme: Color theme to choose. This is the Pygments style (https://pygments.org/docs/styles/#getting-a-list-of-available-styles) which is specified by ``--theme`` option
   :type theme: str, optional


.. py:function:: handle_exception(buildspec, exception)

   Handle exceptions during buildspec validation.


.. py:function:: buildspec_validate_command(configuration, buildspecs=None, excluded_buildspecs=None, tags=None, executors=None, name=None)

   Entry point for ``buildtest buildspec validate``. This method is responsible for discovering buildspec
   with same options used for building buildspecs that includes ``--buildspec``, ``--exclude``, ``--tag``, and
   ``--executor``. Upon discovery we pass each buildspec to ``BuildspecParser`` class to validate buildspec and
   report any errors during validation which is raised as exceptions.

   :param configuration: An instance of SiteConfiguration class which is the loaded buildtest configuration used for validating the buildspecs.
   :type configuration: buildtest.config.SiteConfiguration
   :param buildspecs: List of paths to buildspec file which can be a file or directory. This option is specified via ``buildtest buildspec validate --buildspec``
   :type buildspecs: list, optional
   :param excluded_buildspecs: List of excluded buildspecs which can be a file or directory. This option is specified via ``buildtest buildspec validate --exclude``
   :type excluded_buildspecs: list, optional
   :param tags: List of tag names to search for buildspec to validate. This option is specified via ``buildtest buildspec validate --tag``
   :type tags: list, optional
   :param executors: List of executor names to search for buildspecs to validate. This option is specified via ``buildtest buildspec validate --executor``
   :type executors: list, optional
   :param name: Name of test to validate. This option is specified via ``buildtest buildspec validate --name``
   :type name: str, optional


.. py:function:: summarize_buildspec_cache(pager, configuration, color=None)

   This is a helper method used for printing output of ``buildtest buildspec summary`` with and without
   pagination

   :param configuration: instance of type SiteConfiguration
   :type configuration: buildtest.config.SiteConfiguration
   :param pager: Boolean control output of summary with paging
   :type pager: bool
   :param color: An instance of str, color that the summary should be printed in
   :type color: str, optional


.. py:function:: summary_print(configuration, color=None)

   This method will print summary of buildspec cache file. This method is the core logic
   used for showing output of command ``buildtest buildspec summary``.

   :param configuration: instance of type SiteConfiguration
   :type configuration: buildtest.config.SiteConfiguration
   :param color: An instance of str, color that the summary should be printed in
   :type color: str, optional


.. py:function:: buildspec_maintainers(configuration, breakdown=None, terse=None, header=None, color=None, name=None, row_count=None, count=None, pager=None)

   Entry point for ``buildtest buildspec maintainers`` command.

   :param configuration: instance of type SiteConfiguration
   :type configuration: buildtest.config.SiteConfiguration
   :param terse: Print in terse mode
   :type terse: bool, optional
   :param header: If True disable printing of headers
   :type header: bool, optional
   :param color: Print output of table with selected color
   :type color: bool, optional
   :param name: List all buildspecs corresponding to maintainer name. This command is specified via ``buildtest buildspec maintainers find <name>``
   :type name: str, optional
   :param row_count: Print row count of the maintainer table. This command is specified via ``buildtest --row-count buildspec maintainers -l``
   :type row_count: bool, opotional
   :param count: Number of entries to display in output. This argument contains value of ``buildtest buildspec maintainers --count``
   :type count: int, optional
   :param pager: Enable paging of output
   :type pager: bool, optional


.. py:function:: buildspec_find(args, configuration)

   Entry point for ``buildtest buildspec find`` command

   :param args: Parsed arguments from `ArgumentParser.parse_args <https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args>`_
   :type args: dict
   :param configuration: instance of type SiteConfiguration
   :type configuration: buildtest.config.SiteConfiguration


.. py:function:: open_buildspec_in_editor(buildspec, editor)

   Open a buildspec in the specified editor and print a message.

   :param buildspec: The path to the buildspec file to open.
   :type buildspec: str
   :param editor: The editor to open the buildspec file in.
   :type editor: str


.. py:function:: validate_buildspec(buildspec, configuration)

   Validate a buildspec with JSON Schema and print whether it is valid or not.

   :param buildspec: Path to buildspec file to validate
   :type buildspec: str
   :param configuration: An instance of SiteConfiguration class
   :type configuration: buildtest.config.SiteConfiguration