pyTooling.Sphinx.Unittest

A directive rendering a unit test report as a table: per testsuite and testcase, the counts and the runtime.

The reports are read by pyEDAA.Reports: a JUnit XML file in any dialect - e.g. the reports of pytest or OSVVM -, or a test report in pyTooling’s own XML format, recognized by its root element <TestReport>. The report files are declared in conf.py under pyTooling_Unittest_Testsuites, each with an identifier a directive names in its :reportid: option. An entry with the key pages gets a page per testsuite and testcase, generated below that document name (see pyTooling.Sphinx.UnittestPages):

# doc/conf.py
pyTooling_Unittest_Testsuites = {
  "src": {
    "xml_report": "../report/unit/unittest.xml",
    "pages":      "unittests/src",
  }
}

The reports are read once, when the builder is initialized, and shared by every directive and page using them.

See also

report:unittest-summary

→ The directive’s options and configuration, with examples.

Variables

Classes

Variables

pyTooling.Sphinx.Unittest.CONFIG_PREFIX

Prefix every configuration value of the unit test directive carries in conf.py.

'pyTooling_Unittest'
pyTooling.Sphinx.Unittest.REPORT_FORMATS

The report formats, by the root element of a report file.

{'TestReport': 'pyTooling', 'testsuite': 'JUnit', 'testsuites': 'JUnit'}

Classes

class pyTooling.Sphinx.Unittest.TestsuiteConfiguration[source]

An entry of pyTooling_Unittest_Testsuites, after UnittestSummary.CheckConfiguration() read it.

Inheritance

Inheritance diagram of TestsuiteConfiguration

xml_report: Path

The unit test report, in JUnit XML or pyTooling’s XML format.

pages: str | None

Document name the pages per testsuite and testcase are generated below, if any.

__contains__(key, /)

True if the dictionary has the specified key, else False.

__delitem__(key, /)

Delete self[key].

__eq__(value, /)

Return self==value.

__ge__(value, /)

Return self>=value.

__getitem__(key, /)

Return self[key].

__gt__(value, /)

Return self>value.

__init__(*args, **kwargs)
__ior__(value, /)

Return self|=value.

__iter__()

Implement iter(self).

__le__(value, /)

Return self<=value.

__len__()

Return len(self).

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

classmethod __new__(*args, **kwargs)
__or__(value, /)

Return self|value.

__repr__()

Return repr(self).

__reversed__()

Return a reverse iterator over the dict keys.

__ror__(value, /)

Return value|self.

__setitem__(key, value, /)

Set self[key] to value.

__sizeof__()

Return the size of the dict in memory, in bytes.

clear()

Remove all items from the dict.

copy()

Return a shallow copy of the dict.

classmethod fromkeys(iterable, value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items()

Return a set-like object providing a view on the dict’s items.

keys()

Return a set-like object providing a view on the dict’s keys.

pop(k[, d]) → v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None.  Update D from mapping/iterable E and F.

If E is present and has a .keys() method, then does: for k in E.keys(): D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

Return an object providing a view on the dict’s values.

class pyTooling.Sphinx.Unittest.ShowTestcases(*values)[source]

Which testcases a unit test summary lists, by their status.

A member compares equal to a TestcaseStatus it includes.

Inheritance

Inheritance diagram of ShowTestcases

passed = 1

Passed testcases, and testcases failing as expected.

failed = 2

Failed testcases, and testcases passing unexpectedly.

skipped = 4

Skipped testcases.

excluded = 8

Excluded testcases.

errors = 16

Errored testcases, and testcases whose setup failed.

aborted = 32

Aborted testcases.

all = 63

Every testcase.

not_passed = 62

Every testcase but the passed ones.

__eq__(other)[source]

Check whether a testcase status is included.

Parameters:

other (Any) – A testcase status.

Return type:

bool

Returns:

True if other is a TestcaseStatus this member includes.

class pyTooling.Sphinx.Unittest.UnittestSummary(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)[source]

The report:unittest-summary directive: a table of a unit test report, per testsuite and testcase.

Inheritance

Inheritance diagram of UnittestSummary

directiveName: str = 'unittest-summary'

Name the directive is invoked by.

has_content = False

A boolean; True if content is allowed.

required_arguments = 0

Number of required directive arguments.

optional_arguments = 6

Number of optional arguments.

option_spec: dict[str, Any] = {'class': <function strip>, 'hide-testsuite-summary': <function flag>, 'no-assertions': <function flag>, 'reportid': <function stripAndNormalize>, 'show-testcases': <function stripAndNormalize>, 'testsuite-summary-name': <function strip>}

Mapping of option names to validator functions.

configValues: ClassVar[dict[str, tuple[Any, str, Any]]] = {'Testsuites': ({}, 'env', <class 'dict'>)}

Values added to conf.py, as name: (default, rebuild, types), prefixed by CONFIG_PREFIX.

_testSummaries: ClassVar[dict[str, TestsuiteConfiguration]] = {'example': {'pages': 'unittests/example', 'xml_report': PosixPath('../tests/data/Report/unittest.xml')}, 'pytooling': {'pages': 'unittests/pytooling', 'xml_report': PosixPath('../tests/data/Report/TestReport.xml')}, 'src': {'pages': 'unittests/src', 'xml_report': PosixPath('../report/unit/unittest.xml')}}

Report configurations, by report ID.

_reports: ClassVar[dict[str, TestsuiteSummary]] = {'example': <pyEDAA.Reports.Unittesting.TestsuiteSummary object>, 'pytooling': <pyEDAA.Reports.Unittesting.pyTooling.Document object>, 'src': <pyEDAA.Reports.Unittesting.TestsuiteSummary object>}

Reports read by ReadReports(), by ID.

_readErrors: ClassVar[dict[str, Exception]] = {}

Why a report couldn’t be read, by report ID.

_cssClasses: list[str]

Additional CSS classes of the table.

_reportID: str

Identifier of the report in the configuration.

_noAssertions: bool

Whether the assertion column is left out.

_hideTestsuiteSummary: bool

Whether the testsuite summary’s row is left out.

_testsuiteSummaryName: str | None

Name replacing the testsuite summary’s name, or "".

_showTestcases: ShowTestcases

Which testcases are listed.

_xmlReport: Path

The unit test report.

_testsuite: TestsuiteSummary

The report’s testsuite summary.

_CheckOptions()[source]

Parse all directive options or use default values.

Raises:

  • SphinxExtensionError – If option :class: isn’t a list of CSS class names.

  • SphinxExtensionError – If option :show-testcases: is neither all nor not-passed.

  • SphinxExtensionError – If option :reportid: is missing.

  • ReportExtensionError – If pyTooling_Unittest_Testsuites has no entry for the report ID.

Return type:

None

classmethod CheckConfiguration(sphinxApplication, sphinxConfiguration)[source]

Check the configuration value and load the report configurations.

Parameters:

  • sphinxApplication (Sphinx) – Sphinx application instance.

  • sphinxConfiguration (Config) – Sphinx configuration instance.

Return type:

None

classmethod ReadReports(sphinxApplication)[source]

Read every configured unit test report, once per build.

A report that can’t be read keeps the exception, which a directive or a report’s pages show when they need it.

Parameters:

sphinxApplication (Sphinx) – Sphinx application instance.

Return type:

None

classmethod GetReport(reportID)[source]

Return a report read by ReadReports().

Parameters:

reportID (str) – Identifier of the report.

Return type:

TestsuiteSummary

Returns:

The report’s testsuite summary.

Raises:

ReportExtensionError – If the report wasn’t read, chained to the exception reading it raised.

static _ReadReport(xmlReport)[source]

Read a report file, choosing the reader by the file’s root element.

Parameters:

xmlReport (Path) – The report file.

Return type:

TestsuiteSummary

Returns:

The report’s testsuite summary, aggregated.

Raises:

  • ReportExtensionError – If the file’s root element is no known report format.

  • ReportsPackageMissingError – If pyEDAA.Reports, or its reader for the report’s format, isn’t installed.

classmethod _CheckConfiguration(sphinxConfiguration)[source]

Check and load the report configurations from pyTooling_Unittest_Testsuites.

Parameters:

sphinxConfiguration (Config) – Sphinx configuration instance.

Raises:

  • ReportExtensionError – If the configuration value isn’t registered.

  • ReportExtensionError – If a report configuration has no xml_report.

  • ReportExtensionError – If the xml_report file doesn’t exist.

  • ReportExtensionError – If pages isn’t a string.

  • ReportExtensionError – If pages isn’t a relative document name.
    Use a name like ‘unittests/src’, separated by ‘/’, without ‘.’ or ‘..’.

Return type:

None

_SortedValues(d)[source]

Yield the values of a mapping, sorted by key.

Parameters:

d (Mapping[str, Any]) – The mapping, e.g. a testsuite’s testcases by name.

Return type:

Generator[Any]

Returns:

A generator of the values, sorted by their keys.

static _ConvertTestcaseStatusToSymbol(status)[source]

Return the symbol shown for a testcase’s status.

Parameters:

status (TestcaseStatus) – The testcase’s status.

Return type:

str

Returns:

An emoji, e.g. ✅ for a passed testcase.

static _ConvertTestsuiteStatusToSymbol(status)[source]

Return the symbol shown for a testsuite’s status.

Parameters:

status (TestsuiteStatus) – The testsuite’s status.

Return type:

str

Returns:

An emoji, e.g. ✅ for a passed testsuite.

static _FormatTimedelta(delta)[source]

Format a duration as HH:MM:SS.sss, rounded to milliseconds.

Parameters:

delta (timedelta | None) – The duration, or None.

Return type:

str

Returns:

The formatted duration, or "" for None.

_GenerateTestSummaryTable()[source]

Build the table: a row per testsuite and listed testcase, and a summary row.

Return type:

table

Returns:

The table.

_RenderRoot(tableBody, testsuiteSummary, includeRoot=True, testsuiteSummaryName=None)[source]

Add the rows of a testsuite summary: optionally its own row, its testsuites’ rows, and the summary row.

Parameters:

  • tableBody (tbody) – The table body the rows are added to.

  • testsuiteSummary (TestsuiteSummary) – The testsuite summary.

  • includeRoot (bool) – Optional, whether the testsuite summary gets a row of its own. Default: True.

  • testsuiteSummaryName (str | None) – Optional, name replacing the testsuite summary’s name; "" keeps it. Default: None.

Return type:

None

_RenderTestsuite(tableBody, testsuite, level)[source]

Add a row for a testsuite, then - recursively - rows for its testsuites and its listed testcases.

Parameters:

  • tableBody (tbody) – The table body the rows are added to.

  • testsuite (Testsuite) – The testsuite.

  • level (int) – The testsuite’s depth in the hierarchy, shown by indentation.

Return type:

None

_RenderTestcase(tableBody, testcase, level)[source]

Add a row for a testcase.

Parameters:

  • tableBody (tbody) – The table body the row is added to.

  • testcase (Testcase) – The testcase.

  • level (int) – The testcase’s depth in the hierarchy, shown by indentation.

Return type:

None

_NameEntry(level, state, entity, roleName)[source]

Create the cell naming a testsuite or testcase, indented, and linked to its page if the report has pages.

Parameters:

  • level (int) – The number of indentation steps.

  • state (str) – The status symbol written before the name.

  • entity (Testsuite | Testcase) – The testsuite or testcase.

  • roleName (str) – The role referring to it: ts for a testsuite, tc for a testcase.

Return type:

entry

Returns:

The table cell.

_RenderSummary(tableBody, testsuiteSummary)[source]

Add the summary row: the overall status and counts.

Parameters:

  • tableBody (tbody) – The table body the row is added to.

  • testsuiteSummary (TestsuiteSummary) – The testsuite summary.

Return type:

None

run()[source]

Read the configured report and return it as a table, on landscape pages in LaTeX.

A report with pages adds a hidden table of contents of its top-level testsuites, so the pages are below this document in the navigation.

Return type:

list[Node]

Returns:

A Landscape container holding the table, or the error message; then the table of contents, if the report has pages.

_CreateDoubleRowTableHeader(columns, identifier, classes)

Create a table whose header spans two rows, so a column can group sub-columns.

A column’s subColumns is None when it spans both header rows, and otherwise holds the (title, width) pairs below it.

Parameters:
Return type:

tgroup

Returns:

The table’s column group, with both header rows already in it.

_CreateRotatedTableHeader(columns, identifier, classes)

Create a table whose header titles are rotated, for many narrow columns.

Parameters:

  • columns (list[tuple[str, list[str] | None]]) – One (title, classes) pair per column; the classes are put on the header cell.

  • identifier (str) – Identifier of the table.

  • classes (list[str]) – CSS classes to put on the table.

Return type:

tgroup

Returns:

The table’s column group, with the header row already in it.

_CreateSingleRowTableHeader(columns, identifier, classes)

Create a table with a single header row.

Parameters:

  • columns (list[tuple[str, int | None]]) – One (title, width) pair per column; a width of None leaves it to the writer.

  • identifier (str) – Identifier of the table.

  • classes (list[str]) – CSS classes to put on the table.

Return type:

tgroup

Returns:

The table’s column group, with the header row already in it.

_ParseBooleanOption(optionName, default=None)

Read an option written as yes/true or no/false.

Parameters:

  • optionName (str) – Name of the option to read.

  • default (bool | None) – Optional, the value to return when the option wasn’t given.

Return type:

bool

Returns:

The option’s value.

Raises:

  • SphinxExtensionError – If the option wasn’t given and has no default.

  • SphinxExtensionError – If the option’s value is neither of the two accepted spellings.

_ParseEnumOption(optionName, enumType, default=None)

Read an option naming a member of an enumeration.

The written value is lowered and its dashes become underscores, so horizontal-table in a document selects the horizontal_table member - a document reads in the spelling documents use, and the enumeration keeps the spelling Python uses.

Parameters:

  • optionName (str) – Name of the option to read.

  • enumType (type[_EnumType]) – The enumeration whose members the value is looked up in.

  • default (_EnumType | None) – Optional, the member to return when the option wasn’t given.

Return type:

_EnumType

Returns:

The named member of the enumeration.

Raises:

  • SphinxExtensionError – If the option wasn’t given and has no default.

  • SphinxExtensionError – If the value names no member of the enumeration.

_ParseStringOption(optionName, default=None, regexp='\\w+')

Read an option that has to match a regular expression.

The pattern defaults to one or more word characters.

Parameters:

  • optionName (str) – Name of the option to read.

  • default (str | None) – Optional, the value to return when the option wasn’t given.

  • regexp (str) – Optional, the pattern the value has to match.

Return type:

str

Returns:

The option’s value.

Raises:

  • SphinxExtensionError – If the option wasn’t given and has no default.

  • SphinxExtensionError – If the option’s value doesn’t match the pattern.

__init__(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)
Return type:

None

_internalError(container, location, message, exception)

Report an exception a directive couldn’t recover from, in the log and on the page.

A directive that fails silently leaves a hole in the documentation that nobody notices. This puts the message where a reader sees it and the traceback where a maintainer does. The exception’s notes - e.g. how to install a missing package - are logged below it.

Parameters:

  • container (container) – The container the message is put into.

  • location (str) – Name of the logger, which is what the log line is attributed to.

  • message (str) – What went wrong, in one sentence.

  • exception (Exception) – The exception that was caught.

Return type:

list[Node]

Returns:

The container, as the list a directive’s run returns.

_object_hierarchy_parts(sig_node)

Returns a tuple of strings, one entry for each part of the object’s hierarchy (e.g. ('module', 'submodule', 'Class', 'method')). The returned tuple is used to properly nest children within parents in the table of contents, and can also be used within the _toc_entry_name() method.

This method must not be used outwith table of contents generation.

Return type:

tuple[str, ...]

Parameters:

sig_node (desc_signature)

_toc_entry_name(sig_node)

Returns the text of the table of contents entry for the object.

This function is called once, in run(), to set the name for the table of contents entry (a special attribute _toc_name is set on the object node, later used in environment.collectors.toctree.TocTreeCollector.process_doc().build_toc() when the table of contents entries are collected).

To support table of contents entries for their objects, domains must override this method, also respecting the configuration setting toc_object_entries_show_parents. Domains must also override _object_hierarchy_parts(), with one (string) entry for each part of the object’s hierarchy. The result of this method is set on the signature node, and can be accessed as sig_node['_toc_parts'] for use within this method. The resulting tuple is also used to properly nest children within parents in the table of contents.

An example implementations of this method is within the python domain (PyObject._toc_entry_name()). The python domain sets the _toc_parts attribute within the handle_signature() method.

Return type:

str

Parameters:

sig_node (desc_signature)

add_name(node)

Append self.options[‘name’] to node[‘names’] if it exists.

Also normalize the name string and register it as explicit target.

Return type:

None

add_target_and_index(name, sig, signode)

Add cross-reference IDs and entries to self.indexnode, if applicable.

name is whatever handle_signature() returned.

Return type:

None

Parameters:

  • name (ObjDescT)

  • sig (str)

  • signode (desc_signature)

after_content()

Called after parsing content.

Used to reset information about the current directive context on the build environment.

Return type:

None

assert_has_content()

Throw an ERROR-level DirectiveError if the directive doesn’t have contents.

before_content()

Called before parsing content.

Used to set information about the current directive context on the build environment.

Return type:

None

property config: Config

Reference to the Config object.

Added in version 1.8.

directive_error(level, message)

Return a DirectiveError suitable for being thrown as an exception.

Call “raise self.directive_error(level, message)” from within a directive implementation to return one single system message at level level, which automatically gets the directive block and the line number added.

Preferably use the debug, info, warning, error, or severe wrapper methods, e.g. self.error(message) to generate an ERROR-level directive error.

property env: BuildEnvironment

Reference to the BuildEnvironment object.

Added in version 1.8.

final_argument_whitespace = False

A boolean; True if the last argument may contain spaces.

get_location()

Get current location info for logging.

Added in version 4.2.

Return type:

str

get_signatures()

Retrieve the signatures to document from the directive arguments.

By default, signatures are given as arguments, one per line.

Return type:

list[str]

get_source_info()

Get source and line number.

Added in version 3.0.

Return type:

tuple[str | None, int | None]

handle_signature(sig, signode)

Parse the signature sig.

The individual nodes are then appended to signode. If ValueError is raised, parsing is aborted and the whole sig is put into a single desc_name node.

The return value should be a value that identifies the object. It is passed to add_target_and_index() unchanged, and otherwise only used to skip duplicates.

Return type:

str

Parameters:

  • sig (str)

  • signode (desc_signature)

parse_content_to_nodes(allow_section_headings=False)

Parse the directive’s content into nodes.

Parameters:

allow_section_headings (bool) – Are titles (sections) allowed in the directive’s content? Note that this option bypasses Docutils’ usual checks on doctree structure, and misuse of this option can lead to an incoherent doctree. In Docutils, section nodes should only be children of Structural nodes, which includes document, section, and sidebar nodes.

Return type:

list[Node]

Added in version 7.4.

parse_inline(text, *, lineno=-1)

Parse text as inline elements.

Parameters:

  • text (str) – The text to parse, which should be a single line or paragraph. This cannot contain any structural elements (headings, transitions, directives, etc).

  • lineno (int) – The line number where the interpreted text begins.

Return type:

tuple[list[Node], list[system_message]]

Returns:

A list of nodes (text and inline elements) and a list of system_messages.

Added in version 7.4.

parse_text_to_nodes(text='', /, *, offset=-1, allow_section_headings=False)

Parse text into nodes.

Parameters:

  • text (str) – Text, in string form. StringList is also accepted.

  • allow_section_headings (bool) – Are titles (sections) allowed in text? Note that this option bypasses Docutils’ usual checks on doctree structure, and misuse of this option can lead to an incoherent doctree. In Docutils, section nodes should only be children of Structural nodes, which includes document, section, and sidebar nodes.

  • offset (int) – The offset of the content.

Return type:

list[Node]

Added in version 7.4.

set_source_info(node)

Set source and line number to the node.

Added in version 2.1.

Return type:

None

Parameters:

node (Node)

transform_content(content_node)

Can be used to manipulate the content.

Called after creating the content through nested parsing, but before the object-description-transform event is emitted, and before the info-fields are transformed.

Return type:

None

Parameters:

content_node (desc_content)