Unit Test Summary

The report:unittest-summary directive generates a unittest report summary table. The unittest report file(s) need to be configured in Sphinx’s conf.py for pre-analysis and data aggregation (see below for details). This also allows the directive to supports multiple unittest reports per Sphinx documentation. Each unittest report is referenced by the report option, which matches the dictionary key used in the configuration file.

Minimal Example

Unittest.rst

.. report:unittest-summary::
   :reportid: src

Options

Additional options are offered for fine-tuning and styling of the report:

reportid: Reference the unittest report file and settings as listed in conf.py.
testsuite-summary-name (optional): Override the TestsuiteSummary’s name.
show-testcases (optional): Select if all testcases or only flawed testcases should be listed per testsuite.
no-assertions (optional): If flag is present, no assertion column will be shown.
hide-testsuite-summary (optional): If this flag is present, hide the summary row.
class (optional): User-defined CSS class name(s), which are applied on the HTML table.

Supported Unittest Formats

Reading unittest summary reports in XML format is based on pyEDAA.Reports. It supports Any JUnit XML files and derived dialects. So any XML file format can be used to visualize the results in a Sphinx documentation.

Explicitly, the following report sources have been tested:

Future Ideas

It’s planned to display a per testsuite and maybe per testcase report on a separate Sphinx document (separate HTML page).

Only Testsuites

Testsuites and Testcases

Configuration Entries in `conf.py`

See the overview page on how to setup and enable the sphinx-reports extension in general.

Configure one or more unittest reports in conf.py by adding a new section defining some configuration variables. Each unittest report is identified by an ID, which is later referred to by the report directive or legend directive. Here, the ID is called src (dictionary key). Each analysis report needs 4 configuration entries:

name: Name of the Python package.
xml_report: The unittest report as XML file.

Configuration

# ==============================================================================
# Sphinx-reports - Unittest
# ==============================================================================
report_codecov_packages = {
   "src": {
      "name":       "myPackage",
      "xml_report": "../report/unit/unittest.xml",
   }
}

Example Document

he following Unittest.rst document is an example on how to use the report:unittest-summary directive. The first file consists of three parts:

A page title (headline)
The report:code-coverage directive to generate the report table.
A footer

The second file shows how to integrate that document into the navigation bar / toc-tree.

Unittest.rst

Unittest Summary Report
#######################

.. report:unittest-summary::
   :reportid: src
   :show-testcases: not-passed
   :no-assertions:

----------

Unittest report generated with `pytest <https://github.com/pytest-dev/pytest>`__ and visualized by
`sphinx-reports <https://github.com/pyTooling/sphinx-reports>`__.

index.rst

.. toctree::
   :caption: References and Reports
   :hidden:

   Python Class Reference <sphinx_reports/sphinx_reports>
   Unittest
   CodeCoverage
   Doc. Coverage Report <DocCoverage>
   Static Type Check Report ➚ <typing/index>

.. toctree::
   :caption: Appendix
   :hidden:

Sphinx Directives

The following directives are provided for visualizing unittest reports.

.. report:unittest-summary::

Generate a table summarizing the unittest results per testsuite and testcase. The testsuite hierarchy is visualized by indentation.

:class:

Optional: A list of space separated user-defined CSS class names.

The CSS classes are applied on the HTML <table> tag.

:reportid:: An identifier referencing a dictionary entry (key) in the configuration variable report_unittest_testsuites defined in conf.py.

:testsuite-summary-name:: Optional: Override the TestsuiteSummary’s name.

:show-testcases:: Optional: Select if all testcases (all) or only flawed testcases (not-passed) should be listed per testsuite.

:no-assertions:: Optional: If flag is present, no assertion column will be shown.

:hide-testsuite-summary:: Optional: if this flag is present, hide the summary row.

Sphinx Roles

There are no roles defined.

Custom CSS Styling

Table Styling

The table-tag has 2 additional CSS classes:

report-unittest-table: Allows selecting the table tag, but only for unittest reports.
report-unittest-%reportid%: Allows selecting one specific unittest report. %reportid% gets replaced by the reportid used in the option field of the directive. Here it got replaced by src.

Row Styling

The tr-tag (table row) has 2 additional CSS classes:

report-testsuite/report-testcase/report-summary: This class indicated if the row refers to a testsuite, testcase or the overall coverage summary (last row).
report-%status%: Depending on the testsuite or testcase status, a representing CSS class is added.

Generated HTML Code (condensed)

<table class="report-unittest-table report-unittest-src">
  <thead>
    <tr>
      <th> ..... </th>
      .....
      <th> ..... </th>
    </tr>
  </thead>
  <tbody>
    <tr class="report-testsuite report-passed"> ..... </tr>
    <tr class="report-testcase report-skipped"> ..... </tr>
    .....
    <tr class="report-summary report-passed"> ..... </tr>
  </tbody>
</table>

Example CSS Rules

table.report-unittest-table > thead > tr {
   background: #ebebeb;
}

table.report-unittest-table > tbody > tr.report-summary.report-passed {
   font-weight: bold;
}