CodeCoveragePages.py

Summary

Lines:

166 of 173 covered (96.0%)

Excluded lines:

2

Branches:

52 of 54 taken (96.3%), 2 lines partially

Coverage:

96.0%

Directory:

Sphinx

Missed lines:

162-164, 238-244

Source

pyTooling/Sphinx/CodeCoveragePages.pycoveredpartialuncoveredexcludednot executable
1# ==================================================================================================================== #
2#             _____           _ _               ____        _     _                                                    #
3#  _ __  _   |_   _|__   ___ | (_)_ __   __ _  / ___| _ __ | |__ (_)_ __ __  __                                        #
4# | '_ \| | | || |/ _ \ / _ \| | | '_ \ / _` | \___ \| '_ \| '_ \| | '_ \\ \/ /                                        #
5# | |_) | |_| || | (_) | (_) | | | | | | (_| |_ ___) | |_) | | | | | | | |>  <                                         #
6# | .__/ \__, ||_|\___/ \___/|_|_|_| |_|\__, (_)____/| .__/|_| |_|_|_| |_/_/\_\                                        #
7# |_|    |___/                          |___/        |_|                                                               #
8# ==================================================================================================================== #
9# Authors:                                                                                                             #
10#   Patrick Lehmann                                                                                                    #
11#                                                                                                                      #
12# License:                                                                                                             #
13# ==================================================================================================================== #
14# Copyright 2026-2026 Patrick Lehmann - Bötzingen, Germany                                                             #
15#                                                                                                                      #
16# Licensed under the Apache License, Version 2.0 (the "License");                                                      #
17# you may not use this file except in compliance with the License.                                                     #
18# You may obtain a copy of the License at                                                                              #
19#                                                                                                                      #
20#   http://www.apache.org/licenses/LICENSE-2.0                                                                         #
21#                                                                                                                      #
22# Unless required by applicable law or agreed to in writing, software                                                  #
23# distributed under the License is distributed on an "AS IS" BASIS,                                                    #
24# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                                             #
25# See the License for the specific language governing permissions and                                                  #
26# limitations under the License.                                                                                       #
27#                                                                                                                      #
28# SPDX-License-Identifier: Apache-2.0                                                                                  #
29# ==================================================================================================================== #
30#
31"""
32A page per directory and per source file of a code coverage report, generated without a source file.
33
34A report declared in ``pyTooling_CodeCoverage_Packages`` with the key ``pages`` gets a document per directory and per
35source file, named below that key's document name by its path in the report, e.g. ``<pages>/src/Counter.vhdl``. A
36chain of directories holding no file and one directory each is one page, e.g. ``pyTooling/Sphinx`` (see
37:func:`~pyTooling.Sphinx.CodeCoverage.compactDirectory`).
38
39* A directory's page shows its counters, and tables of its directories and files, each linked to its page.
40* A source file's page shows its counters, its missed lines, and its listing: the source read from the report's
41  ``sources`` directory, syntax highlighted with the Pygments lexer of the file's name, each line marked by its
42  coverage state.
43
44The pages are built by the machinery of :class:`~pyTooling.Sphinx.Pages.ReportPages`, so they are in the navigation
45below the directive ``report:code-coverage``. Every directory and file with a page is an object of domain ``report``,
46referred to by the role ``:cov:`` by its path - e.g. ``:cov:`src/Counter.vhdl``` -, which may end in ``#<line>`` to
47link to a line of a file's listing.
48
49.. seealso::
50
51   :ref:`DIR/CodeCoverage/Pages`
52      |rarr| The pages and the role, with examples.
53   :mod:`pyTooling.Sphinx.Pages`
54      |rarr| The pages of a report: document names, registration, generation and navigation.
55"""
56from __future__                    import annotations
57
58from pathlib                       import Path
59from typing                        import TYPE_CHECKING, ClassVar, Iterable, Optional as Nullable, Union
60
61from docutils                      import nodes
62from pygments.lexers               import get_lexer_for_filename
63from pygments.lexers.special       import TextLexer
64from pygments.token                import STANDARD_TYPES, _TokenType
65from pygments.util                 import ClassNotFound
66from sphinx.application            import Sphinx
67from sphinx.util.logging           import getLogger
68
69from pyTooling.Common              import getTypeName
70from pyTooling.Decorators          import export
71
72from pyTooling.Sphinx              import ReportExtensionError
73from pyTooling.Sphinx.CodeCoverage import CodeCoverageBase, compactDirectory, compactedName, isCompacted
74from pyTooling.Sphinx.Node         import CoverageListing
75from pyTooling.Sphinx.Pages        import ReportPages
76
77if TYPE_CHECKING:  # pragma: no cover
78	# pyEDAA.Reports is an optional dependency (extra 'reports'), imported where a report is read.
79	from pyEDAA.Reports.CodeCoverage import BaseWithPath, CoverageSummary, Directory, File
80
81
82__all__ = ["LINE_STATES"]
83
84LINE_STATES = {
85	"Covered":          "covered",
86	"PartiallyCovered": "partial",
87	"Uncovered":        "uncovered",
88	"Excluded":         "excluded",
89	"Unknown":          "",
90}
91"""
92The state a listing marks a line with, by the name of the line's
93:class:`~pyEDAA.Reports.CodeCoverage.LineCoverageStatus`.
94"""
95
96
97@export
98def tokenClass(tokenType: _TokenType) -> str:
99	"""
100	Return the short CSS class Pygments writes for a token type, as its HTML formatter does.
101
102	A keyword gets ``k``. A token type without a class of its own gets its nearest parent's class, extended by the names
103	below it.
104
105	:param tokenType: The token type.
106	:returns:         The CSS class; empty for plain text.
107	"""
108	suffix = ""
109	while (name := STANDARD_TYPES.get(tokenType)) is None:
110		suffix = f"-{tokenType[-1]}{suffix}"
111		tokenType = tokenType.parent
112
113	return f"{name}{suffix}"
114
115
116@export
117def tokenizeSource(source: str, fileName: str) -> list[list[tuple[str, str]]]:
118	"""
119	Lex a source file with the Pygments lexer of its file name, and split the tokens into lines.
120
121	The whole file is lexed, as a token may span lines - e.g. a doc-string -; such a token is split at the line ends, each
122	part keeping its class. A file name without a lexer is lexed as plain text.
123
124	:param source:   The source file's content.
125	:param fileName: The source file's name, choosing the lexer.
126	:returns:        Per line, its tokens as ``(CSS class, text)`` pairs.
127	"""
128	try:
129		lexer = get_lexer_for_filename(fileName, stripnl=False, ensurenl=True)
130	except ClassNotFound:
131		lexer = TextLexer(stripnl=False, ensurenl=True)
132
133	lines: list[list[tuple[str, str]]] = [[]]
134	for tokenType, value in lexer.get_tokens(source):
135		cssClass = tokenClass(tokenType)
136		for index, part in enumerate(value.split("\n")):
137			if index > 0:
138				lines.append([])
139
140			if part != "":
141				lines[-1].append((cssClass, part))
142
143	return lines[:-1] if len(lines[-1]) == 0 else lines
144
145
146@export
147def createListing(file: File, sources: Path, anchors: bool) -> Union[CoverageListing, nodes.paragraph]:
148	"""
149	Create a source file's listing: its lines, syntax highlighted, each marked by its coverage state.
150
151	A report naming a line beyond the source file's end was measured on another version of the file; this is logged as
152	a warning.
153
154	:param file:    The file of the report.
155	:param sources: The directory the report's file paths are relative to.
156	:param anchors: Whether a line gets the ID ``L<number>``, so the role ``:cov:`` can link to it.
157	:returns:       The listing; or a paragraph saying so, if the source file can't be read.
158	"""
159	sourceFile = sources / file.Path
160	try:
161		source = sourceFile.read_text(encoding="utf-8", errors="replace")
162	except OSError as ex:
163		getLogger(__name__).warning(f"Source file '{sourceFile}' of the code coverage report can't be read: {ex}")
164		return nodes.paragraph(text=f"The source file '{file.Path.as_posix()}' can't be read.")
165
166	tokens = tokenizeSource(source, file._name)
167	if (lastLineNumber := file.LastLineNumber) > len(tokens):
168		getLogger(__name__).warning(
169			f"The code coverage report names line {lastLineNumber} of '{file.Path.as_posix()}', which has {len(tokens)} "
170			f"lines: the report wasn't measured on this version of the file."
171		)
172
173	fileLines = file.Lines
174	lines = []
175	for number, lineTokens in enumerate(tokens, start=1):
176		if number > lastLineNumber or (line := fileLines[number]) is None:
177			lines.append((number, "", None, 0, 0, lineTokens))
178		else:
179			state = LINE_STATES[line._status.name]
180			lines.append((number, state, line._coverageCount, len(line._branches), line.CoveredBranches, lineTokens))
181
182	return CoverageListing(source, source, lines=lines, anchors=anchors, path=file.Path.as_posix(), language="text")
183
184
185@export
186class CodeCoverageReportPages(ReportPages["BaseWithPath"]):
187	"""
188	The pages of one code coverage report: a page per directory and per source file.
189	"""
190
191	_pages:     ClassVar[dict[str, CodeCoverageReportPages]] = {}  #: The pages of every code coverage report, by ID.
192	_separator: ClassVar[str] = "/"                                #: Separator of the names in a path.
193
194	_sources: Path                                   #: Directory the report's file paths are relative to.
195	_levels:  dict[Union[int, str], dict[str, str]]  #: The report's coverage levels, coloring the tables.
196
197	def __init__(
198		self,
199		reportID: str,
200		prefix: str,
201		reportFile: Path,
202		coverage: CoverageSummary,
203		sources: Path,
204		levels: dict[Union[int, str], dict[str, str]]
205	) -> None:
206		"""
207		Compute the document names and entries of a report's directories and source files.
208
209		:param reportID:   Identifier of the report.
210		:param prefix:     Document name the pages are generated below.
211		:param reportFile: The report file.
212		:param coverage:   The report's root directory.
213		:param sources:    Directory the report's file paths are relative to.
214		:param levels:     The report's coverage levels.
215		"""
216		super().__init__(reportID, prefix, reportFile, self._DirectoryChildren(coverage))
217
218		self._sources = sources
219		self._levels =  levels
220
221	@classmethod
222	def CreatePages(cls, sphinxApplication: Sphinx) -> None:
223		"""
224		Call-back for Sphinx' ``builder-inited`` event, after the reports are read: compute and register the pages.
225
226		Only a report with the key ``pages`` gets pages. A report that couldn't be read gets none; this is logged as an
227		error.
228
229		:param sphinxApplication: Sphinx application instance.
230		"""
231		cls._pages = {}
232		for reportID, packageConfiguration in CodeCoverageBase._packageConfigurations.items():
233			if (prefix := packageConfiguration["pages"]) is None:
234				continue
235
236			try:
237				coverage = CodeCoverageBase.GetReport(reportID)
238			except ReportExtensionError as ex:
239				cause = ex if ex.__cause__ is None else ex.__cause__
240				getLogger(__name__).error(
241					f"Caught {getTypeName(ex)} when generating the pages of code coverage report '{reportID}'.\n"
242					f"  {getTypeName(cause)}: {cause}"
243				)
244				continue
245
246			pages = cls(
247				reportID, prefix, packageConfiguration["report"], coverage, packageConfiguration["sources"],
248				packageConfiguration["levels"]
249			)
250			cls._pages[reportID] = pages
251			pages.Register(sphinxApplication.env)
252
253	@staticmethod
254	def _DirectoryChildren(directory: Directory) -> list[tuple[str, BaseWithPath]]:
255		"""
256		Return a directory's directories - each compacted with its single subdirectories -, then its files, each sorted by
257		name.
258
259		:param directory: The directory.
260		:returns:         The children as ``(role name, object)`` pairs.
261		"""
262		children: list[tuple[str, BaseWithPath]] = []
263		children.extend(("cov", compactDirectory(directory._directories[name])) for name in sorted(directory._directories))
264		children.extend(("cov", directory._files[name]) for name in sorted(directory._files))
265
266		return children
267
268	def _Name(self, entity: BaseWithPath) -> str:
269		"""
270		Return the name a directory or file is shown with; a compacted directory's includes its parents' names.
271
272		:param entity: The directory or file.
273		:returns:      The name, e.g. ``Counter.vhdl`` or ``pyTooling/Sphinx``.
274		"""
275		return compactedName(entity)
276
277	def _PathElements(self, name: str) -> tuple[str, ...]:
278		"""
279		Return the elements a name adds to the path: a compacted directory's name adds one per directory.
280
281		:param name: The directory's or file's name.
282		:returns:    The path elements.
283		"""
284		return tuple(name.split("/"))
285
286	def _Title(self, entity: BaseWithPath) -> Nullable[str]:
287		"""
288		Return no title, as a report has none for its directories and files.
289
290		:param entity: The directory or file.
291		:returns:      ``None``
292		"""
293		return None
294
295	def _Children(self, roleName: str, entity: BaseWithPath) -> Iterable[tuple[str, BaseWithPath]]:
296		"""
297		Return a directory's directories, then its files; a file has none.
298
299		:param roleName: The role referring to the object: ``cov``.
300		:param entity:   The directory or file.
301		:returns:        The children as ``(role name, object)`` pairs.
302		"""
303		if hasattr(entity, "_directories"):
304			return self._DirectoryChildren(entity)
305
306		return []
307
308	def _Page(self, docName: str, roleName: str, entity: BaseWithPath) -> nodes.section:
309		"""
310		Build a directory's or file's page.
311
312		:param docName:  Name of the page's document.
313		:param roleName: The role referring to the object: ``cov``.
314		:param entity:   The directory or file.
315		:returns:        The page's top section.
316		"""
317		if hasattr(entity, "_directories"):
318			return self._DirectoryPage(docName, entity)
319		else:
320			return self._FilePage(docName, entity)
321
322	def _SummarySection(self, docName: str, entity: BaseWithPath) -> tuple[nodes.section, nodes.field_list]:
323		"""
324		Create a page's section *Summary*: the line and branch counters, the coverage, and the parent directory.
325
326		:param docName: Name of the page's document.
327		:param entity:  The directory or file.
328		:returns:       The section, and its field list, which a page may add fields to.
329		"""
330		section = nodes.section("", nodes.title("Summary", "Summary"), ids=["summary"])
331		section += (fieldList := nodes.field_list(classes=["report-codecov-fields"]))
332
333		self._Field(fieldList, "Lines", nodes.Text(
334			f"{entity.CoveredLines} of {entity.TotalLines} covered ({entity.LineCoverage:.1%})"
335		))
336		if entity.ExcludedLines > 0:
337			self._Field(fieldList, "Excluded lines", nodes.Text(f"{entity.ExcludedLines}"))
338
339		if entity.TotalBranches > 0:
340			self._Field(fieldList, "Branches", nodes.Text(
341				f"{entity.CoveredBranches} of {entity.TotalBranches} taken ({entity.BranchCoverage:.1%}), "
342				f"{entity.PartialLines} lines partially"
343			))
344		self._Field(fieldList, "Coverage", nodes.Text(f"{entity.Coverage:.1%}"))
345
346		parent = entity._parent
347		while isCompacted(parent):
348			parent = parent._parent
349
350		if parent is not None and parent._parent is not None:
351			self._Field(fieldList, "Directory", self._Reference("cov", parent, docName))
352
353		return section, fieldList
354
355	def _DirectoryPage(self, docName: str, directory: Directory) -> nodes.section:
356		"""
357		Build a directory's page: its counters, then tables of its directories and files, each linked to its page.
358
359		:param docName:   Name of the page's document.
360		:param directory: The directory.
361		:returns:         The page's top section.
362		"""
363		name = compactedName(directory)
364		page = nodes.section("", nodes.title(name, name), ids=["report-codecov-page"])
365		summarySection, _ = self._SummarySection(docName, directory)
366		page += summarySection
367
368		children: list[str] = []
369		subdirectories = [child for _, child in self._DirectoryChildren(directory) if hasattr(child, "_directories")]
370		files = [directory._files[fileName] for fileName in sorted(directory._files)]
371		for title, column, identifier, entities in (
372			("Directories", "Directory", "directories", subdirectories),
373			("Files",       "File",      "files",       files),
374		):
375			if len(entities) == 0:
376				continue
377
378			page += (section := nodes.section("", nodes.title(title, title), ids=[identifier]))
379			table, tableBody = self._Table(
380				[(column, 6), ("Lines", 2), ("Branches", 2), ("Coverage", 1)],
381				["report-codecov-table", "report-codecov-children"]
382			)
383			section += table
384			for entity in entities:
385				level = CodeCoverageBase.LevelOf(self._levels, entity.Coverage, "class")
386				tableBody += (row := nodes.row("", classes=[level]))
387				row += nodes.entry("", nodes.paragraph("", "", self._Reference("cov", entity, docName)))
388				row += nodes.entry("", nodes.paragraph("", f"{entity.CoveredLines} of {entity.TotalLines}"))
389				row += nodes.entry("", nodes.paragraph("", f"{entity.CoveredBranches} of {entity.TotalBranches}"))
390				row += nodes.entry("", nodes.paragraph("", f"{entity.Coverage:.1%}"))
391				if (entry := self.Entry(entity)) is not None:
392					children.append(entry.docName)
393
394		if len(children) > 0:
395			# body elements precede a section's subsections, so the hidden table of contents goes before 'Summary'
396			page.insert(page.index(summarySection), self._TableOfContents(docName, children))
397
398		return page
399
400	def _FilePage(self, docName: str, file: File) -> nodes.section:
401		"""
402		Build a source file's page: its counters, its missed lines, and its listing.
403
404		:param docName: Name of the page's document.
405		:param file:    The source file.
406		:returns:       The page's top section.
407		"""
408		page = nodes.section("", nodes.title(file._name, file._name), ids=["report-codecov-page"])
409		summarySection, fieldList = self._SummarySection(docName, file)
410		page += summarySection
411
412		if (missed := self._MissedLines(file)) != "":
413			self._Field(fieldList, "Missed lines", nodes.Text(missed))
414
415		page += (section := nodes.section("", nodes.title("Source", "Source"), ids=["source"]))
416		section += createListing(file, self._sources, anchors=True)
417
418		return page
419
420	@staticmethod
421	def _MissedLines(file: File) -> str:
422		"""
423		Return a file's lines, which never ran, as ranges of consecutive executable lines, as coverage.py writes them.
424
425		:param file: The source file.
426		:returns:    The ranges, e.g. ``8, 14-17``; empty if every executable line ran.
427		"""
428		ranges: list[str] = []
429		start = end = None
430		for line in file.IterateLines():
431			number = line._lineNumber
432			state =  line._status.name
433			if state == "Excluded":
434				continue
435
436			if state == "Uncovered":
437				start = number if start is None else start
438				end = number
439			elif start is not None:
440				ranges.append(f"{start}" if start == end else f"{start}-{end}")
441				start = None
442
443		if start is not None:
444			ranges.append(f"{start}" if start == end else f"{start}-{end}")
445
446		return ", ".join(ranges)