CompletePipeline

The CompletePipeline job template is the combination of almost all job templates offered by pyTooling/Actions in a single workflow template. If fulfills all needs to test, package, document, publish and release Python code from GitHub. It can be used for simple Python packages as well as namespace packages.

Features

Testing

Run unit tests before packaging using pytest.
Run platform tests before packaging using pytest.
Run application tests using packaged code on target platform.

Code Quality

Collect code coverage using Coverage.py.
Check documentation coverage using docstr_coverage and interrogate.
Check static typing closure using mypy.
Static Application Security Testing (SAST) using bandit

Report Handling

Merge unit test results into one report using pyEDAA.Reports.
Merge code coverage results into one report using Coverage.py.

Documentation

Compile documentation using Sphinx as HTML and LaTeX.
Translate LaTeX documentation to PDF using MikTeX.

Publishing Results

GitHub Pipeline Summary
- Publish unittest results using Test Reporter.
GitHub Pages
- Publish HTML documentation to GitHub Pages.
Codecov
- Publish code coverage to CodeCov.
- Publish unittest results to CodeCov.
Codacy
- Publish code coverage to Codacy.

Packaging

Package as wheel using build.
Install wheel on target platform using pip.
Upload to PyPI using twine.

Releasing

Automatic tagging of merge commits on main branch to trigger a tagged pipeline (release pipeline).
Create a release page with text derived from pull request description.

Behavior

Classify ${{ github.ref }} into branch, tag or pull-request and find the pull-request associated with a release commit.
Extract Python project settings from pyproject.toml.
Compute the job matrices and the artifact names based on system, Python version, environment, … for job variants (unittest_python_version_list, unittest_system_list, unittest_include_list, unittest_exclude_list, unittest_disable_list).
Verify that the version in the Python code matches the version derived from the tag or pull-request title (version_file), and on a release commit, that PyPI has no release of this version yet (check_pypi_duplicate).
Both checks run early. If one fails, the pipeline still runs, but steps 17 to 19 are refused.
Run unit tests using pytest and collect code coverage.
Verify type annotations using static typing analysis using mypy.
Check documentation coverage using docstr_coverage and interrogate.
Check code quality: security scanning using bandit (bandit), metrics and complexity using radon, linting using pylint (pylint).
Package code as source distribution and wheel.
Install the wheel on every target platform and verify the installed version.
Run application tests against the installed package using pytest (apptest, apptest_python_version_list, apptest_system_list).
Merge unit test results and code coverage results and publish them to GitHub (dorny), Codecov (codecov) and Codacy (codacy).
Delete the per-matrix-job artifacts that have been merged (cleanup).
Generate HTML and LaTeX documentations using Sphinx (documentation_steps: html, latex).
Translate LaTeX documentation to PDF using MikTeX (documentation_steps: pdf, miktex_image, miktex_update).
Publish documentation to GitHub Pages (documentation_steps: pages).
Tag a release commit, which triggers a second pipeline run for the new tag (auto_tag). The tag is refused, if the tag or a release page already exists.
Create a GitHub release page with text derived from the pull-request description and upload release assets.
Publish wheel to PyPI.
Delete the remaining artifacts (cleanup).

Steps 17 to 19 form the release path: step 17 runs on the release branch and creates the tag, steps 18 and 19 run in the tag pipeline that this tag triggers.

Steps 8, 11, 13, 14, 15, 16, 17 and 20 can be switched off by the parameters listed with them. A step that is switched off is skipped; every other step is executed as usual. Steps 1 to 7, 9, 10, 12, 18 and 19 are always executed - step 12 only decides per service whether the results are published.

Instantiation

The following instantiation example creates a SimplePackage job derived from job template CompletePipeline version @r8. It only requires the package_name parameter to run a full pipeline suitable for a Python project.

Simple Package

name: Pipeline

jobs:
  SimplePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:

      package_name: myPackage

Namespace Package

name: Pipeline

jobs:
  NamespacePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:
      package_namespace: myFramework
      package_name:      Extension

Simple Package

📂ProjectRoot/
  📂myFramework/

    📦SubPackage/
      🐍__init__.py
      🐍SubModuleA.py
    🐍__init__.py
    🐍ModuleB.py

Namespace Package

📂ProjectRoot/
  📂myFramework/
    📂Extension/
      📦SubPackage/
        🐍__init__.py
        🐍SubModuleA.py
      🐍__init__.py
      🐍ModuleB.py

Parameter Summary

Goto input parameters

Parameter Name	Required	Type	Default
package_namespace	no	string	`''`
package_name	yes	string	— — — —
version_file	no	string	`'__init__.py'`
unittest_python_version	no	string	`'3.14'`
unittest_python_version_list	no	string	`'3.10 3.11 3.12 3.13 3.14'`
unittest_system_list	no	string	`'ubuntu ubuntu-arm windows windows-arm macos macos-arm ucrt64'`
unittest_include_list	no	string	`''`
unittest_exclude_list	no	string	`'windows-arm:3.9 windows-arm:3.10'`
unittest_disable_list	no	string	`'windows-arm:pypy-3.10 windows-arm:pypy-3.11'`
unittest_parallel	no	string	`'auto'`
apptest_python_version	no	string	`'3.14'`
bandit	no	string	`'false'`
pylint	no	string	`'false'`
documentation_steps	no	string	`'html pages'`
miktex_image	no	string	`'pytooling/miktex:sphinx'`
miktex_update	no	string	`'false'`
publish_pages_on	no	string	four conditions - see description
auto_tag	no	string	`'true'`
check_pypi_duplicate	no	string	`'true'`
pypi_url	no	string	`'https://pypi.org'`
apptest_python_version_list	no	string	`''`
apptest_system_list	no	string	`'ubuntu ubuntu-arm windows windows-arm macos macos-arm ucrt64'`
apptest_include_list	no	string	`''`
apptest_exclude_list	no	string	`'windows-arm:3.9 windows-arm:3.10'`
apptest_disable_list	no	string	`'windows-arm:pypy-3.10 windows-arm:pypy-3.11'`
apptest_parallel	no	string	`'auto'`
apptest	no	string	`'false'`
codecov	no	string	`'false'`
codacy	no	string	`'false'`
dorny	no	string	`'false'`
pypi_dry_run	no	string	`'false'`
pypi_upload_url	no	string	`'https://upload.pypi.org/legacy/'`
cleanup	no	string	`'true'`

Goto secrets

Token Name	Required	Type	Default
PYPI_TOKEN	no	string	— — — —
CODECOV_TOKEN	no	string	— — — —
CODACY_TOKEN	no	string	— — — —

Goto output parameters

This job template has no output parameters.

Input Parameters

package_namespace

Type:

string

Required:

no

Default Value:

''

Possible Values:

Any valid Python namespace.

Description:

In case the package is a Python namespace package, the name of the library’s or package’s namespace needs to be specified using this parameter.
In case of a simple Python package, this parameter must be specified as an empty string (''), which is the default.

Example:

Example Instantiation

name: Pipeline

jobs:
  NamespacePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:
      package_namespace: myFramework
      package_name:      Extension

Example Directory Structure

📂ProjectRoot/
  📂myFramework/
    📂Extension/
      📦SubPackage/
        🐍__init__.py
        🐍SubModuleA.py
      🐍__init__.py
      🐍ModuleB.py

package_name

Type:

string

Required:

yes

Default Value:

— — — —

Possible Values:

Any valid Python package name.

Description:

In case of a simple Python package, this package’s name is specified using this parameter.
In case the package is a Python namespace package, the parameter package_namespace must be specified, too.

Example:

Example Instantiation

name: Pipeline

jobs:
  SimplePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:
      package_name: myPackage

Example Directory Structure

📂ProjectRoot/
  📂myFramework/
    📦SubPackage/
      🐍__init__.py
      🐍SubModuleA.py
    🐍__init__.py
    🐍ModuleB.py

version_file

Type:: string
Required:: no
Default Value:: '__init__.py'
Possible Values:: Any path relative to the package directory.
Description:: Module inside the package that carries the __version__ variable. Forwarded to version_file.
A namespace package has no __init__.py in its root, so its version is kept in a sub-package, e.g. 'Common/__init__.py' for pyTooling.*.
On a release commit and in the tag pipeline, this version must match the version derived from the pull-request title or tag, otherwise the release is refused.

unittest_python_version

Type:

string

Required:

no

Default Value:

'3.14'

Possible Values:

Any valid Python version conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.
See actions/python-versions - available Python versions and actions/setup-python - configurable Python versions.

Description:

The default Python version used for intermediate jobs using Python tools.

In case unittest_python_version_list is empty, this default version is used to populate the unittest_python_version_list parameter.

unittest_python_version_list

Type:

string

Required:

no

Default Value:

'3.10 3.11 3.12 3.13 3.14'

Possible Values:

A space separated list of valid Python versions conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.

Description:

The list of space-separated Python versions used for unit testing.

Possible values

3.8, 3.9, 3.10 , 3.11, 3.12, 3.13, 3.14
pypy-3.7, pypy-3.8, pypy-3.9, pypy-3.10, pypy-3.11

Icon	Version	Maintained until	Comments
⚫	3.8	2024.10	outdated
🔴	3.9	2025.10
🟠	3.10	2026.10
🟡	3.11	2027.10
🟢	3.12	2028.10
🟢	3.13	2029.10	latest CPython
🟣	3.14	2030.10	Python 3.14 alpha, beta (or RC) will be used.
⟲⚫	pypy-3.7	????.??
⟲⚫	pypy-3.8	????.??
⟲🔴	pypy-3.9	????.??
⟲🟠	pypy-3.10	????.??
⟲🟡	pypy-3.11	????.??	latest PyPy

unittest_system_list

Type:

string

Required:

no

Default Value:

'ubuntu ubuntu-arm windows windows-arm macos macos-arm ucrt64'

Possible Values:

A space separated list of system names.

Description:

The list of space-separated systems used for unit testing.

Possible values

Native systems: ubuntu, ubuntu-arm, windows, windows-arm, macos, macos-arm
MSYS2 runtimes: msys, clang32, clang64, ucrt64

The image used per system is configurable via the *_image parameters of Parameters. The versions listed below are the defaults.

Icon	System	Used version	Image parameter	Comments
🪟	windows	Windows Server 2025	`windows_image`
🏢	windows-arm	Windows 11 on ARM64	`windows_arm_image`
🐧	ubuntu	Ubuntu 26.04 (LTS)	`ubuntu_image`
⛄	ubuntu-arm	Ubuntu 26.04 (LTS) on ARM64	`ubuntu_arm_image`
🍎	macos	macOS 15 (Intel)	`macos_intel_image`
🍏	macos-arm	macOS 15 (ARM64)	`macos_arm_image`
🪟🟪	msys	MSYS2 - MSYS	runtime of `windows_image`
🪟🟫	clang32	MSYS2 - Clang32	runtime of `windows_image`	deprecated
🪟🟧	clang64	MSYS2 - Clang64	runtime of `windows_image`
🪟🟨	ucrt64	MSYS2 - UCRT64	runtime of `windows_image`

Source: Images provided by GitHub

unittest_include_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be included into the list of unittest variants.

For more details see include_list.

unittest_exclude_list

Type:

string

Required:

no

Default Value:

'windows-arm:3.9 windows-arm:3.10'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be excluded from the list of unittest variants.

For more details see exclude_list.

unittest_disable_list

Type:

string

Required:

no

Default Value:

'windows-arm:pypy-3.10 windows-arm:pypy-3.11'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be temporarily disabled from the list of unittest variants.
Each disabled item creates a warning in the workflow log.

For more details see disable_list.

unittest_parallel

Type:: string
Required:: no
Default Value:: 'auto'
Possible Values:: 'auto', a number of workers, or 'false'.
Description:: Number of pytest-xdist workers for the unit tests. With 'false', the tests run serially.
Passed on as parallel.

apptest_python_version

Type:

string

Required:

no

Default Value:

'3.14'

Possible Values:

Any valid Python version conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.
See actions/python-versions - available Python versions and actions/setup-python - configurable Python versions.

Description:

The default Python version used for intermediate jobs using Python tools.

In case apptest_python_version_list is empty, this default version is used to populate the apptest_python_version_list parameter.

apptest_python_version_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of valid Python versions conforming to the pattern <major>.<minor> or pypy-<major>.<minor>`.

Description:

The list of space-separated Python versions used for application testing.

As this list is empty by default, the value is derived from apptest_python_version.

Possible values

3.8, 3.9, 3.10 , 3.11, 3.12, 3.13, 3.14
pypy-3.7, pypy-3.8, pypy-3.9, pypy-3.10, pypy-3.11

Icon	Version	Maintained until	Comments
⚫	3.8	2024.10	outdated
🔴	3.9	2025.10
🟠	3.10	2026.10
🟡	3.11	2027.10
🟢	3.12	2028.10
🟢	3.13	2029.10	latest CPython
🟣	3.14	2030.10	Python 3.14 alpha, beta (or RC) will be used.
⟲⚫	pypy-3.7	????.??
⟲⚫	pypy-3.8	????.??
⟲🔴	pypy-3.9	????.??
⟲🟠	pypy-3.10	????.??
⟲🟡	pypy-3.11	????.??	latest PyPy

apptest_system_list

Type:

string

Required:

no

Default Value:

'ubuntu ubuntu-arm windows windows-arm macos macos-arm ucrt64'

Possible Values:

A space separated list of system names.

Description:

The list of space-separated systems used for application testing.

Possible values

Native systems: ubuntu, ubuntu-arm, windows, windows-arm, macos, macos-arm
MSYS2 runtimes: msys, clang32, clang64, ucrt64

The image used per system is configurable via the *_image parameters of Parameters. The versions listed below are the defaults.

Icon	System	Used version	Image parameter	Comments
🪟	windows	Windows Server 2025	`windows_image`
🏢	windows-arm	Windows 11 on ARM64	`windows_arm_image`
🐧	ubuntu	Ubuntu 26.04 (LTS)	`ubuntu_image`
⛄	ubuntu-arm	Ubuntu 26.04 (LTS) on ARM64	`ubuntu_arm_image`
🍎	macos	macOS 15 (Intel)	`macos_intel_image`
🍏	macos-arm	macOS 15 (ARM64)	`macos_arm_image`
🪟🟪	msys	MSYS2 - MSYS	runtime of `windows_image`
🪟🟫	clang32	MSYS2 - Clang32	runtime of `windows_image`	deprecated
🪟🟧	clang64	MSYS2 - Clang64	runtime of `windows_image`
🪟🟨	ucrt64	MSYS2 - UCRT64	runtime of `windows_image`

Source: Images provided by GitHub

apptest_include_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be included into the list of application test variants.

For more details see include_list.

apptest_exclude_list

Type:

string

Required:

no

Default Value:

'windows-arm:3.9 windows-arm:3.10'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be excluded from the list of application test variants.

For more details see exclude_list.

apptest_disable_list

Type:

string

Required:

no

Default Value:

'windows-arm:pypy-3.10 windows-arm:pypy-3.11'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be temporarily disabled from the list of application test variants.
Each disabled item creates a warning in the workflow log.

For more details see disable_list.

apptest_parallel

Type:: string
Required:: no
Default Value:: 'auto'
Possible Values:: 'auto', a number of workers, or 'false'.
Description:: Number of pytest-xdist workers for the application tests. With 'false', the tests run serially.
Passed on as parallel.

apptest

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Run application tests via ApplicationTesting.
Application testing installs the built wheel and exercises the package as an installed package, so it needs a packaging step to have run first.
'true' - run the application tests.
'false' - skip the application testing jobs.

bandit

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Run Static Application Security Testing (SAST) using bandit.
'true' - run the Bandit job.
'false' - skip it.

pylint

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Run Python linting using pylint.
'true' - run the PyLint job.
'false' - skip it.

codecov

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Publish merged code coverage results and a merged unit test summary to CodeCov.
'true' - publish; secret CODECOV_TOKEN must be set.
'false' - do not publish.

codacy

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Publish merged code coverage results to Codacy.
'true' - publish; secret CODACY_TOKEN must be set.
'false' - do not publish.

dorny

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Publish a merged unit test summary as pipeline result using Test Reporter.
'true' - create the report page.
'false' - do not create it.

documentation_steps

Type:

string

Required:

no

Default Value:

'html pages'

Possible Values:

A space separated list of none, html, latex, pdf, pages, asset or all.

Description:

Documentation steps to run.

html:: Build the HTML documentation using Sphinx.
latex:: Build the LaTeX documentation using Sphinx.
pdf:: Translate the LaTeX documentation to PDF using MikTeX. Requires latex.
pages:: Publish the HTML documentation to GitHub Pages. Requires html.
asset:: Attach the documentation to the release page.
all:: All of the above.
none:: No documentation at all.

A step that is not listed is skipped and its artifact is not produced.

Note

pages additionally requires the pipeline’s ref to match publish_pages_on.

miktex_image

Type:: string
Required:: no
Default Value:: 'pytooling/miktex:sphinx'
Possible Values:: Any Docker image providing a MikTeX installation with latexmk, e.g. an image of pyTooling/MiKTeX.
Description:: Docker image used to translate LaTeX to PDF.
Forwarded to miktex_image.

miktex_update

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Update the MikTeX packages before building the PDF.
Forwarded to update.
'true' - update the packages inside the container first.
'false' - use the packages shipped with the image.

publish_pages_on

Type:: string
Required:: no
Default Value:: default-branch, development-branch, release-tag and nightly-tag, one per line.
Possible Values:: A newline separated list of conditions, see publish_pages_on.
Description:: Conditions on the pipeline’s ref under which the documentation is published to GitHub Pages, if documentation_steps contains pages. Forwarded to publish_pages_on.
A github-pages environment restricted to selected branches or tags rejects a deployment from any other ref before a runner is assigned, so the job fails after one second without a log. The conditions should not admit more refs than the environment does.

auto_tag

Type:: string
Required:: no
Default Value:: 'true'
Possible Values:: 'true' / 'false'
Description:: Create a release tag when a pull-request was merged into the release branch and its title matches the release tag pattern.
The new tag triggers a second, tagged pipeline run which publishes the release. Forwarded to auto_tag.
'true' - tag the release commit.
'false' - never tag automatically.

check_pypi_duplicate

Type:: string
Required:: no
Default Value:: 'true'
Possible Values:: 'true' / 'false'
Description:: Refuse to tag a release commit, if pypi_url - by default PyPI - already has a release of this version. Forwarded to check_pypi_duplicate.
PyPI never accepts a version twice, but it refuses the upload only at the end of the tag pipeline - after the tag was created and the release page was published.
'true' - query PyPI before tagging.
'false' - don’t query PyPI, e.g. for a package that isn’t published there.

pypi_url

Type:: string
Required:: no
Default Value:: 'https://pypi.org'
Possible Values:: Base URL of a package registry offering the PyPI JSON API, e.g. 'https://test.pypi.org'.
Description:: The package registry checked for an existing release, if check_pypi_duplicate is 'true'. Forwarded to pypi_url.
Set pypi_upload_url to the same registry.

pypi_dry_run

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true' / 'false'
Description:: Validate the built packages with twine check instead of uploading them to PyPI. Forwarded to dry_run.
Intended for a pipeline that builds a package which is never published - a fixture used to exercise the job templates, or a fork that must not push to the upstream project’s PyPI name.
'true' - check the packages and publish nothing.
'false' - publish the packages.

pypi_upload_url

Type:: string
Required:: no
Default Value:: 'https://upload.pypi.org/legacy/'
Possible Values:: Upload URL of a package registry, e.g. 'https://test.pypi.org/legacy/'.
Description:: The package registry the packages are uploaded to. Forwarded to pypi_upload_url.
Set pypi_url to the same registry.

cleanup

Type:: string
Required:: no
Default Value:: 'true'
Possible Values:: 'true' / 'false'
Description:: Remove intermediate artifacts once they were merged or consumed.
'true' - delete them.
'false' - keep them, which helps debugging artifact handovers between jobs.

Secrets

The workflow template uses the following secrets to publish results to other services.

PYPI_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish and upload packages on PyPI.

CODECOV_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish code coverage and unit test results to CodeCov.

CODACY_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish code coverage results to Codacy.

Outputs

This job template has no output parameters.

Optimizations

The following optimizations can be used to reduce the template’s runtime.

Todo

CompletePipeline::Optimizations Needs a list of optimizations.