CompletePipeline

The CompletePipeline job template is the combination of almost all job templates offered by pyTooling/Actions in a single workflow template. If fulfills all needs to test, package, document, publish and release Python code from GitHub. It can be used for simple Python packages as well as namespace packages.

Features

Testing

Run unit tests before packaging using pytest.
Run platform tests before packaging using pytest.
Run application tests using packaged code on target platform.

Code Quality

Collect code coverage using Coverage.py.
Check documentation coverage using docstr_coverage and interrogate.
Check static typing closure using mypy.
Static Application Security Testing (SAST) using bandit

Report Handling

Merge unit test results into one report using pyEDAA.Reports.
Merge code coverage results into one report using Coverage.py.

Documentation

Compile documentation using Sphinx as HTML and LaTeX.
Translate LaTeX documentation to PDF using MikTeX.

Publishing Results

GitHub Pipeline Summary
- Publish unittest results using Test Reporter.
GitHub Pages
- Publish HTML documentation to GitHub Pages.
Codecov
- Publish code coverage to CodeCov.
- Publish unittest results to CodeCov.
Codacy
- Publish code coverage to Codacy.

Packaging

Package as wheel using build.
Install wheel on target platform using pip.
Upload to PyPI using twine.

Releasing

Automatic tagging of merge commits on main branch to trigger a tagged pipeline (release pipeline).
Create a release page with text derived from pull request description.

Behavior

Infer information from ${{ github.ref }} variable.
Extract Python project settings from pyproject.toml.
Compute job matrix based on system, Python version, environment, … for job variants.
Run unit tests using pytest and collect code coverage.
Run platform tests using pytest and collect code coverage.
Run application tests using pytest.
Package code as wheel.
Check documentation coverage using docstr_coverage and interrogate.
Verify type annotation using static typing analysis using mypy.
Merge unit test results and code coverage results.
Generate HTML and LaTeX documentations using Sphinx.
Translate LaTeX documentation to PDF using MikTeX.
Publish unit test and code coverage results to cloud services.
Publish documentation to GitHub Pages.
Publish wheel to PyPI.
Create a GitHub release page and upload release assets.

Instantiation

The following instantiation example creates a SimplePackage job derived from job template CompletePipeline version @r6. It only requires the package_name parameter to run a full pipeline suitable for a Python project.

Simple Package

name: Pipeline

jobs:
  SimplePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r6
    with:

      package_name: myPackage

Namespace Package

name: Pipeline

jobs:
  NamespacePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r6
    with:
      package_namespace: myFramework
      package_name:      Extension

Simple Package

📂ProjectRoot/
  📂myFramework/

    📦SubPackage/
      🐍__init__.py
      🐍SubModuleA.py
    🐍__init__.py
    🐍ModuleB.py

Namespace Package

📂ProjectRoot/
  📂myFramework/
    📂Extension/
      📦SubPackage/
        🐍__init__.py
        🐍SubModuleA.py
      🐍__init__.py
      🐍ModuleB.py

Parameter Summary

Goto input parameters

Parameter Name	Required	Type	Default
package_namespace	no	string	`''`
package_name	yes	string	— — — —
unittest_python_version	no	string	`'3.14'`
unittest_python_version_list	no	string	`'3.10 3.11 3.12 3.13 3.14'`
unittest_system_list	no	string	`'ubuntu windows macos macos-arm ucrt64'`
unittest_include_list	no	string	`''`
unittest_exclude_list	no	string	`'windows-arm:3.9 windows-arm:3.10'`
unittest_disable_list	no	string	`'windows-arm:pypy-3.10 windows-arm:pypy-3.11'`
apptest_python_version	no	string	`'3.14'`
apptest_python_version_list	no	string	`''`
apptest_system_list	no	string	`'ubuntu windows macos macos-arm ucrt64'`
apptest_include_list	no	string	`''`
apptest_exclude_list	no	string	`'windows-arm:3.9 windows-arm:3.10'`
apptest_disable_list	no	string	`'windows-arm:pypy-3.10 windows-arm:pypy-3.11'`
codecov	no	string	`'false'`
codacy	no	string	`'false'`
dorny	no	string	`'false'`
cleanup	no	string	`'true'`

Goto secrets

Token Name	Required	Type	Default
PYPI_TOKEN	no	string	— — — —
CODECOV_TOKEN	no	string	— — — —
CODACY_TOKEN	no	string	— — — —

Goto output parameters

This job template has no output parameters.

Input Parameters

package_namespace

Type:

string

Required:

no

Default Value:

''

Possible Values:

Any valid Python namespace.

Description:

In case the package is a Python namespace package, the name of the library’s or package’s namespace needs to be specified using this parameter.
In case of a simple Python package, this parameter must be specified as an empty string (''), which is the default.

Example:

Example Instantiation

name: Pipeline

jobs:
  NamespacePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r6
    with:
      package_namespace: myFramework
      package_name:      Extension

Example Directory Structure

📂ProjectRoot/
  📂myFramework/
    📂Extension/
      📦SubPackage/
        🐍__init__.py
        🐍SubModuleA.py
      🐍__init__.py
      🐍ModuleB.py

package_name

Type:

string

Required:

yes

Default Value:

— — — —

Possible Values:

Any valid Python package name.

Description:

In case of a simple Python package, this package’s name is specified using this parameter.
In case the package is a Python namespace package, the parameter package_namespace must be specified, too.

Example:

Example Instantiation

name: Pipeline

jobs:
  SimplePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r6
    with:
      package_name: myPackage

Example Directory Structure

📂ProjectRoot/
  📂myFramework/
    📦SubPackage/
      🐍__init__.py
      🐍SubModuleA.py
    🐍__init__.py
    🐍ModuleB.py

unittest_python_version

Type:

string

Required:

no

Default Value:

'3.14'

Possible Values:

Any valid Python version conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.
See actions/python-versions - available Python versions and actions/setup-python - configurable Python versions.

Description:

The default Python version used for intermediate jobs using Python tools.

In case unittest_python_version_list is empty, this default version is used to populate the unittest_python_version_list parameter.

unittest_python_version_list

Type:

string

Required:

no

Default Value:

'3.10 3.11 3.12 3.13 3.14'

Possible Values:

A space separated list of valid Python versions conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.

Description:

The list of space-separated Python versions used for unit testing.

Possible values

3.8, 3.9, 3.10 , 3.11, 3.12, 3.13, 3.14
pypy-3.7, pypy-3.8, pypy-3.9, pypy-3.10, pypy-3.11

Icon	Version	Maintained until	Comments
⚫	3.8	2024.10	outdated
🔴	3.9	2025.10
🟠	3.10	2026.10
🟡	3.11	2027.10
🟢	3.12	2028.10
🟢	3.13	2029.10	latest CPython
🟣	3.14	2030.10	Python 3.14 alpha, beta (or RC) will be used.
⟲⚫	pypy-3.7	????.??
⟲⚫	pypy-3.8	????.??
⟲🔴	pypy-3.9	????.??
⟲🟠	pypy-3.10	????.??
⟲🟡	pypy-3.11	????.??	latest PyPy

unittest_system_list

Type:

string

Required:

no

Default Value:

'ubuntu windows macos macos-arm mingw64 ucrt64'

Possible Values:

A space separated list of system names.

Description:

The list of space-separated systems used for unit testing.

Possible values

Native systems: ubuntu, windows, macos
MSYS2: msys, mingw32, mingw64, clang32, clang64, ucrt64

Icon	System	Used version	Comments
🪟	Windows	Windows Server 2025 (latest)
🐧	Ubuntu	Ubuntu 24.04 (LTS) (latest)
🍎	macOS	macOS Ventura 13 (latest)	While this marked latest, macOS Ventura 13 is already provided.
🍏	macOS-arm	macOS Sonoma 14 (latest)	While this marked latest, macOS Ventura 13 is already provided.
🟪	MSYS
⬛	MinGW32
🟦	MinGW64
🟫	Clang32
🟧	Clang64
🟨	UCRT64

Source: Images provided by GitHub

unittest_include_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be included into the list of unittest variants.

For more details see include_list.

unittest_exclude_list

Type:

string

Required:

no

Default Value:

'windows-arm:3.9 windows-arm:3.10'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be excluded from the list of unittest variants.

For more details see exclude_list.

unittest_disable_list

Type:

string

Required:

no

Default Value:

'windows-arm:pypy-3.10 windows-arm:pypy-3.11'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be temporarily disabled from the list of unittest variants.
Each disabled item creates a warning in the workflow log.

For more details see disable_list.

apptest_python_version

Type:

string

Required:

no

Default Value:

'3.14'

Possible Values:

Any valid Python version conforming to the pattern <major>.<minor> or pypy-<major>.<minor>.
See actions/python-versions - available Python versions and actions/setup-python - configurable Python versions.

Description:

The default Python version used for intermediate jobs using Python tools.

In case apptest_python_version_list is empty, this default version is used to populate the apptest_python_version_list parameter.

apptest_python_version_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of valid Python versions conforming to the pattern <major>.<minor> or pypy-<major>.<minor>`.

Description:

The list of space-separated Python versions used for application testing.

As this list is empty by default, the value is derived from apptest_python_version.

Possible values

3.8, 3.9, 3.10 , 3.11, 3.12, 3.13, 3.14
pypy-3.7, pypy-3.8, pypy-3.9, pypy-3.10, pypy-3.11

Icon	Version	Maintained until	Comments
⚫	3.8	2024.10	outdated
🔴	3.9	2025.10
🟠	3.10	2026.10
🟡	3.11	2027.10
🟢	3.12	2028.10
🟢	3.13	2029.10	latest CPython
🟣	3.14	2030.10	Python 3.14 alpha, beta (or RC) will be used.
⟲⚫	pypy-3.7	????.??
⟲⚫	pypy-3.8	????.??
⟲🔴	pypy-3.9	????.??
⟲🟠	pypy-3.10	????.??
⟲🟡	pypy-3.11	????.??	latest PyPy

apptest_system_list

Type:

string

Required:

no

Default Value:

'ubuntu windows macos macos-arm mingw64 ucrt64'

Possible Values:

A space separated list of system names.

Description:

The list of space-separated systems used for application testing.

Possible values

Native systems: ubuntu, windows, macos
MSYS2: msys, mingw32, mingw64, clang32, clang64, ucrt64

Icon	System	Used version	Comments
🪟	Windows	Windows Server 2025 (latest)
🐧	Ubuntu	Ubuntu 24.04 (LTS) (latest)
🍎	macOS	macOS Ventura 13 (latest)	While this marked latest, macOS Ventura 13 is already provided.
🍏	macOS-arm	macOS Sonoma 14 (latest)	While this marked latest, macOS Ventura 13 is already provided.
🟪	MSYS
⬛	MinGW32
🟦	MinGW64
🟫	Clang32
🟧	Clang64
🟨	UCRT64

Source: Images provided by GitHub

apptest_include_list

Type:

string

Required:

no

Default Value:

''

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be included into the list of application test variants.

For more details see include_list.

apptest_exclude_list

Type:

string

Required:

no

Default Value:

'windows-arm:3.9 windows-arm:3.10'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be excluded from the list of application test variants.

For more details see exclude_list.

apptest_disable_list

Type:

string

Required:

no

Default Value:

'windows-arm:pypy-3.10 windows-arm:pypy-3.11'

Possible Values:

A space separated list of <system>:<python_version> tuples.

Description:

List of space-separated <system>:<python_version> tuples to be temporarily disabled from the list of application test variants.
Each disabled item creates a warning in the workflow log.

For more details see disable_list.

codecov

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true', 'false'
Description:: If true, publish merged code coverage results and a merged unit test summary to CodeCov.
Secret CODECOV_TOKEN must be set.

codacy

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true', 'false'
Description:: If true, publish merged code coverage results to Codacy.
Secret CODACY_TOKEN must be set.

dorny

Type:: string
Required:: no
Default Value:: 'false'
Possible Values:: 'true', 'false'
Description:: If true, publish a merged unit test summary as pipeline result.

cleanup

Type:: string
Required:: no
Default Value:: 'true'
Possible Values:: 'true', 'false'
Description:: If false, do not remove intermediate artifacts.
This might help debugging artifact handovers between jobs.

Secrets

The workflow template uses the following secrets to publish results to other services.

PYPI_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish and upload packages on PyPI.

CODECOV_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish code coverage and unit test results to CodeCov.

CODACY_TOKEN

Type:: string
Required:: no
Default Value:: — — — —
Description:: The token to publish code coverage results to Codacy.

Outputs

This job template has no output parameters.

Optimizations

The following optimizations can be used to reduce the template’s runtime.

Todo

CompletePipeline::Optimizations Needs a list of optimizations.