---

pyTooling Actions Documentation

pyTooling Actions are reusable steps and workflows for GitHub Actions easing the creation and maintenance of workflows for Python projects on GitHub.

Introduction

GitHub Actions workflows, actions and documentation are mostly focused on JavaScript/TypeScript as the scripting language for writing reusable CI code. However, Python being equally popular and capable, usage of JS/TS might be bypassed, with some caveats. This repository gathers reusable CI tooling for testing, packaging and distributing Python projects and documentation.

GitHub Actions

Post-Processing

• with-post-step

Deprecated

• Releaser

GitHub Action Job Templates

The following list categorizes all pre-defined job templates, which can be instantiated in a pipeline (GitHub Action Workflow):

All-In-One

• CompletePipeline
  
  
  
  

Pipeline Setup

• Parameters

• PrepareJob

• ExtractConfiguration

Documentation

• SphinxDocumentation

• LaTeXDocumentation

• PublishToGitHubPages

• CheckDocumentation

Testing and Code Coverage

• UnitTesting

• ApplicationTesting

Code Quality

• StaticTypeCheck

• CheckCodeQuality

• VerifyDocs (on hold)

• CheckMarketplaceMetadata

• code formatting (planned)

Build and Packaging

• Package

• InstallPackage (beta)

Publishing

• PublishOnPyPI

• PublishTestResults

• PublishCoverageResults

Releasing

• CheckReleaseVersion

• PublishReleaseNotes

• TagReleaseCommit

• UpdateVersionBranch

Cleanup

• CleanupArtifacts

Deprecated

• IntermediateCleanUp

• ArtifactCleanUp

Example Pipelines

CompletePipeline combines almost all job templates into a single workflow template. Python package/tool developers can instantiate it in their repository to get a full pipeline straight away. The minimal required modification is:

• Set the package_name input.

Everything else - Python versions, systems, documentation steps, code quality checks and the release path - is configurable and has a default.

Behavior

1. Classify ${{ github.ref }} into branch, tag or pull-request and find the pull-request associated with a release commit.

2. Extract Python project settings from pyproject.toml.

3. Compute the job matrices and the artifact names based on system, Python version, environment, … for job variants (unittest_python_version_list, unittest_system_list, unittest_include_list, unittest_exclude_list, unittest_disable_list).

4. Verify that the version in the Python code matches the version derived from the tag or pull-request title (version_file), and on a release commit, that PyPI has no release of this version yet (check_pypi_duplicate).
   Both checks run early. If one fails, the pipeline still runs, but steps 17 to 19 are refused.

5. Run unit tests using pytest and collect code coverage.

6. Verify type annotations using static typing analysis using mypy.

7. Check documentation coverage using docstr_coverage and interrogate.

8. Check code quality: security scanning using bandit (bandit), metrics and complexity using radon, linting using pylint (pylint).

9. Package code as source distribution and wheel.

10. Install the wheel on every target platform and verify the installed version.

11. Run application tests against the installed package using pytest (apptest, apptest_python_version_list, apptest_system_list).

12. Merge unit test results and code coverage results and publish them to GitHub (dorny), Codecov (codecov) and Codacy (codacy).

13. Delete the per-matrix-job artifacts that have been merged (cleanup).

14. Generate HTML and LaTeX documentations using Sphinx (documentation_steps: html, latex).

15. Translate LaTeX documentation to PDF using MikTeX (documentation_steps: pdf, miktex_image, miktex_update).

16. Publish documentation to GitHub Pages (documentation_steps: pages).

17. Tag a release commit, which triggers a second pipeline run for the new tag (auto_tag). The tag is refused, if the tag or a release page already exists.

18. Create a GitHub release page with text derived from the pull-request description and upload release assets.

19. Publish wheel to PyPI.

20. Delete the remaining artifacts (cleanup).

Steps 17 to 19 form the release path: step 17 runs on the release branch and creates the tag, steps 18 and 19 run in the tag pipeline that this tag triggers.

Steps 8, 11, 13, 14, 15, 16, 17 and 20 can be switched off by the parameters listed with them. A step that is switched off is skipped; every other step is executed as usual. Steps 1 to 7, 9, 10, 12, 18 and 19 are always executed - step 12 only decides per service whether the results are published.

Directory Structure

<RepositoryRoot>/
  .github/
    workflows/
      Pipeline.yml
  dist/
    requirements.txt
  docs/
    conf.py
    index.rst
    requirements.txt
  myPackage/
    ModuleA.py
    __init__.py
    py.typed
  tests/
    unit/
      TestA.py
      requirements.txt
    requirements.txt
  .editorconfig
  .gitignore
  LICENSE.md
  pyproject.toml
  README.md
  requirements.txt
  setup.py

Simple Package

name: Pipeline

on:
  push:
  workflow_dispatch:
  schedule:
# Every Friday at 22:00 - rerun pipeline to check for dependency-based issues
    - cron: '0 22 * * 5'

jobs:
  SimplePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:
      package_name: myPackage
      codecov:      true
      codacy:       true
      dorny:        true
    secrets:
      PYPI_TOKEN:    ${{ secrets.PYPI_TOKEN }}
      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
      CODACY_TOKEN:  ${{ secrets.CODACY_TOKEN }}

Namespace Package

name: Pipeline

on:
  push:
  workflow_dispatch:
  schedule:
# Every Friday at 22:00 - rerun pipeline to check for dependency-based issues
    - cron: '0 22 * * 5'

jobs:
  NamespacePackage:
    uses: pyTooling/Actions/.github/workflows/CompletePipeline.yml@r8
    with:
      package_namespace: myFramework
      package_name:      Extension
      codecov:           true
      codacy:            true
      dorny:             true
    secrets:
      PYPI_TOKEN:    ${{ secrets.PYPI_TOKEN }}
      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
      CODACY_TOKEN:  ${{ secrets.CODACY_TOKEN }}

pyproject.toml

[build-system]
requires = ["setuptools >= 83.0", "pyTooling ~= 8.17"]
build-backend = "setuptools.build_meta"

[tool.mypy]
packages = ["myPackage"]
python_version = "3.14"
strict = true
pretty = true
show_error_context = true
show_error_codes = true
namespace_packages = true
html_report = "report/typing"

[tool.pytest]
junit_xml = "report/unit/UnittestReportSummary.xml"

[tool.pyedaa-reports]
junit_xml = "report/unit/unittest.xml"

[tool.pytest.ini_options]
addopts = "--tb=native"
python_files = "*"
python_functions = "test_*"
filterwarnings = ["error::DeprecationWarning", "error::PendingDeprecationWarning"]
junit_logging = "all"

[tool.interrogate]
color = true
verbose = 1             # possible values: 0 (minimal output), 1 (-v), 2 (-vv)
fail-under = 59
ignore-setters = true

[tool.coverage.run]
branch = true
relative_files = true
omit = ["*site-packages*", "setup.py", "tests/unit/*"]

[tool.coverage.report]
skip_covered = false
skip_empty = true
exclude_lines = ["pragma: no cover", "raise NotImplementedError"]
omit = ["tests/*"]

[tool.coverage.xml]
output = "report/coverage/coverage.xml"

[tool.coverage.json]
output = "report/coverage/coverage.json"

[tool.coverage.html]
directory = "report/coverage/html"
title="Code Coverage of myPackage"

References

• hdl/containers#48

Contributors

• Patrick Lehmann (Maintainer)

• Unai Martinez-Corral

• and more…

License

This Python package (source code) is licensed under Apache License 2.0.
The accompanying documentation is licensed under Creative Commons - Attribution 4.0 (CC-BY 4.0).