#Calunga
Red Hat Trusted Libraries (RHTL) - Secure Python supply chain, from source to pip install
Calunga is Red Hat's initiative for building, verifying, and distributing secure Python wheels. Seven repositories form a complete supply chain pipeline - plumbing provides build infrastructure, index manages the package catalog and triggers builds, supported by companion repos for npm packaging, source verification, CI analysis, prioritization, and wheel patching.
#Why Calunga?
#The Problem
-
PyPI wheels are pre-built binaries - you trust the uploader
-
No guarantee of source-to-binary correspondence
-
Supply chain attacks on popular packages (typosquatting, account takeover)
-
No embedded SBOM or provenance metadata
-
Enterprise compliance requires verifiable supply chains
#Calunga's Answer
-
Build from source - 17 C/C++ libs + Python compiled in trusted environments
-
SBOM in every wheel - CycloneDXSBOM standard. Fromager generates CycloneDX SBOMs embedded as redhat.spdx.json, listing all components and their provenance.
redhat.spdx.json -
6 security scans per build (Clair, Snyk, ClamAV, Coverity, ShellCheck, Unicode)
-
Enterprise Contract PolicyKonflux policy framework. Validates that builds meet Red Hat compliance requirements (provenance, signatures, SBOM presence) before allowing releases. - Red Hat compliance enforcement
-
Attestation & signing - cosignSigstore tool for signing and verifying container images and artifacts. Used for attestation and provenance in the release pipeline. provenance
-
Hermetic builds - optional zero-network mode
#Getting Started
To install wheels from Red Hat Trusted Libraries, you need a service account from the terms-based registry and Python 3.12 installed on your system.
#pip Configuration
# pip.conf (venv/, ~/.config/pip/, or /etc/pip.conf)
[global]
index-url = https://<username>:<password>@packages.redhat.com/trusted-libraries/python/
# Install packages
python3.12 -m venv venv && source venv/bin/activate
pip install numpy
pip install --only-binary=:all: -r requirements.txt
#uv Configuration
# pyproject.toml
[[tool.uv.index]]
name = "trusted-libraries"
url = "https://packages.redhat.com/trusted-libraries/python"
default = true
# Export credentials
export UV_INDEX_INTERNAL_PROXY_USERNAME=<username>
export UV_INDEX_INTERNAL_PROXY_PASSWORD=<password>
#Viewing SBOMs
Every wheel contains an SBOM at <package>.dist-info/sboms/redhat.spdx.json:
pip download numpy==2.3.3
unzip numpy-2.3.3-0-cp312-cp312-manylinux_2_28_x86_64.whl -d extracted
cat extracted/numpy-2.3.3.dist-info/sboms/redhat.spdx.json
#Seven Repos, One System
#calunga-project-plumbing
Build Infrastructure
- Container images (builder, utils)
- TektonKubernetes-native CI/CD framework. Pipelines defined as YAML CRDs (Tasks, Pipelines, PipelineRuns). Each step runs in its own container. Used here via Konflux. tasks & pipelines
- Build scripts (FromagerPython wheel build orchestrator. Resolves full dependency graph from source distributions, builds each package in order, and embeds CycloneDX SBOMs. Ensures wheels are built from source - not downloaded pre-built from PyPI., auditwheelPost-build compliance tool. Inspects Linux wheels for shared library dependencies, bundles required .so files into the wheel, and relabels it with the correct manylinux platform tag.)
- SBOM patching & artifact collection
├── Containerfile # Multi-stage builder image
├── build_scripts/ # 37 build & install scripts
├── scripts/ # build-wheels entry point
└── tests/
tasks/ # Tekton bundles
pipelines/ # integration tests
#calunga-project-index
Package Registry
- 1,547 onboarded package JSONs
- Version detection (12h cron)
- Auto-PR creation & merge
- KonfluxRed Hat's enterprise CI/CD platform built on Tekton. Adds Enterprise Contract Policy enforcement, release management, nudge-based auto-updates, and OCI Trusted Artifacts for secure data passing between tasks. release pipeline
├── numpy.json
├── requests.json
└── ... (1,547 files)
hack/ # check-for-updates.py
.tekton/ # build-pipeline.yaml
#Companion Repositories
#npm-registry
npm Package Onboarding
- npm package definitions (
packages/) - Lint & identification scripts (
hack/) - Tekton build pipeline (
.tekton/)
#check-source-origin
Source Verification
- Python CLI: verify published sdists match VCS source
- Commands: resolve, download, diff, verify
#dfd (Dumpster Fire Diving)
CI Failure Analysis
- AI-powered failure analysis (FastAPI + Claude)
- React dashboard
- KubeArchive collector
#bounty-board
Package Prioritization
- Package prioritization tracker
- Python scripts + GitHub Actions
- Static HTML dashboard
#wheel-patcher
Wheel Post-Processing
- Patch wheel files (add SBOM, license files)
- Zero runtime dependencies
#Technology Stack
| Layer | Technology | Role |
|---|---|---|
| Languages | Bash, Python 3.12, YAML | Scripts, automation, pipeline definitions |
| Container Base | UBI8Universal Base Image. Red Hat's freely redistributable, minimal container base image. UBI8 provides a RHEL 8 userspace for the builder image. (builder), UBI10 (utils/index) | Red Hat Universal Base Images |
| CI/CD | TektonKubernetes-native CI/CD framework. Pipelines defined as YAML CRDs (Tasks, Pipelines, PipelineRuns). Each step runs in its own container. + KonfluxRed Hat's enterprise CI/CD platform built on Tekton. Adds Enterprise Contract Policy enforcement, release management, nudge-based auto-updates, and OCI Trusted Artifacts. | Kubernetes-native pipelines with enterprise features |
| Container Build | BuildahDaemonless container image builder. Builds OCI/Docker images from Containerfiles without requiring a Docker daemon. Used by Tekton tasks to produce the builder and utils images. | Daemonless OCI image builder |
| Wheel Builder | FromagerPython wheel build orchestrator. Resolves full dependency graph from source distributions, builds each package in order, and embeds CycloneDX SBOMs. (≥0.81.0) | Source-only wheel build orchestrator |
| Compliance | auditwheelPost-build compliance tool. Inspects Linux wheels for shared library dependencies, bundles required .so files, and relabels with the correct manylinux platform tag. | manylinuxPEP-defined platform compatibility standard for Linux Python wheels. Specifies maximum glibc version and allowed shared library dependencies. manylinux_2_28 targets glibc 2.28+ (RHEL 8+). repair & .so bundling |
| Security | Clair, Snyk, ClamAV, Coverity, ShellCheck | 6 scan types per build |
| SBOM | CycloneDX / SPDXSBOM standards. Fromager generates CycloneDX SBOMs embedded in each wheel as redhat.spdx.json, listing all components and their provenance. | Software Bill of Materials in every wheel |
| Registries | Quay.io, packages.redhat.com | Images & wheels |
| Automation | GitHub Actions, NudgeKonflux auto-update mechanism. When plumbing publishes a new image or task bundle digest, nudge creates PRs in downstream repos to update pinned SHA256 references. | Version detection & dependency updates |
| Compilers | gcc-toolset-14Red Hat's SCL-packaged GCC 14 compiler suite for UBI8/RHEL 8. Provides modern C/C++ compilation without replacing system GCC., clang 21.1.5, Rust 1.95.0 | Native extension compilation |
#End-to-End Flow
#Plumbing: Container Images
#Builder Image: Source-Built Libraries
| Library | Version | Why |
|---|---|---|
| OpenSSL | 3.5.4 | TLS, cryptography wheels |
| curl | 8.17.0 | HTTP client for downloads |
| git | 2.51.2 | Source fetching |
| SQLite | 3.51.0 | Python stdlib module |
| OpenBLAS | latest | NumPy/SciPy linear algebra |
| libjpeg-turbo | latest | Pillow image processing |
| libxml2 / libxslt | latest | lxml XML parsing |
| libyaml | latest | PyYAML C extension |
| zlib / bzip2 / zstd | latest | Compression |
| libffi | latest | ctypes / cffi |
| mpdecimal | latest | Python decimal module |
| libomp | latest | OpenMP parallelism |
| libpng / libtiff | latest | Image format support |
| Tcl/Tk | 8.6.17 | CPython stdlib GUI module |
| libxcrypt | 4.5.1 | crypt() password hashing |
#Build Details
- Base: UBI8 (RHEL 8 userspace)
- Python: CPython 3.12.12 from source
- Alt runtime: PyPy 3.11
- C/C++: gcc-toolset-14
- C/C++ (static): clang 21.1.5
- Rust: 1.95.0
- Deps: pip-compileDependency resolver from pip-tools. Generates fully pinned requirements.txt with SHA256 hashes from a loose requirements.in, ensuring reproducible installs. --require-hashes
#Key Tools
- FromagerPython wheel build orchestrator. Resolves full dependency graph from source distributions, builds each package in order, and embeds CycloneDX SBOMs. - orchestrates full dep graph builds
- auditwheelPost-build compliance tool. Inspects Linux wheels for shared library deps, bundles required .so files, relabels with correct manylinux platform tag. - bundles .so, sets manylinux tag
- pip-compile - hash-pinned requirements
#Key Scripts & Tekton Pipelines
#Scripts
| Script | Purpose |
|---|---|
build-wheels | Main entry point: Fromager resolve → build → auditwheel → SBOM patch |
collect-build-files | Gathers SBOM + build artifacts from completed builds |
patch-sbom-purl | Adds file_name qualifier to PURLs, renames SBOM to redhat.spdx.json |
check-for-updates.py | Async PyPI/Pulp version checker (index repo) |
identify-packages | Detects which package JSONs changed in PR |
add_system_deps_to_sbom.py | Adds system dependency metadata to SBOM |
test-add-system-deps | Tests for add_system_deps_to_sbom.py |
#Build Pipeline (18 tasks)
#Builder Image: Self-Tests
The builder image runs a comprehensive test suite (builder/tests/run_tests.sh) to validate every component before any wheel build occurs.
Verifies expected number of Python installations (CPython, PyPy, GraalPy) are present and functional.
Every interpreter tested for correct OpenSSL configuration and cert verification.
CPython optional modules (sqlite3, ssl, ctypes, decimal, etc.) verified loadable.
C extension project (
forty-two) built, repaired with auditwheel, installed and executed per interpreter.Source-built libs verified via
pkg-config and ldconfig (libjpeg, libyaml, libxml2, etc.).Autotools and CMake projects compiled against installed headers and libraries (SQLite, C++).
auditwheel, autoconf, automake, libtool, patchelf, git, git-lfs, cmake, swig, nox all verified.
Wheels installed via both pip and uv to verify compatibility with modern tooling.
#Index: Package Management
#1,547 Onboarded Packages
Each package has a JSON file in onboarded_packages/:
{
"version": "2.4.6",
"ignored_versions": ["0.9.6", "0.9.8", "2.4.3"]
}
version- current target version to buildignored_versions- unsupported, vulnerable, or skipped releases
#Package Lifecycle
check-for-updates.py queries PyPI + Pulp (every 12h)identify-packages detects changed JSONsbuild-python-wheels task#Automated Version Detection
Smart filtering - pre-releases, yanked releases, and
ignored_versions all excluded.Labels: PRs tagged "automated build" for tracking.
#Index Build Pipeline
#Onboarding New Packages
Adding a new package to Calunga uses hack/onboard_package.py in the index repo:
python hack/onboard_package.py <package-name>ignored_versionsonboarded_packages/<name>.json#Input
python hack/onboard_package.py flask
#Output: flask.json
{
"version": "3.1.1",
"ignored_versions": ["0.1", "0.2", ...]
}
#How Repos Work Together
#Dependency Chain
#Update Propagation
#Integration Testing
#Cross-OS Verification
Built wheels tested across 6 OS images:
#Per-Wheel Test Steps
install-and-import-wheels.yamlSpans both repos - plumbing defines it, index triggers it.
#Release Pipeline
After wheels pass security scans and Enterprise Contract checks, the release pipeline handles attestation, signing, and upload to the registry.
#Attestation Flow
- SHA256 hash computed for each wheel
- SLSA v0.2 provenance predicate generated
- Signed with cosign (key from K8s secret)
- DSSE envelope converted to PEP 740 format
- Attestation uploaded alongside wheel
#Upload Logic
- Skip if wheel already exists in Pulp
- Skip if attestation file missing
- Fail if any upload errors or missing attestations
- Summary report: uploaded / skipped / failed counts
#Observability
#Log Forwarding
A separate GitHub Actions workflow (forward_logs.yml) triggers after every version-check run:
get_new_package_versions workflow completesgh run view --log#PR Tracking
- Automated version PRs labeled "automated build"
- Konflux nudge PRs labeled "konflux-nudge"
- Branch naming:
update-<package>==<version> - All PRs auto-merged with rebase strategy
#Build Artifacts
- PR builds expire after 5 days
- Push builds permanent (tagged by git revision)
- All images referenced by SHA256 digest
#Supply Chain Security
All Python deps use --require-hashes. Container/task refs pinned by SHA256.
17 C/C++ libs compiled from source with verified checksums. No pre-built binaries.
Every wheel contains redhat.spdx.json with supplier info and PURLs.
6 scans per build: Clair, Snyk, ClamAV, Coverity, ShellCheck, Unicode.
Red Hat compliance enforcement on every release.
Provenance and signing via cosign (Sigstore).
Optional fully isolated mode - zero network access during build.
Data passes between tasks via OCI artifacts, not shared volumes.
#Support Matrix & Roadmap
#Python Versions
| Version | Status |
|---|---|
| Python 2.x | Not supported |
| Python 3.9 – 3.10 | Not supported |
| Python 3.12 | Fully supported |
| Python 3.11, 3.13, 3.14 | Planned |
#Alternative Python Runtimes
The builder image includes multiple Python runtimes beyond CPython, all SHA256-pinned:
| Runtime | Versions | Architectures |
|---|---|---|
| CPython | 3.12.12 (from source) | x86_64, aarch64 |
| PyPy | 3.11 | x86_64, aarch64 |
| GraalPy | 3.11 (24.2.2), 3.12 (25.0.1) | x86_64, aarch64 |
python_versions.json for reproducibility.
#Technology Glossary
#Quick Reference
#Key Paths (Plumbing)
builder/Containerfile |
Main builder image (393 lines) |
builder/scripts/build-wheels |
Wheel build entry point |
builder/build_scripts/ |
21 library build scripts |
builder/requirements.txt |
Hash-pinned Fromager deps |
tasks/build-python-wheels-oci-ta.yaml |
Tekton task definition |
pipelines/wheel-integration-test.yaml |
Cross-OS test pipeline |
utils/scripts/pulp-upload |
Pulp registry upload with attestation |
utils/scripts/generate-and-sign-attestations |
cosign attestation + PEP 740 conversion |
.tekton/ |
PipelineRun definitions |
#Key Paths (Index)
onboarded_packages/*.json |
1,547 package definitions |
hack/check-for-updates.py |
Version detection script |
hack/identify-packages |
Changed-package detector |
hack/onboard_package.py |
New package onboarding script |
hack/onboard.sh |
Package onboarding shell wrapper |
hack/batch-onboard.sh |
Bulk package onboarding |
hack/build-locally.sh |
Local development build |
hack/debug-package.sh |
Package build debugging |
hack/generate-available-packages.py |
Available packages list generator |
hack/merge-bot |
Auto-merge eligible PRs |
hack/rebase-version-prs |
Rebase version update PRs |
hack/replace-package |
Replace existing package definition |
hack/retry-onpush.sh |
Retry failed on-push builds |
.tekton/build-pipeline.yaml |
Main build pipeline |
konflux/ecp.yaml |
Enterprise Contract Policy |
#User Configuration
# pip.conf
[global]
extra-index-url =
https://packages.redhat.com/trusted-libraries/python/simple/
# pyproject.toml (uv)
[[tool.uv.index]]
url = "https://packages.redhat.com/trusted-libraries/python/simple/"
name = "rhtl"
#Key Numbers
| Packages | 1,547+ |
| Source-built libs | 17 |
| Security scans | 6 types |
| Test OS targets | 6 |
| Pipeline tasks | 18 |
| Version check interval | 12 hours |
| Release grace period | 7 days |