{"id":20286947,"url":"https://github.com/psf/sboms-for-python-packages","last_synced_at":"2025-04-11T09:37:40.758Z","repository":{"id":261140057,"uuid":"881473996","full_name":"psf/sboms-for-python-packages","owner":"psf","description":"Software Bill-of-Materials documents for Python packages","archived":false,"fork":false,"pushed_at":"2025-02-21T21:50:06.000Z","size":3044,"stargazers_count":35,"open_issues_count":11,"forks_count":4,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-04-06T06:17:15.379Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/psf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.CC0-1.0","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-31T16:37:15.000Z","updated_at":"2025-02-21T21:50:09.000Z","dependencies_parsed_at":"2024-12-05T23:18:39.900Z","dependency_job_id":"89b60744-bfe1-470d-83a2-9e1af13eda81","html_url":"https://github.com/psf/sboms-for-python-packages","commit_stats":null,"previous_names":["sethmlarson/sboms-for-python-packages","psf/sboms-for-python-packages"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Fsboms-for-python-packages","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Fsboms-for-python-packages/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Fsboms-for-python-packages/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Fsboms-for-python-packages/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/psf","download_url":"https://codeload.github.com/psf/sboms-for-python-packages/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248368718,"owners_count":21092440,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T14:37:32.012Z","updated_at":"2025-04-11T09:37:40.736Z","avatar_url":"https://github.com/psf.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Software Bill-of-Materials for Python packages\n\n\u003e **NOTE:**\n\u003e This project is a work-in-progress and is soliciting feedback from experts in the respective areas. Please open questions as [new GitHub issues](https://github.com/sethmlarson/sboms-for-python-packages/issues). Thank you for your feedback!\n\n* [Terminology](#terminology)\n* [Motivation](#motivation)\n* [Rationale](#rationale)\n* [Proposal](#proposal)\n* [How does it all fit together?](#how-does-it-all-fit-together)\n* [License](#license)\n\n## Terminology\n\nThis repository uses terminology consistent with the [Python Packaging User Guide Glossary](https://packaging.python.org/en/latest/glossary/).\nNote for non-Python package users: this terminology may be different compared to other software ecosystems,\nplease be aware of these differences when reading and contributing.\n\n## Motivation\n\n### Regulations\n\nSoftware Bill-of-Materials documents (SBOMs) are a technology and ecosystem-agnostic\nformat for describing software composition, provenance, and other metadata. SBOMs\nare required by recent software security regulations, like the [Secure Software Development Framework](https://csrc.nist.gov/Projects/ssdf) (SSDF)\nand the [Cyber Resilience Act](https://digital-strategy.ec.europa.eu/en/policies/cyber-resilience-act) (CRA).\nDue to their inclusion in these regulations, the demand for SBOM documents of open source projects is expected to be high.\nFor example, the Tennessee Valley Authority has already begun attempting to collect SBOM documents\nfrom open source projects like CPython.\n\nThe goal is to minimize the demands on open source project maintainers by enabling\nopen source users that need SBOMs to self-serve using existing tooling. Another goal\nis to enable contributions to create or annotate projects with SBOM information from those\nsame users that need SBOM documents from projects. Today there is no mechanism to propagate\nthe results of those contributions into SBOM tooling so there is no reason to contribute this type of work.\n\n### Phantom dependencies\n\nPython packages are particularly affected by the \"[phantom dependency](https://www.endorlabs.com/learn/dependency-resolution-in-python-beware-the-phantom-dependency)\" problem,\nwhere software that isn't written in Python are included in Python packages\nfor many reasons, such as ease of installation and compatibility with standards:\n\n* Python serves scientific, data, and machine-learning use-cases which use compiled or non-Python languages like Rust, C, C++, Fortran, JavaScript, and others.\n* The Python wheel format is preferred by users due to the ease-of-installation. No code is executed during the installation step, only extracting the archive.\n* The Python wheel format requires bundling shared compiled libraries without a method to encode metadata about these libraries.\n\nThis software can't be described accurately using Python package metadata and so\nis likely to be missed by software composition analysis (SCA) software which can mean\nvulnerable software components aren't reported accurately.\n\n## Rationale\n\nAttempting to adopt every field offered by SBOM standards into Python core metadata would result in an explosion of\nnew core metadata fields including needing to keep up-to-date as\nSBOM standards continue to evolve to suit new needs in that space.\nInstead, this proposal delegates metadata to SBOM documents and formats\nand adds Python package metadata for linking to SBOM documents contained within a Python package.\n\nThis standard also doesn't aim to replace Python core metadata with SBOMs, instead\nfocusing on the SBOM information being supplemental to core metadata.\nCore metadata fields MUST be used as the authoritative location for information\nabout a Python package itself and included SBOMs MUST only contain information\nabout dependencies included in the package archive OR information\nabout the software in the package that can't be encoded into core metadata\nbut is relevant for the SBOM use-case (such as, \"software identifier\", \"purpose\", \"support level\", etc).\n\n## Proposal\n\nToday there is no method to encode information for cross-language/ecosystem software\ndependencies into Python package metadata. This project proposes using SBOM formats\nfor this purpose and allowing SBOM documents to be included in Python packages archives\nto self-describe software within those package archives. Included SBOM documents are then\nreferenced using a new Python metadata field `Sbom-File` so they are discoverable within a Python package.\n\nFor example, a Python wheel for numpy containing an SBOM document:\n\n```\nnumpy-2.1.3.dist-info/sboms/bundled.cdx.json\n```\n\n...where that SBOM file contains information about software like `lapack-lite` which the numpy team bundles themselves\nand `libgfortran` which was \"repaired\" into the wheel by `auditwheel`:\n\n```json5\n{\n  \"bomFormat\": \"CycloneDX\",\n  \"specVersion\": \"1.6\",\n  \"metadata\": {\n    // Primary component is numpy\n    \"component\": {\n      \"type\": \"library\",\n      \"name\": \"numpy\",\n      \"version\": \"2.1.3\"\n    }\n  },\n  // Sub-components described here:\n  \"components\": [\n    {\n      \"name\": \"lapack-lite\",\n      // ...\n    },\n    {\n      \"name\": \"libgfortran\",\n      \"purl\": \"pkg:rpm/almalinux/libgfortran@8.5.0-22\"\n    }\n    // ...\n  ]\n}\n```\n\nThe proposal would require:\n\n### Survey of Python packages, Python package tools, and SBOM tooling\n\nSurvey Python package tools, answer the questions \"Can these tools adopt this standard?\" \"How difficult is creating quality SBOM information for Python projects?\"\n\n* Python packages using non-Python dependencies (numpy, pandas, jupyter, cryptography, pydantic)\n* Build backends supporting cross-ecosystem software ([setuptools](https://github.com/pypa/setuptools), [Maturin](https://github.com/PyO3/maturin))\n* Wheel processing tools for vendoring dependencies ([auditwheel](https://github.com/pypa/auditwheel), [repairwheel](https://github.com/jvolkman/repairwheel), [delocate](https://github.com/matthew-brett/delocate), [delvewheel](https://github.com/adang1345/delvewheel))\n* Tools for vendoring Python dependencies ([vendoring](https://github.com/pradyunsg/vendoring))\n* Runtimes for building wheels ([cibuildwheel](https://github.com/pypa/cibuildwheel), [multibuild](https://github.com/multi-build/multibuild))\n\nSurvey SBOM tools and standards, answer the question: \"how useful is the information encoded by this standard?\"\nUsing popular SBOM generation tools, can SBOMs be generated to meet the following regulations?\n\n* [Cyber Resilience Act](https://digital-strategy.ec.europa.eu/en/library/cyber-resilience-act) (CRA)\n* [NIST Secure Software Development Framework](https://csrc.nist.gov/Projects/ssdf) (SSDF)\n* [NTIA Minimum Elements For a Software Bill of Materials](https://www.ntia.gov/report/2021/minimum-elements-software-bill-materials-sbom) (rev3 from CISA)\n\nThe survey will inform whether this proposal can be adopted by these tools and how useful this standard would be to downstream consumers.\nSome of this survey will come in the form of a pre-PEP and PEP discussion of the subproject below.\n\n### New standards (PEP) for encoding SBOM information\n\nThis subproject will require the above subproject to be complete to be \"ready for submission\" to be reviewed and approved,\nbut is not blocked on starting the draft PEP and discussion process.\n\n* New core metadata field: `Sbom-File` for specifying the location(s) of one or more SBOM files in a package. New package metadata version for the new field.\n* pyproject.toml field `sbom-files` added to `[project]` table for conditional and unconditional inclusion of SBOM documents in Python packages.\n  Conditional SBOM files use markers.\n* New directory for containing SBOM files (`/.dist-info/sboms/`) in Python packages and installed locations.\n  This directory will be similar to the [`/.dist-info/licenses/` directory specified in PEP 639](https://peps.python.org/pep-0639/#license-files-in-project-formats).\n* How to self-reference software within a Python package as an SBOM component.\n* Provide all examples using common SBOM standards like CycloneDX and SPDX. Provide no preference to either standard.\n* Set of filtering requirements to add to popular package indexes like PyPI to ensure other tools are adhering to standards.\n* Adoption by two build backends/tools and PyPI.\n\n### Golden examples for SBOM tool developers\n\nThis subproject aims to provide high-quality calibration materials\nfor SBOM tool developers. This subproject can be worked on concurrently\nto the above two subprojects and then updated later once the above PEP is accepted.\n\n* Create an informational PEP on how to transform Python package metadata and included SBOM documents into one SBOM document\n  for an installed Python package.\n* Create a list of example projects and \"golden\" SBOMs against complex Python packages (pip, numpy, pandas, pydantic, etc)\n\nThese examples can then be used by SBOM tool developers to verify their software is working\nfor Python packages.\n\n## How does it all fit together?\n\nThere's a [published blog post with diagrams](https://sethmlarson.dev/visualizing-the-python-package-sbom-data-flow#end-to-end-sbom-data-flow) on the end-to-end data flow.\n\n* If project dependencies are checked into version control then an SBOM file\n  can be created and checked into version control alongside those dependencies.\n  This SBOM file is referenced within `pyproject.toml` under `project.sbom-files`.\n* If the bundled dependency is conditional (for example, \"Windows only\"), then `markers` will be applied to the `sbom-files` entry.\n* The Python package build backend (ie `setuptools`) processes the `sbom-files` field and adds\n  corresponding `Sbom-File` Python package metadata for every SBOM file that is referenced and matches with markers.\n* Build backends might also generate their own SBOM documents to include in Python packages. For example\n  Maturin does Rust dependency management so could generate an SBOM document for all Rust dependencies\n  to include in the Python package.\n* After Python wheels are generated, wheels can be further augmented or patched with\n  shared and dynamic libraries by bundling (ie `auditwheel`). Tools that augment an existing wheels\n  should generate their own SBOM document that details the shared libraries that were bundled\n  if that information is available. For example, cibuildwheel for manylinux commonly uses AlmaLinux's packaging system.\n* Python packages are uploaded to PyPI. PyPI does some quality checks for whether SBOM files exist\n  and aren't invalid.\n* Python packages are installed from PyPI. When installed, the SBOM files are stored in `.dist-info/sboms/...`\n  directory in the environment.\n* SBOM tools generating an SBOM for a Python package or environment with a list of installed Python packages\n  can inspect these SBOM files and use them when generating an SBOM for a package or environment.\n\n## License\n\nThis repository is placed in the public domain or under the [CC0-1.0-Universal license](https://creativecommons.org/publicdomain/zero/1.0/), whichever is more permissive.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsf%2Fsboms-for-python-packages","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpsf%2Fsboms-for-python-packages","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsf%2Fsboms-for-python-packages/lists"}