{"id":13936085,"url":"https://github.com/explosion/wheelwright","last_synced_at":"2025-04-10T02:25:28.087Z","repository":{"id":37790097,"uuid":"146241411","full_name":"explosion/wheelwright","owner":"explosion","description":"🎡 Automated build repo for Python wheels and source packages","archived":false,"fork":false,"pushed_at":"2024-06-21T11:59:52.000Z","size":679,"stargazers_count":174,"open_issues_count":0,"forks_count":27,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-29T15:44:45.583Z","etag":null,"topics":["azure-pipelines","linux","macos","manylinux","manylinux1","multibuild","python","spacy","wheels","windows"],"latest_commit_sha":null,"homepage":"https://dev.azure.com/explosion-ai/public/_build?definitionId=15","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/explosion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-27T03:17:55.000Z","updated_at":"2024-08-02T23:33:38.000Z","dependencies_parsed_at":"2023-12-14T13:46:34.691Z","dependency_job_id":"547051fc-a699-40d6-8c4e-c08e1dac2cab","html_url":"https://github.com/explosion/wheelwright","commit_stats":{"total_commits":388,"total_committers":7,"mean_commits":55.42857142857143,"dds":0.6391752577319587,"last_synced_commit":"5fcf8ccd20002585d2e2c58c28264d728a1c3f95"},"previous_names":[],"tags_count":1440,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fwheelwright","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fwheelwright/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fwheelwright/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fwheelwright/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/explosion","download_url":"https://codeload.github.com/explosion/wheelwright/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248143621,"owners_count":21054809,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-pipelines","linux","macos","manylinux","manylinux1","multibuild","python","spacy","wheels","windows"],"created_at":"2024-08-07T23:02:22.089Z","updated_at":"2025-04-10T02:25:28.050Z","avatar_url":"https://github.com/explosion.png","language":"Python","readme":"\u003ca href=\"https://explosion.ai\"\u003e\u003cimg src=\"https://explosion.ai/assets/img/logo.svg\" width=\"125\" height=\"125\" align=\"right\" /\u003e\u003c/a\u003e\n\n# wheelwright\n\nThis repo builds **release wheels and source packages** for Python libraries\navailable as GitHub repositories. We're currently using it to build wheels for\n[spaCy](https://github.com/explosion/spaCy) and our\n[other libraries](https://github.com/explosion). The build repository integrates\nwith\n[Azure Pipelines](https://azure.microsoft.com/de-de/services/devops/pipelines/)\nand builds the artifacts for **macOS**, **Linux** and **Windows** on **Python\n3.6+**. All wheels are available in the\n[releases](https://github.com/explosion/wheelwright/releases).\n\n🙏 **Special thanks** to [Nathaniel J. Smith](https://github.com/njsmith/) for\nhelping us out with this, to [Matthew Brett](https://github.com/matthew-brett)\nfor [`multibuild`](https://github.com/matthew-brett/multibuild), and of course\nto the [PyPA](https://www.pypa.io/en/latest/) team for\n[`cibuildwheel`](https://github.com/pypa/cibuildwheel) and their hard work on\nPython packaging.\n\n\u003e ⚠️ This repo has been updated to use Azure Pipelines instead of Travis and\n\u003e Appveyor (see the [`v1`](https://github.com/explosion/wheelwright/tree/v1)\n\u003e branch for the old version). We also dropped support for Python 2.7. The code\n\u003e is still experimental and currently mostly intended to build wheels for our\n\u003e projects. For more details on how it works, check out the [FAQ](#faq) below.\n\n[![Azure Pipelines](https://img.shields.io/badge/Azure%20Pipelines-view%20builds-green.svg?longCache=true\u0026style=flat-square\u0026logo=azure-pipelines)](https://dev.azure.com/explosion-ai/public/_build?definitionId=15)\n\n\u003cimg width=\"803\" alt=\"\" src=\"https://user-images.githubusercontent.com/13643239/68905692-598efb00-0742-11ea-800b-630767858201.png\"\u003e\n\n## 🎡 Usage\n\n### Quickstart\n\n1. Fork or clone this repo and run `pip install -r requirements.txt` to install\n   its requirements.\n2. Generate a personal [GitHub token](https://github.com/settings/tokens/new)\n   with access to the `repo`, `user` and `admin:repo_hook` scopes and put it in\n   a file `github-secret-token.txt` in the root of the repo. Commit the changes.\n   Don't worry, the secrets file is excluded in the `.gitignore`.\n3. Set up a\n   [GitHub service connection](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops\u0026tabs=yaml#sep-github)\n   on Azure Pipelines with a personal access token and name it `wheelwright`.\n   This will be used to upload the artifacts to the GitHub release.\n4. Run `python run.py build your-org/your-repo [commit/tag]`.\n5. Once the build is complete, the artifacts will show up in the GitHub release\n   `wheelwright` created for the build. They'll also be available as release\n   artifacts in Azure Pipelines, so you can add a release process that uploads\n   them to PyPi.\n\n### Package requirements\n\nWheelwright currently makes the following assumptions about the packages you're\nbuilding and their repos:\n\n- The repo includes a `requirements.txt` that lists all dependencies for\n  building and testing.\n- The project uses `pytest` for testing and tests are shipped inside the main\n  package so they can be run from an installed wheel.\n- The package setup takes care of the whole setup and no other steps are\n  required. `build --sdist` builds the sdist and `cibuildwheel` builds the\n  wheels.\n\n### Setup and Installation\n\nMake a local clone of this repo:\n\n```bash\ngit clone https://github.com/explosion/wheelwright\n```\n\nNext, install its requirements (ideally in a virtual environment):\n\n```bash\npip install -r requirements.txt\n```\n\n[Click here to generate a personal GitHub token.](https://github.com/settings/tokens/new)\nGive it some memorable description, and check the box to give it the \"repo\"\nscope. This will give you some gibberish like\n`f7d4d475c85ba2ae9557391279d1fc2368f95c38`. Next go into your `wheelwright`\ncheckout, and create a file called `github-secret-token.txt` and write the\ngibberish into this file.\n\nDon't worry, `github-secret-token.txt` is listed in `.gitignore`, so it's\ndifficult to accidentally commit it. Instead of adding the file, you can also\nprovide the token via the `GITHUB_SECRET_TOKEN` environment variable.\n\n#### Security notes\n\n- Be careful with this gibberish; anyone who gets it can impersonate you to\n  GitHub.\n\n- If you're ever worried that your token has been compromised, you can\n  [delete it here](https://github.com/settings/tokens), and then generate a new\n  one.\n\n- This token is only used to access the `wheelwright` repository, so if you want\n  to be extra-careful you could create a new GitHub user, grant them access to\n  this repo only, and then use a token generated with that user's account.\n\n### Building wheels\n\nNote that the `run.py` script requires Python 3.6+. If you want to build wheels\nfor the `v1.31.2` tag inside the `explosion/cymem` repository, then run:\n\n```bash\ncd wheelwright\npython run.py build explosion/cymem v1.31.2\n```\n\nEventually, if everything goes well, you'll end up with wheels attached to a new\nGitHub release and in Azure Pipelines. You can then either publish them via a\ncustom release process, or download them manually:\n\n```bash\npython run.py download cymem-v1.31.2\n```\n\nIn Azure Pipelines, the artifacts are available via the \"Artifacts\" button. You\ncan also set up a\n[release pipeline](https://docs.microsoft.com/en-us/azure/devops/pipelines/release/?view=azure-devops)\nwith `twine` authentication, so you can publish your package to PyPi in one\nclick. Also see\n[this blog post](https://iscinumpy.gitlab.io/post/azure-devops-releases/) for an\nexample.\n\n\u003cimg width=\"1383\" alt=\"\" src=\"https://user-images.githubusercontent.com/13643239/68909644-51898800-074f-11ea-9b6f-38f8a9d5a4b0.png\"\u003e\n\n## 🎛 API\n\n### \u003ckbd\u003ecommand\u003c/kbd\u003e `run.py build`\n\nBuild wheels for a given repo and commit / tag.\n\n```bash\npython run.py build explosion/cymem v1.32.1\n```\n\n| Argument         | Type       | Description                                                                                                                                                 |\n| ---------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `repo`           | positional | The repository to build, in `user/repo` format.                                                                                                             |\n| `commit`         | positional | The commit to build.                                                                                                                                        |\n| `--package-name` | option     | Optional alternative Python package name, if different from repo name.                                                                                      |\n| `--universal`    | flag       | Build sdist and universal wheels (pure Python with no compiled extensions). If enabled, no platform-specific wheels will be built.                          |\n| `--llvm`         | flag       | Build requires LLVM to be installed, which will trigger an additional step in Windows build pipeline.                                                       |\n| `--rust`         | flag       | Build request Rust to be installed, which will trigger an additional step in Windows build pipeline. (Rust is installed by default in all other pipelines.) |\n| `--skip-tests`   | flag       | Don't run tests (e.g. if package doesn't have any). Only supported for `--universal` builds.                                                                |\n\n### \u003ckbd\u003ecommand\u003c/kbd\u003e `run.py download`\n\nDownload existing wheels for a release ID (name of build repo tag). The\ndownloaded wheels will be placed in a directory `wheels`.\n\n```bash\npython run.py download cymem-v1.31.2\n```\n\n| Argument     | Type       | Description                      |\n| ------------ | ---------- | -------------------------------- |\n| `release-id` | positional | Name of the release to download. |\n\n### Environment variables\n\n| Name                     | Description                                                                  | Default                               |\n| ------------------------ | ---------------------------------------------------------------------------- | ------------------------------------- |\n| `WHEELWRIGHT_ROOT`       | Root directory of the build repo.                                            | Same directory as `run.py`.           |\n| `WHEELWRIGHT_WHEELS_DIR` | Directory for downloaded wheels.                                             | `/wheels` in root directory.          |\n| `WHEELWRIGHT_REPO`       | Build repository in `user/repo` format.                                      | Automatically read from `git config`. |\n| `GITHUB_SECRET_TOKEN`    | Personal GitHub access token, if not provided via `github-secret-token.txt`. | -                                     |\n\n## ⁉️ FAQ\n\n### What does this actually do?\n\nThe `build` command uses the GitHub API to create a GitHub release in this repo,\ncalled something like `cymem-v1.31.2`. Don't be confused: this is not a real\nrelease! We're just abusing GitHub releases to have a temporary place to collect\nthe wheel files as we build them. Then it creates a new branch of this repo, and\nin the branch it creates a file called `build-spec.json` describing which\nproject and commit you want to build.\n\nWhen Azure Pipelines sees this branch, it springs into action, and starts build\njobs running on a variety of architectures and Python versions. These build jobs\nread the `build-spec.json` file, and then check out the specified\nproject/revision, build it, test it, and finally attach the resulting wheel to\nthe GitHub release we created earlier.\n\n### What if something goes wrong?\n\nIf the build fails, you'll see the failures in the Azure Pipelines build logs.\nAll artifacts that have completed will still be available to download from the\nGitHub release.\n\nIf you resubmit a build, then `run.py` will notice and give it a unique build ID\n– so if you run `run.py build explosion/cymem v1.31.2` twice, the first time\nit'll use the id `cymem-v1.31.2`, and the second time it will be\n`cymem-v1.31.2-2`, etc. This doesn't affect the generated wheels in any way;\nit's just to make sure we don't get mixed up between the two builds.\n\n### As a package maintainer, what do I need to know about the build process?\n\nWheels are built using `cibuildwheel`. For native linux `aarch64` builds, we use\n[`ec2buildwheel`](https://github.com/explosion/ec2buildwheel) to run\n`cibuildwheel` on an EC2 instance.\n\nOur projects use a single `requirements.txt` that includes both the build and\ntest requirements. You could imagine splitting those into two separate files, in\norder to make sure that dependency resolution is working, that we don't have any\nrun-time dependency on Cython, etc., but currently we don't.\n\nWe assume that projects use pytest for testing, and that they ship their tests\ninside their main package, so that you can run the tests directly from an\ninstalled wheel without access to a source checkout.\n\nFor simplicity, we assume that the repository name (in the clone URL) is the\nsame as the Python import name (in the `pytest` command). You can override this\non a case-by-case basis passing `--package ...` to the `build` command, but of\ncourse doing this every time is going to be annoying.\n\nAside from modifying the package setup, there isn't currently any way for a\nspecific project to further customize the build, e.g. if they need to build some\ndependency like libblis that's not available on PyPI.\n\n### I'm not Explosion, but I want to use this too!\n\nCurrently we'd recommend using `cibuildwheel` instead for most use cases, but\nwheelwright is under the MIT license, so feel free if it makes sense for your\nproject!\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexplosion%2Fwheelwright","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexplosion%2Fwheelwright","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexplosion%2Fwheelwright/lists"}