{"id":15576430,"url":"https://github.com/njsmith/cymem-wheels","last_synced_at":"2026-06-23T04:31:08.624Z","repository":{"id":145901783,"uuid":"145831231","full_name":"njsmith/cymem-wheels","owner":"njsmith","description":null,"archived":false,"fork":false,"pushed_at":"2018-08-27T03:20:15.000Z","size":50,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-19T17:39:26.889Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/njsmith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-23T09:25:26.000Z","updated_at":"2018-08-27T03:17:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"2cb7d94e-8544-4951-8085-80f90d5ea0b1","html_url":"https://github.com/njsmith/cymem-wheels","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/njsmith/cymem-wheels","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fcymem-wheels","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fcymem-wheels/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fcymem-wheels/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fcymem-wheels/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/njsmith","download_url":"https://codeload.github.com/njsmith/cymem-wheels/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fcymem-wheels/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34675970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-23T02:00:07.161Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T18:49:08.316Z","updated_at":"2026-06-23T04:31:08.589Z","avatar_url":"https://github.com/njsmith.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# What is this?\n\nThis is a magic Github repo that builds release wheels for Explosion\nAI projects.\n\n\n# How do I use it to build wheels?\n\n## Setup\n\nMake a local clone of this repo:\n\n```sh\n$ git clone https://github.com/explosion/magic-build-repo.git\n```\n\n[Click here to generate a personal Github\ntoken.](https://github.com/settings/tokens/new) Give it some memorable\ndescription, and check the box to give it the \"repo\" scope. This will\ngive you some gibberish like:\n\n```\nf7d4d475c85ba2ae9557391279d1fc2368f95c38\n```\n\nSecurity notes:\n\n* Be careful with this gibberish; anyone who gets it can impersonate you\n  to Github.\n\n* If you're ever worried that your token has been compromised, you can\n  [delete it here](https://github.com/settings/tokens), and then\n  generate a new one.\n\n* This token is only used to access the magic-build-repo repository,\n  so if you want to be extra-careful you could create a new Github\n  user, grant them access to this repo only, and then use a token\n  generated with that user's account.\n\nNext go into your `magic-build-repo` checkout, and create a file\ncalled `github-secret-token.txt`. Write the gibberish into this file:\n\n```sh\n$ cd magic-build-repo\n$ my-editor github-secret-token.txt\n$ cat github-secret-token.txt\nf7d4d475c85ba2ae9557391279d1fc2368f95c38\n```\n\nDon't worry, `github-secret-token.txt` is listed in `.gitignore`, so\nit's difficult to accidentally commit it.\n\nInstall our script's dependencies: `pip install -Ur mb-requirements.txt`\n\nNow you're ready to build wheels.\n\n\n## Actually building a wheel\n\nIf you want to build wheels for the `v1.31.2` tag inside the\n`explosion/cymem` repository, then run:\n\n```sh\n$ cd magic-build-repo\n$ python mb.py magic-build cymem v1.31.2\n```\n\nEventually, if everything goes well, you'll end up with wheels in a\ndirectory named `cymem-v1.31.2-wheels`:\n\n```sh\n$ ls cymem-v1.31.2-wheels\ncymem-1.32.1-cp27-cp27mu-manylinux1_i686.whl\ncymem-1.32.1-cp27-cp27mu-manylinux1_x86_64.whl\n... and so on ...\n```\n\nNow you can upload them to PyPI:\n\n```sh\n$ twine upload cymem-v1.31.2-wheels/*.whl\n```\n\nThis only uploads wheels. Don't forget to also upload an sdist!\n\n\n# What did that actually do?\n\nThe `magic-build` command uses the Github API to create a Github\nrelease in this repo, called something like `cymem-v1.31.2-wheels`.\nDon't be confused: this is not a real release! We're just abusing\nGithub releases to have a temporary place to collect the wheel files\nas we build them.\n\nThen it creates a new branch of this repo, and in the branch it\ncreates a file called `build-spec.json` describing which project and\ncommit you want to build.\n\nWhen Travis and Appveyor see this branch, they spring into action, and\nstart build jobs running on a variety of architectures and Python\nversions. They're configured to the `build-spec.json` file, and then\ncheck out the specified project/revision, build it, test it, and\nfinally attach the resulting wheel to the Github release we created\nearlier.\n\nThe `magic-build` command waits until Travis and Appveyor have\nfinished. If they succeeded, then it downloads all the wheels from the\nGithub release into a local directory, ready for uploading to PyPI.\n\n\n# What if something goes wrong?\n\nIf the build fails, the script will say so, and won't download any\nwheels. While it runs it prints links to the Travis/Appveyor build\nlogs, the release object, etc., which you can use to get more details\nabout what went wrong.\n\nIf for some reason you want to download the wheels from an existing\nrelease, you can do that with:\n\n```sh\n$ python mb.py download-release-assets cymem-v1.31.2-wheels\n```\n\nThis might be useful if you accidentally killed a `magic-build`\ncommand before it finished, or if you want to get partial results from\na failed build.\n\nIf you resubmit a build, then `mb.py` will notice and give it a unique\nbuild id – so if you run `mb.py submit cymem v1.31.2` twice, the first\ntime it'll use the id `cymem-v1.31.2-wheels`, and the second time it\nwill be `cymem-v1.31.2-wheels-2`, etc. This doesn't affect the\ngenerated wheels in any way; it's just to make sure we don't get mixed\nup between the two builds.\n\n\n# As a package maintainer, what do I need to know about the build process?\n\nEssentially we run:\n\n```sh\n# Setup\n$ git clone https://github.com/explosion/PROJECT-NAME.git checkout\n$ cd checkout\n$ git checkout REVISION\n\n# Build\n$ cd checkout\n$ pip install -Ur requirements.txt\n$ python setup.py bdist_wheel\n\n# Test\n$ cd empty-directory\n$ pip install -Ur ../checkout/requirements.txt\n$ pip install THE-BUILT-WHEEL\n$ pytest --pyargs PROJECT-NAME\n```\n\nSome things to note:\n\nThe build/test phases currently have varying levels of isolation:\n\n* On Windows, they use the same Python environment.\n* On macOS, they use different virtualenvs.\n* On Linux, they run in different docker containers, which are running\n  different Linux distros, to make sure the binaries really are\n  portable.\n\nWe use the same `requirements.txt` for both building and testing. You\ncould imagine splitting those into two separate files, in order to\nmake sure that dependency resolution is working, that we don't have\nany run-time dependency on Cython, etc., but currently we don't. If\ndoing this then it would also make sense to be more careful about\nsplitting up the build/test environments, and about separating the\n`mb.py` helper script from the build/test environments.\n\nWe assume that projects use pytest for testing, and that they ship\ntheir tests inside their main package, so that you can run the tests\ndirectly from an installed wheel without access to a source checkout.\n\nFor simplicity, we assume that the repository name (in the clone URL)\nis the same as the Python import name (in the `pytest` command). You\ncan override this on a case-by-case basis passing `--clone-url ...` to\nthe `magic-build` command, but of course doing this every time is\ngoing to be annoying.\n  \nAside from modifying `setup.py`, there isn't currently any way for a\nspecific project to further customize the build, e.g. if they need to\nbuild some dependency like libblis that's not available on PyPI.\n\n\n# What do I need to know to maintain this repo itself?\n\nInternally, this builds on [Matthew Brett's multibuild\nproject](https://github.com/matthew-brett/multibuild). A snapshot of\nmultibuild is included as a git submodule, in the `multibuild/`\ndirectory. You might want to update that submodule occasionally to\npull in new multibuild fixes:\n\n```sh\n$ cd multibuild\n$ git pull\n$ cd ..\n$ git commit -am \"Updated multibuild snapshot\n```\n\nMultibuild was originally designed to do Linux and macOS builds, and\nwith the idea that you'd create a separate repo for each project with\ncustom configuration. We kluge it into working for us by reading\nconfiguration out of the `build-spec.json` file and using it to\nconfigure various settings.\n\nMost of the actual configuration is in the `.travis.yml` and\n`appveyor.yml` files. These use the `mb.py` script to perform various\nactions, ranging from parsing the `build-spec.json` file, to uploading\nwheels, to (on Windows) coordinating the whole build/test process.\n\nUnfortunately, there are currently no automated tests. Sorry :-(. They\nwould need Github permissions and all kinds of things.\n\n\n## Secrets\n\nTo upload the wheels, the CI builds need access to a Github token that\nhas write permissions on this repository. Specifically, you need a\ntoken with `repo` access to this repo, and it should be stored in an\nenvvar named `GITHUB_SECRET_TOKEN`. To do this, we use\n[Appveyor's](https://www.appveyor.com/docs/build-configuration/#secure-variables)\nand\n[Travis's](https://docs.travis-ci.com/user/environment-variables/#defining-encrypted-variables-in-travisyml)\nsupport for encrypted environment variables.\n\nFirst, create your Github token (as in the \"setup\" section above).\nThen, for Appveyor:\n\n* Log in as whichever Appveyor user initially enabled Appveyor for the\n  magic build repo\n\n* Go to https://ci.appveyor.com/tools/encrypt\n\n* Paste in the token (just the token, do **not** include the\n  `GITHUB_SECRET_TOKEN=` part)\n\n* Copy the \"Encrypted value\" it gives you back into `appveyor.yml`\n\nAnd for Travis, we need to get a copy of the `travis` program, and run\n`travis encrypt GITHUB_SECRET_TOKEN=\u003c...\u003e` (notice that here you *do*\nhave to include the `GITHUB_SECRET_TOKEN=` in the encrypted text). On\nUbuntu, I was able to get it working by doing:\n  \n```sh\n$ sudo apt install ruby-dev\n$ gem install --user-install travis\n$ ~/.gem/ruby/*/bin/travis encrypt GITHUB_SECRET_TOKEN=f7d4d475c85ba2ae9557391279d1fc2368f95c38\n```\n\nThen copy the gibberish it gives you into `.travis.yml`.\n\n\n## Other CI configuration\n\nTravis and Appveyor are both configured to not build tags (because we\ndon't want our Github releases to trigger builds), and to only build\nbranches matching the pattern `branch-for-*` (because we don't want\ncommits to `master` to trigger builds). This is all done through the\n`.yml` files.\n\nOn Travis, we can also use the `.yml` to disable building of PRs, and\nso we do. On Appveyor, though this, isn't possible! In fact, Appveyor\nsimply doesn't provide any way to disable PR builds in general. But,\nthere is a hack: if we stop Github from telling Appveyor about PRs,\nthen it can't build them.\n\nTherefore, after setting up Appveyor, we go the Github settings for\nour repository, in [the Webhooks\nsection](https://github.com/explosion/magic-build-repo/settings/hooks),\nclick \"Edit\" on the Appveyor webhook, **un**check the box labeled\n\"Pull requests\", and then click \"Update webhook\" to save our settings.\n\n\n# I'm not Explosion AI, but I want to use this too!\n\nIt's all under the MIT license, so feel free! It would be great to\nsomehow convert this into a generic reusable piece of infrastructure,\nthough it's not entirely clear how given how Rube-Goldergian the whole\nthing is – you can't just slap it up on PyPI. (Maybe a cookiecutter\ntemplate that generates a repo like this?) In the mean time, you can\nprobably get something working by:\n\n* forking the repo\n\n* enabling Travis/Appveyor on your new repo\n\n* manually disabling PR builds for Appveyor (see above)\n\n* adding your encrypted Github token to `.travis.yml` and\n  `appveyor.yml` (see above)\n\n* editing the magic variables at the top of `mb.py` to point to your\n  repo\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjsmith%2Fcymem-wheels","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnjsmith%2Fcymem-wheels","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjsmith%2Fcymem-wheels/lists"}