{"id":37079944,"url":"https://github.com/apparebit/tsutsumu","last_synced_at":"2026-01-14T09:40:16.174Z","repository":{"id":179527355,"uuid":"663648575","full_name":"apparebit/tsutsumu","owner":"apparebit","description":"Simple, flexible module bundling for Python","archived":true,"fork":false,"pushed_at":"2023-07-20T18:30:58.000Z","size":299,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"boss","last_synced_at":"2025-09-09T09:29:59.676Z","etag":null,"topics":["bundler","importer","importlib","loader","meta-path-finder","module-bundler","modules","python","software-distribution"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apparebit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-07T19:39:32.000Z","updated_at":"2024-10-14T14:25:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"19a52ab7-bb98-433c-bbab-f856425d3d40","html_url":"https://github.com/apparebit/tsutsumu","commit_stats":null,"previous_names":["apparebit/tsutsumu"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/apparebit/tsutsumu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apparebit%2Ftsutsumu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apparebit%2Ftsutsumu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apparebit%2Ftsutsumu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apparebit%2Ftsutsumu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apparebit","download_url":"https://codeload.github.com/apparebit/tsutsumu/tar.gz/refs/heads/boss","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apparebit%2Ftsutsumu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28416120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bundler","importer","importlib","loader","meta-path-finder","module-bundler","modules","python","software-distribution"],"created_at":"2026-01-14T09:40:15.502Z","updated_at":"2026-01-14T09:40:16.151Z","avatar_url":"https://github.com/apparebit.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tsutsumu: A Python Module Bundler and Runtime\n\n\u003e つつむ (tsutsumu), Japanese for bundle\n\nTsutsumu makes it **easy to create a module bundle**, i.e., a single file that,\nlike a file archive, contains many more modules and supporting resources and,\nlike a shell script, imports and executes such bundled modules. That way,\nTsutsumu **enables self-contained scripts** that run anywhere a suitable Python\ninterpreter is available—*without* creating a virtual environment or installing\npackages first. Tsutsumu comes with **its own, simple, textual bundle format**\nfor when trust is lacking and recipients prefer to inspect code before running.\nTsutsumu can also **target Python's builtin\n[`zipapp`](https://docs.python.org/3/library/zipapp.html) format**, which has\nthe benefits of being more compact and hence more suitable for larger bundles.\nIndependent of format, Tsutsumu **automatically determines the packages—and also\nthe package extras—to include in a bundle**, something `zipapp` doesn't know how\nto do. (zipapp support will ship with release 0.2. So does automatic dependency\nresolution.)\n\nHaving said that, Tsutsumu isn't the only option for more easily distributing\nPython code and it may very well not be the right option for your use case. In\naddition to the already mentioned `zipapp` module in the standard library, there\nalso are, for example, [PEX](https://github.com/pantsbuild/pex) and\n[PyInstaller](https://pyinstaller.org/en/stable/), which combine bundling with\nvirtual environments (PEX) and further include the Python runtime as well\n(PyInstaller). That makes them more sophisticated but also significantly more\nresource-intensive. Tsutsumu's simplicity makes it best suited to tools and\napplications of modest size that need to run in untrusted or resource-limited\nenvironments.\n\nThe rest of this document covers Tsutsumu thusly:\n\n 1. [Just Download and\n    Run!](https://github.com/apparebit/tsutsumu#1-just-download-and-run)\n 2. [Make a Bundle](https://github.com/apparebit/tsutsumu#2-make-a-bundle)\n 3. [The Workings of a\n    Bundle](https://github.com/apparebit/tsutsumu#3-the-workings-of-a-bundle)\n     1. [Layout of Bundled\n        Files](https://github.com/apparebit/tsutsumu#31-layout-of-bundled-files)\n     2. [A Bundle's Manifest: Offsets and\n        Lengths](https://github.com/apparebit/tsutsumu#32-a-bundles-manifest-offsets-and-lengths)\n     3. [On-Disk vs\n        In-Memory](https://github.com/apparebit/tsutsumu#33-on-disk-vs-in-memory)\n     4. [Meta-Circular\n        Bundling](https://github.com/apparebit/tsutsumu#34-meta-circular-bundling)\n     5. [importlib.resources Considered\n        Harmful](https://github.com/apparebit/tsutsumu#35-importlibresources-considered-harmful)\n     6. [Add a Resource Manifest\n        Instead](https://github.com/apparebit/tsutsumu#36-add-a-resource-manifest-instead)\n 4. [Coming Soon](https://github.com/apparebit/tsutsumu#4-coming-soon)\n\n\n## 1. Just Download and Run!\n\nThere is nothing to install. There is no virtual environment to set up. Just\ndownload [this one Python\nscript](https://raw.githubusercontent.com/apparebit/tsutsumu/boss/bundles/bundler.py)\nand run it:\n\n```sh\n% curl -o tsutsumu.py \\\n    \"https://raw.githubusercontent.com/apparebit/tsutsumu/boss/bundles/bundler.py\"\n% python tsutsumu.py -h\nusage: tsutsumu [-h] [-b] [-m MODULE] [-o FILENAME] [-r] [-v]\n                PKGROOT [PKGROOT ...]\n\nCombine Python modules and related resources into a single,\n...\n```\n\nYup. I used Tsutsumu to bundle its own modules into `bundler.py`. As a result,\ngetting started with Tsutsumu is as easy as downloading a file and running it.\nBundled scripts can be this easy and convenient!\n\n\n### The Complicated Route Still Works\n\nBut just in case that you prefer to take the slow and familiar route, you can do\nthat, too. It just requires 3.5 times more command invocations and takes quite a\nbit longer. But sure, here you go:\n\n```sh\n% mkdir tsutsumu\n% cd tsutsumu\n% python -m venv .venv\n% source .venv/bin/activate\n(.venv) % pip install --upgrade pip\n(.venv) % pip install tsutsumu\n(.venv) % python -m tsutsumu -h\nusage: tsutsumu [-h] [-b] [-m MODULE] [-o FILENAME] [-r] [-v]\n                PKGROOT [PKGROOT ...]\n\nCombine Python modules and related resources into a single,\n...\n```\n\nSo how about bundling your Python modules?\n\n\n## 2. Make a Bundle\n\nThe only challenge in making a bundle is in selecting the right directories for\ninclusion. Right now, you need to list every package that should be included in\nthe bundle as a separate directory argument to Tsutsumu. Alas, for most Python\ntools and applications, that's just the list of regular dependencies. While\nmodule-level tree-shaking might still be desirable, automating package selection\nbased on a project's `pyproject.toml` is an obvious next step.\n\nWhen Tsutsumu traverses provided directories, it currently limits itself to a\nfew textual formats based on file extension. In particular, it includes plain\ntext, Markdown, ReStructured Text, HTML, CSS, JavaScript, and most importantly\nPython sources. Out of these, Tsutsumu only knows how to execute Python code.\nThe other files serve as resources. Adding support for Base85-encoded binary\nformats seems like another obvious next step.\n\n\n## 3. The Workings of a Bundle\n\nThis section is a hands-on exploration of Tsutsumu's inner workings. Its\nimplementation is split across the following modules:\n\n  * `tsutsumu` for Tsutsumu's `__version__` and nothing else;\n  * `tsutsumu.__main__` for the `main()` entry point and command line interface;\n  * `tsutsumu.debug` for validating the manifest and contents of bundle scripts;\n  * `tsutsumu.maker` for generating bundles with the `BundleMaker` class;\n  * `tsutsumu.bundle` for importing from bundles with the `Bundle` class.\n\nAs that breakdown should make clear, `tsutsumu.maker` and `tsutsumu.bundle`\nprovide the critical two classes that do all the heavy lifting. Hence I'll be\nfocusing on them in this section. To illustrate their workings, I rely on the\n`spam` package also contained in Tsutsumu's source repository. In addition to\nits own `__init__` package module, the package contains two Python modules,\n`__main__` and `bacon`, as well as a very stylish webpage, `ham.html`, that also\nincludes an image, `bacon.jpg`.\n\nAll subsequent code examples have been validated with Python's `doctest` tool.\nRunning the tool over this file is part of Tsutsumu's [test\nsuite](https://github.com/apparebit/tsutsumu/blob/boss/test.py).\n\nLet's get started making a bundle with the contents of the `spam` directory:\n\n```py\n\u003e\u003e\u003e import tsutsumu.maker\n\u003e\u003e\u003e maker = tsutsumu.maker.BundleMaker(['spam'])\n\u003e\u003e\u003e maker\n\u003ctsutsumu-maker spam\u003e\n\u003e\u003e\u003e\n```\n\nThe bundle maker ultimately needs to produce a Python script. To get there, the\nbundle maker processes data from byte to file granularity, which is quite the\nspread. At the same time, it's easy enough to format strings that are entire\nlines and, similarly, break down larger blobs into individual lines. Hence, most\nbundle maker methods treat the source line as the common unit of abstraction.\nHowever, since files are stored as byte string, not character strings, and byte\ncounts do matter, those source lines are `bytes`, *not* `str`, and include\nnewlines, *just* `\\n`.\n\nHaving said that, the bundle maker starts out by iterating over the contents of\ndirectories, yielding the files to be bundled. For each such file, it yields an\noperating-system-specific `Path`—suitable for reading the file's contents from\nthe local file system—as well as a relative `str` key with forward slashes—for\nidentifying the file in the bundle's manifest. Here are the bundle maker's keys\nfor `spam`:\n\n```py\n\u003e\u003e\u003e files = list(sorted(maker.list_files(), key=lambda f: f.key))\n\u003e\u003e\u003e for file in files:\n...     print(file.key)\n...\nspam/__init__.py\nspam/__main__.py\nspam/bacon.jpg\nspam/bacon.py\nspam/ham.html\n\u003e\u003e\u003e\n```\n\nThose are just the five files we expect:\n\n  * `spam/__init__.py` contains `spam`'s package module;\n  * `spam/__main__.py` is the package's main entry point;\n  * `spam/bacon.jpg` is a package resource;\n  * `spam/bacon.py` contains the `spam.bacon` submodule;\n  * `spam/ham.html` is a package resource, too.\n\n\n### 3.1 Layout of Bundled Files\n\nNow that we know which files to include in the bundle, we can turn to their\nlayout in bundle scripts. The current format tries to reconcile two\n contradictory requirements: First, the layout must be valid Python source code.\nThat pretty much limits us to string literals for file names and contents.\nFurthermore, since the collection of file names and contents obviously forms a\nmapping, we might as well use a `dict` literal for the file data.\n\nSecond, the code must not retain the bundled data. Otherwise, all bundled files\nare loaded into memory at startup and remain there for the duration of the\napplication's runtime. Ideally, the Python runtime doesn't even instantiate the\n`dict` literal and just scans for its end. To facilitate that, the bundle script\ndoes not assign the `dict` literal to a variable and, on top of that, includes\nit only inside an `if False:` branch.\n\n```py\n\u003e\u003e\u003e writeall = tsutsumu.maker.BundleMaker.writeall\n\u003e\u003e\u003e writeall(tsutsumu.maker._BUNDLE_START.splitlines(keepends=True))\nif False: {\n\u003e\u003e\u003e\n```\n\nI don't know whether CPython does optimize parsing along those lines. Though I\ndo know that Donald Knuth's TeX (which dates back to the late 1970s) does\noptimize just this case: Once TeX knows that a conditional branch is not taken,\nit simply scans upcoming tokens, taking only `\\if` (and variations thereof),\n`\\else`, and `\\fi` into account, until it has found the end of the branch, after\nwhich TeX resumes regular processing.\n\nLet's hope that Python is just as clever and fill in the file name, content\npairs for each bundled file. We start with `spam/__init__.py`:\n\n```py\n\u003e\u003e\u003e writeall(maker.emit_file(*files[0]))\n# ------------------------------------------------------------------------------\n\"spam/__init__.py\": b\"print('spam/__init__.py')\\n\",\n\u003e\u003e\u003e\n```\n\nAs shown, the file name or key is a `str` literal, whereas the file contents are\n`bytes`. We use bytes instead of characters for the latter because, at their\nmost basic, files are just that, bytestrings. We get characters only after\ndecoding, nowadays typically from UTF-8.\n\nBeware of bytestring literals in Python: They are limited to ASCII characters\nand require that all other code points be escaped. In other words, the majority\nof code points in bytestring literals must be escaped. That would make for a\nrather verbose encoding if values were more evenly distributed. However, in the\ncase of Tsutsumu, the bundled files are mostly text files, in particular Python\nsource code. That strongly biases bundled files towards ASCII and makes this an\nefficient and human-readable encoding.\n\nThe `spam` package's `__main__` module isn't so different from the `__init__`\nmodule, except that it takes up several lines and hence uses triple-quotes:\n\n```py\n\u003e\u003e\u003e writeall(maker.emit_file(*files[1]))\n# ------------------------------------------------------------------------------\n\"spam/__main__.py\":\nb\"\"\"print('spam/__main__.py')\nimport spam.bacon\n\u003cBLANKLINE\u003e\nprint('also:', __file__)\n\"\"\",\n\u003e\u003e\u003e\n```\n\nUnlike the other files in the bundle, the next file contains bitmap image and\nhence is binary. Its contents are represented by a bytestring literal too, but\nthe contents have been encoded in Base85. For readability, the literal adds\nnewlines every 76 characters. By design, Base85 uses only ASCII characters and\nhence there should be no escape sequences in the bytestring literals for binary\nfiles.\n\n```py\n\u003e\u003e\u003e writeall(maker.emit_file(*files[2]))   # doctest: +ELLIPSIS\n# ------------------------------------------------------------------------------\n\"spam/bacon.jpg\": b\"\"\"\ns4IA0!\"_al8O`[\\!\u003c\u003c,,!42_+s5\u003csN7\u003ciNY!!#_f!%IsK!!iQ.!\u003e5A7!!!!\"!!*'\"!?(qA!!!!\"!\n!!!k!?2\"B!!!!\"!!!!s!AOQU!!!!5!!!\"\u0026LM6_k!!!!\"!!!\":z!!!#+!!!!\"!!!#+!!!!\"6\"FnCA\nKXHVEb0H5Ebf_=6W5c@!!Akp!!\u003c3$!!*'#!!\u0026Yn!!E9%!!*'\"!;`\u003ej!!E9%!!*'\"!/U[U!!*\u0026d!'\n!egDffo=BQ%i41G1?]3'p22\"9\\])z3'p22\"=4$J!!!!1e/aM$NrZHgl$s)-m.`nrs1eUH#QT\\]q?\n...\n\u003e\u003e\u003e\n```\n\nNote that the complete encoded image is larger than what would fit into four\nmeasly lines of Base85, a bit more than 13,000 bytes larger.\n\nFor the fourth and fifth file, we are back to text again:\n\n```py\n\u003e\u003e\u003e writeall(maker.emit_file(*files[3]))\n# ------------------------------------------------------------------------------\n\"spam/bacon.py\": b\"print('spam/bacon.py')\\n\",\n\u003e\u003e\u003e writeall(maker.emit_file(*files[4]))\n# ------------------------------------------------------------------------------\n\"spam/ham.html\":\nb\"\"\"\u003c!DOCTYPE html\u003e\n\u003chtml lang=en\u003e\n\u003cmeta charset=utf-8\u003e\n\u003ctitle\u003eHam?\u003c/title\u003e\n\u003cstyle\u003e\n* {\n    margin: 0;\n    padding: 0;\n}\nhtml {\n    height: 100%;\n}\nbody {\n    min-height: 100%;\n    display: grid;\n    justify-content: center;\n    align-content: center;\n}\nimg {\n    height: calc(15vmin + 1vmax);\n    width: auto;\n    display: block;\n    position: relative;\n    left: -20%;\n    top: 40%;\n}\np {\n    font-family: system-ui, sans-serif;\n    font-size: calc(30vmin + 3vmax);\n    font-weight: bolder;\n}\n\u003c/style\u003e\n\u003cimg src=bacon.jpg\u003e\u003cp\u003eHam!\n\"\"\",\n\u003e\u003e\u003e\n```\n\nWith that, we can close the dictionary again:\n\n```py\n\u003e\u003e\u003e writeall(tsutsumu.maker._BUNDLE_STOP.splitlines(keepends=True))\n}\n\u003cBLANKLINE\u003e\n\u003e\u003e\u003e\n```\n\n\n### 3.2 A Bundle's Manifest: Offsets and Lengths\n\nA bundle's files are encoded as a `dict` literal by design—so that the script\nparses—but are *not* assigned to any variable by design as well—so that the\nscript does not retain access to the data, which would only increase memory\npressure. So if the script doesn't retain a reference to the data, how does\nit access the data when it's needed?\n\nI've already hinted at the solution: While turning file names and contents into\nyielded lines of the bundle script, the bundle maker tracks the byte offset and\nlength of each content literal. It helps that the bundle maker is implemented as\na class with several methods that are generators instead of as a bunch of\ngenerator functions. That way, accumulating state while yielding lines only\nrequires another method call, with the state stored by the bundle maker\ninstance. It also helps that the bundle maker emits bundle contents first, at\nthe beginning of the content script and that it relies on named string constants\nfor the boilerplate before, between, and right after the file contents\ndictionary.\n\nOnce the bundle maker is done with the file contents, it emits the manifest\nwith the offset and length for each file included in the bundle:\n\n```py\n\u003e\u003e\u003e writeall(maker.emit_manifest())\n__manifest__ = {\n    \"spam/__init__.py\": (\"t\", 305, 30),\n    \"spam/__main__.py\": (\"t\", 438, 77),\n    \"spam/bacon.jpg\": (\"b\", 620, 13_003),\n    \"spam/bacon.py\": (\"t\", 13_726, 27),\n    \"spam/ham.html\": (\"t\", 13_853, 534),\n}\n\u003e\u003e\u003e\n```\n\nThe data collected while yielding the file contents is one datum more granular\nthan offset and length. But the generator for the manifest consumes the output\nof another generator that accumulates the original three length values per file.\nAs you can see, Tsutsumu's not so secret sauce are generator functions and\nmethods!\n\nTsutsumu's source repository does not just include the `spam` package. But its\nso far tiny collection of [prebundled\nscripts](https://github.com/apparebit/tsutsumu/tree/boss/bundles) includes\n`can.py`, which already bundles the package. If you check `can.py`'s contents,\nyou should see the exact same files in the same order with the same offsets and\nlengths. That means that we can use the bundle to illustrate how the bundle\nruntime reads a file such as `spam/bacon.py`:\n\n```py\n\u003e\u003e\u003e with open('bundles/can.py', mode='rb') as file:\n...     _ = file.seek(13_726)\n...     data = file.read(27)\n...\n\u003e\u003e\u003e data\nb'b\"print(\\'spam/bacon.py\\')\\\\n\"'\n\u003e\u003e\u003e\n```\n\nAs you can see, the returned bytes aren't just the file contents, but also the\nleading and trailing characters necessary for turning the contents into a valid\nPython bytestring literal. We need those \"decorations\" in the script, so that\nPython knows to parse the bytestring. But why read those extra characters?\n\nPython bytestring literals represent 256 values per byte with ASCII characters\nonly. As a result, some code points necessarily require escape sequences. In\nfact, there are more code points that require escaping than printable ASCII\ncharacters. Nonetheless, this is a reasonable encoding for this domain because\nPython source code draws on ASCII mostly and remains human-readable under the\nencoding.\n\nStill, we can't escape escape sequences—as the above example illustrates. Notice\nthe trailing `\\\\n`? That's an escaped newline taking up two bytes in the\nbytestring. So why read a bytestring, as indicated by the leading `b'`,\ncontaining a bytestring literal, as indicated by the subsequent `b\"`, when we\nreally want proper bytes?\n\nHere's why:\n\n```py\n\u003e\u003e\u003e eval(data)\nb\"print('spam/bacon.py')\\n\"\n\u003e\u003e\u003e\n```\n\nIt only takes an `eval` to turn two consecutive bytestring prefixes and\nbackslash characters into one each, producing real `bytes`.\n\n\n### 3.3 On-Disk vs In-Memory\n\nAs presented so far, **bundled files are named by relative paths with forward\nslashes**. That makes sense for bundle scripts while they are inert and being\ndistributed. After all, the raison d'être for Tsutsumu's bundle scripts is to be\neasily copied to just about any computer and run right there. That wouldn't be\npractical if the names used in the bundle were tied to the originating file\nsystem or limited to some operating system only.\n\nHowever, the naming requirements change fundamentally the moment a bundle starts\nto execute on some computer. That instance should seamlessly integrate with the\nlocal Python runtime and operating system, while also tracking provenance, i.e.,\nwhether modules originate from the bundle or from the local machine. In other\nwords, a **running bundle uses absolute paths with the operating system's path\nsegment separator**. Sure enough, the constructor for `tsutsumu.bundle.Bundle`\nperforms the translation from relative, system-independent paths to absolute,\nsystem-specific paths by joining the absolute path to the bundle script with\neach key.\n\nLet's see how that plays out in practice on the example of the `can.py` bundle:\n\n```py\n\u003e\u003e\u003e import bundles.can\n\u003e\u003e\u003e manifest = bundles.can.__manifest__\n\u003e\u003e\u003e for key in manifest.keys():\n...     print(key)\n...\nspam/__init__.py\nspam/__main__.py\nspam/bacon.jpg\nspam/bacon.py\nspam/ham.html\n\u003e\u003e\u003e\n```\n\nClearly, the `__manifest__` is using relative paths.\n\nSince `bundles.can` isn't `__main__`, importing the bundle resulted in the\ndefinition of the `__manifest__` dictionary and the `Bundle` class but it did\nnot install a new `Bundle` instance in the module loading machinery. Before we\nmanually install the bundle, there's a bit of housekeeping to do. We need to cut\noff our ability to load modules from the regular file system. Otherwise, we\nmight inadvertently import the `spam` package from its sources and get mightily\nconfused. (Not that that ever happened to me...)\n\n```py\n\u003e\u003e\u003e bundles.can.Toolbox.restrict_sys_path()\n\u003e\u003e\u003e import spam\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\nModuleNotFoundError: No module named 'spam'\n\u003e\u003e\u003e\n```\n\nNo more readily available `spam`? Time to open `bundles.can`:\n\n```py\n\u003e\u003e\u003e from pathlib import Path\n\u003e\u003e\u003e can_path = Path('.').absolute() / 'bundles' / 'can.py'\n\u003e\u003e\u003e version = bundles.can.__version__\n\u003e\u003e\u003e can_content = bundles.can.Bundle.install(can_path, version, manifest)\n\u003e\u003e\u003e import spam\nspam/__init__.py\n\u003e\u003e\u003e\n```\n\nW00t, our supply of spam is secured. That's great. But how does it work? What\ndid `Bundle.install()` do exactly?\n\nWell, a `Bundle` is what `importlib`'s documentation calls an *importer*, a\nclass that is both a *meta path finder* and a *loader*. When Python tries to\nload a module that hasn't yet been loaded, it (basically) invokes\n`find_spec(name)` on each object in `sys.meta_path`, asking that meta path\nfinder whether it recognizes the module. If the meta path finder does, it\nreturns a description of the module. Most fields of that *spec* are just\ninformative, i.e., strings, but one field, surprisingly called `loader`, is an\nobject with methods for loading and executing the module's Python code. It just\nhappens that `Bundle` does not delegate to a separate class for loading but does\nall the work itself.\n\nIn short, `Bundle.install()` creates a new `Bundle()` and makes that bundle the\nfirst entry of `sys.meta_path`.\n\nOk. But what about the bundle using absolute paths?\n\n```py\n\u003e\u003e\u003e for key in can_content._manifest.keys():\n...     path = Path(key)\n...     assert path.is_absolute()\n...     print(str(path.relative_to(can_path)).replace('\\\\', '/'))\n...\nspam/__init__.py\nspam/__main__.py\nspam/bacon.jpg\nspam/bacon.py\nspam/ham.html\n\u003e\u003e\u003e\n```\n\nClearly, the installed `can_content` bundle is using absolute paths. Also, each\nkey now starts with the bundle script's path, which we recreated in `CAN`. While\nwe usually don't worry much about these paths when importing modules in Python,\nwe do need to use them when loading resources from a package:\n\n```py\n\u003e\u003e\u003e data = can_content.get_data(can_path / 'spam' / 'ham.html')\n\u003e\u003e\u003e data[-5:-1]\nb'Ham!'\n\u003e\u003e\u003e\n```\n\nHam! it is.\n\nMy apologies to vegetarians. You probably are tired of all this ham-fisted humor\nby now. So let's make sure we stop right here:\n\n```py\n\u003e\u003e\u003e can_content.uninstall()\n\u003e\u003e\u003e import spam.bacon\nTraceback (most recent call last):\n  ...\nModuleNotFoundError: No module named 'spam.bacon'\n\u003e\u003e\u003e\n```\n\nAlas, already imported modules are much harder to expunge. In fact, it may just\nbe impossible. In this case, however, it is feasible:\n\n```py\n\u003e\u003e\u003e import sys\n\u003e\u003e\u003e 'spam' in sys.modules\nTrue\n\u003e\u003e\u003e import spam\n\u003e\u003e\u003e del sys.modules['spam']\n\u003e\u003e\u003e 'spam' in sys.modules\nFalse\n\u003e\u003e\u003e import spam\nTraceback (most recent call last):\n  ...\nModuleNotFoundError: No module named 'spam'\n\u003e\u003e\u003e\n```\n\n\n### 3.4 Meta-Circular Bundling\n\nTsutsumu can bundle any application that is not too large and written purely in\nPython. That includes itself. Tsutsumu can bundle itself because it avoids the\nfile system when including its own `tsutsumu/bundle.py` in the bundle script.\nInstead, it uses the module loader's `get_data()` method, which is designed for\naccessing packaged resources and whose use I just demonstrated.\n\nOne drawback of Tsutsumu treating its own source code just like other Python\nfiles is the effective duplication of `tsutsumu/bundle.py`, once as part of the\nbundled files and once as part of the bundle script itself. While that may be\ndesirable, for example, when experimenting with a new version of Tsutsumu, it\nalso wastes almost 8 kb. To avoid that overhead, you can use the\n`-r`/`--repackage` command line option when bundling Tsutsumu. Under that\noption, Tsutsumu special cases the `tsutsumu` and `tsutsumu.bundle` modules and\nrecreates them during startup—with `tsutsumu`'s `bundle` attribute referencing\nthe `tsutsumu.bundle` module and `tsutsumu.bundle`'s `Bundle` attribute\nreferencing the corresponding class.\n\n\n## 3.5 importlib.resources Considered Harmful\n\nWhile Tsutsumu does support `Loader.get_data()`, it does *not* support the more\nrecent `Loader.get_resource_reader()` and probably never will. The API simply is\ntoo complex for what it does, i.e., providing yet another way of traversing a\nhierarchy of directory-like and file-like entities. Furthermore, the\ndocumentation's claims about the benefits of integration with Python's import\nmachinery seem farfetched at best.\n\nA look at the 8 (!) modules implementing `importlib.resources` in the standard\nlibrary bears this out: In addition to the documented `ResourceReader`,\n`Traversable`, and `TraversableReader` abstract classes, there are undocumented\n`FileReader`, `ZipReader`, and `NamespaceReader` implementations, the\n`SimpleReader` fallback implementation, and the `CompatibilityFiles` adapter.\nFurthermore, since `Traversable` is designed to have \"a subset of `pathlib.Path`\nmethods,\" the code in `importlib.resources` makes heavy use of the `Path`\nimplementations provided by `pathlib` and `zipfile`. Taken together, that's a\nlot of code for exposing a hierarchy of directory- and file-like entities.\nWorse, despite the documentation's claims to the contrary, none of this code\nleverages core `importlib` machinery—besides hanging off loaders and hence\ntouching on `ModuleType` and `ModuleSpec`. In fact, it doesn't even integrate\nwith the previous resource API, the much simpler `get_data()` method on loaders.\nIn summary, `importlib.resources` does not offer what it claims and is far too\ncomplex for what it offers. It should be scrapped!\n\n\n### 3.6 Add a Resource Manifest Instead\n\nWhen you compare the two ways of accessing resources, `Loader.get_data()` and\n`Loader.get_resource_reader()`, the latter obviously wins on traversing a\npackage's namespace. But that's a non-feature when it comes to resource access.\nWhen code needs a resource, it shouldn't need to search for the resource by\nsearching them all, it should be able to just access the resource, possibly\nthrough one level of indirection. In other words, if a package's resources may\nvary, the package should include a resource manifest at a well-known location,\nsay, `manifest.toml` relative to the package's path. Once the package includes a\nmanifest, `Loader.get_data()` more than suffices for retrieving resources.\n`Loader.get_resource_reader()` only adds useless complexity.\n\n\n## 4. Coming Soon\n\nI believe that Tsutsumu is ready for real-world use. However, since it hasn't\nseen wide usage, I'd hold off on mission-critical deployments for now.\nMeanwhile, Tsutsumu could use a few more features. I can think of three:\n\n  * [x] Automatically determine module dependencies\n  * [x] Support inclusion of binary files in bundles\n  * [-] Support the bundling of namespace packages\n\nThe first two features are mostly implemented and the third also has basic\nsupport. Alas none of them have been released. They are scheduled for v0.2.\nSince `zipapp` also lacks the automatic dependency discovery, Tsutsumu will\nlikely add support for that format, too.\n\nWhat else?\n\n---\n\nTsutsumu is © 2023 [Robert Grimm](https://apparebit.com) and has been released\nunder the Apache 2.0 license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapparebit%2Ftsutsumu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapparebit%2Ftsutsumu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapparebit%2Ftsutsumu/lists"}