{"id":13453592,"url":"https://github.com/mbachry/exxo","last_synced_at":"2025-04-05T22:06:17.016Z","repository":{"id":69393364,"uuid":"46455577","full_name":"mbachry/exxo","owner":"mbachry","description":"Build portable Python binaries","archived":false,"fork":false,"pushed_at":"2023-01-26T18:13:26.000Z","size":128,"stargazers_count":459,"open_issues_count":6,"forks_count":9,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-03-29T20:09:57.649Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mbachry.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst"}},"created_at":"2015-11-19T00:02:42.000Z","updated_at":"2025-02-12T14:17:48.000Z","dependencies_parsed_at":"2023-07-09T22:31:05.746Z","dependency_job_id":null,"html_url":"https://github.com/mbachry/exxo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbachry%2Fexxo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbachry%2Fexxo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbachry%2Fexxo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbachry%2Fexxo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mbachry","download_url":"https://codeload.github.com/mbachry/exxo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247406087,"owners_count":20933803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T08:00:43.953Z","updated_at":"2025-04-05T22:06:16.992Z","avatar_url":"https://github.com/mbachry.png","language":"Python","funding_links":[],"categories":["Python","Freezers"],"sub_categories":[],"readme":"====\nexxo\n====\n\n.. image:: https://travis-ci.org/mbachry/exxo.svg?branch=master\n    :alt: Build status\n    :target: https://travis-ci.org/mbachry/exxo\n\n**Only Linux x86_64 supported for now**\n\nBuild your Python package into a portable one-file binary and deploy\nit just by copying it to target machine. The binary links to libc only\nand doesn't require Python to be installed on the machine. Exxo was\ncreated with DevOps professionals in mind, but the target audience may\nbecome larger as project matures.\n\nExxo uses the excellent `PyRun`_ project and `zipapp`_ library. The\ntarget binary is simply a pyrun binary with zipped application\nconcatenated at the end. This simple design was chosen in hope it\nproves to be the most portable one (PyRun already works on most major\nplatforms). There's also an assumption that over time zipapps become\nmore popular and more essential packages will become zip safe.\n\nIn order for exxo to be practical, PyRun and CPython were patched in\nthe following ways:\n\n* zipimport supports loading C extensions (otherwise too many pip\n  libraries would be useless)\n\n* zipimport is capable of loading code from ``__pycache__`` if Python\n  3 is used\n\n* original PyRun comes with few C modules distributed separately (with\n  notable examples for multiprocessing or ctypes); exxo's PyRun on the\n  other hand is a truly one file binary with all modules included (at\n  the cost of portability loss, for now)\n\n* all libraries standard Python extensions depend on (like sqlite3 or\n  ncurses) are statically compiled in (again, it makes PyRun Linux\n  only)\n\n* few I/O functions in CPython are patched to make most zip unsafe\n  packages work out of the box (read `Zip safety hacks`_ section\n  below)\n\n.. _PyRun: https://www.egenix.com/products/python/PyRun/\n.. _zipapp: https://docs.python.org/3/library/zipapp.html\n\nDownload\n--------\n\nExxo is self-hosting. You can download it `here`_. The archive\ncontains just one file: exxo binary you should put somewhere in your\n``PATH``.\n\n.. _here: https://bintray.com/artifact/download/mbachry/exxo/exxo-0.0.7.tar.xz\n\nQuick start\n-----------\n\nYour package needs to have a working ``setup.py`` script. We'll use a\nsample project from ``example`` directory in `exxo git\nrepository`_. It's a simple Flask application that prints connecting\nIP address. It demonstrates using a C extension (gevent), handling\ndata files (Flask templates and static assets) and embedding gunicorn\n- all in one portable binary.\n\nCreate a new virtualenv and activate it with::\n\n    exxo venv /tmp/myenv\n    . /tmp/myenv/bin/activate\n\nThe default Python version is 3.5. Use ``exxo venv -p 2.7`` for Python\n2.7.\n\nYou can use the virtualenv in regular way. To build the target binary::\n\n    cd example\n    exxo build\n\nYou'll find the binary under ``dist`` directory. Go on and copy it to\nsome server and see if it works.\n\nIf you have upx installed (``apt-get install upx`` or ``dnf install\nupx``) you can use ``-c`` flag (``exxo build -c``) to compress PyRun\nbinary and save some space.\n\n.. _exxo git repository: https://github.com/mbachry/exxo/\n\nShowcase\n--------\n\nI plan to build few popular Python projects as a showcase of exxo\ncapabilities. For now there's only one.\n\nSentry\n~~~~~~\n\n`Link to download`_\n\nRequired shared libs: ``apt-get install -y libpq5 libxml2 libxslt1.1``\n\n`Sentry`_ is a great error capturing and reporting tool - a life\nsaver, if your code lives in a highly distributed environment. It's\nalso a huge Django app, which makes it a good test bed for exxo.\n\n.. _Link to download: https://bintray.com/artifact/download/mbachry/exxo/sentry-8.0.1.tar.xz\n.. _Sentry: https://getsentry.com/welcome/\n\nDifferences with similar projects\n---------------------------------\n\nThere's already a significant competition for exxo, including\nprominent projects like `pex`_ or `PyInstaller`_. Here are few things\nI'd like to see exxo doing differently:\n\n* single binary\n\n* the binary should be almost 100% standalone (i.e. Python doesn't\n  have to be preinstalled)\n\n* good startup performance so that exxo can be used for small, short\n  lived apps (this means not unpacking everything to temporary\n  locations)\n\n* user-friendliness: should stick to familiar solutions like virtualenv\n  or setuptools\n\n* should support at least most popular packages out of the box without\n  manual tweaks at user side\n\n* investment in zipapps - even if they need special attention today,\n  they are the cleanest and most modern way of bundling Python apps\n\n.. _pex: https://pex.readthedocs.org/en/stable/\n.. _PyInstaller: http://www.pyinstaller.org/\n\nCaveats\n-------\n\nAlthough exxo binary itself is statically linked, included C\nextensions (if any) are not. All required shared libraries must be\ninstalled on the target machine. For example, if you use ``lxml``, you\nmust install ``libxml2``. This shortcoming may be fixed in the future.\n\nAlso, exxo still links dynamically glibc for practical reasons\n(nsswitch support, etc.). Although glibc uses ELF symbol versioning,\nyou shouldn't build your project on a machine with much newer version\nof glibc than installed on destination server. Exxo release itself is\nbuilt on Ubuntu 10.04 (with openssl 1.0+) to make sure it runs on\nevery distro, including Centos 6.\n\nBecause your application is run as zipapp, it should be zip safe. This\napplies to all dependencies too, although exxo is armed in few hacks\nto make many third-party packages run out of the box (see `Zip safety\nhacks`_ section below). The main violation against zip safety is using\nfilesystem API to read data files from inside your package. Don't do\nthis::\n\n    open(os.path.join(os.path.dirname(__file__), 'index.html'))\n\nInstead use `pkgutil`_ module and its ``get_data`` function::\n\n    pkgutil.get_data('mypackage', 'index.html')\n\nor `pkg_resources`_ module from ``setuptools`` for more sophisticated\nAPI.\n\nNote that your ``setup.py`` must have one (and exactly one)\n``console_scripts`` entry point defined for ``exxo build`` to work\ncorrectly.\n\nAlthough exxo tries hard to load everything directly from an\nexecutable, some resources still have to be unzipped to a temporary\ndirectory due to OS limitations. This applies mostly to bundled\nbinaries (C extensions, shared libraries for ``ctypes``, ELF\nbinaries), as it's nearly impossible to ``dlopen`` directly from a\nzip. One serious limitation coming from this behaviour is that an exxo\nbinary won't work, if your ``/tmp`` directory happens to be mounted with\n``-o noexec``.\n\n.. _pkgutil: https://docs.python.org/3/library/pkgutil.html\n.. _pkg_resources: https://pythonhosted.org/setuptools/pkg_resources.html\n.. _example/myip/myip.py: https://github.com/mbachry/exxo/blob/master/example/myip/myip.py\n\nZip safety hacks\n----------------\n\nExxo implements few patches over CPython to improve zip compatibility\nout of the box.\n\nMany popular Python packages are zip unsafe (including\nDjango). Luckily most of zip unsafe code follows the same pattern of\nloading bundled resources mentioned in previous section::\n\n    open(os.path.join(os.path.dirname(__file__), 'templates', 'index.html'))\n\nIf loaded from an unpatched exxo binary, it will fail with an\nexception like::\n\n    NotADirectoryError: [Errno 20] Not a directory: '/usr/bin/djangoapp/app/templates/index.html'\n\nThe erroneous path is clearly built from two parts: a path to exxo\nbinary (``/usr/bin/djangoapp``) and a path inside zip\n(``app/templates/index.html``). Exxo patches several standard I/O\nfunctions inside CPython to detect the above pattern and return an\nobject from zip instead of an error. This simple hack vastly improves\nzip compatibility - to the point it's possible to build Django apps\nout of the box.\n\nHere's a list of functions and modules patched so far:\n\n* ``open``\n\n* ``os.stat``\n\n* ``os.listdir``\n\n* ``ctypes`` (requires unpacking to temporary location)\n\n* ``subprocess`` (requires unpacking to temporary location)\n\nBuilding exxo from sources\n--------------------------\n\nBuilding was tested only on Ubuntu. Python 3.5 is also required.\n\nInstall build dependencies with::\n\n    apt-get install -y gcc make patch wget tar gzip bzip2 xz-utils blt-dev libbluetooth-dev libbz2-dev libc-dev-bin libc6-dev libdb4.8-dev libexpat1-dev libffi-dev libfontconfig1-dev libfreetype6-dev libncurses5-dev libncursesw5-dev libpthread-stubs0-dev libreadline-dev libreadline6-dev libsqlite3-dev libssl-dev libstdc++6-4.4-dev libx11-dev libxau-dev libxcb1-dev libxdmcp-dev libxext-dev libxft-dev libxrender-dev libxss-dev linux-libc-dev tcl8.5-dev tk8.5-dev x11proto-core-dev x11proto-input-dev x11proto-kb-dev x11proto-render-dev x11proto-scrnsaver-dev x11proto-xext-dev xtrans-dev zlib1g-dev liblzma-dev upx\n\nBuild PyRun binaries with::\n\n    python3 -m exxo.bootstrap all\n\nFrom this point exxo is usable as ``python3 -m exxo.exxo``. Type\n``make build`` to build exxo binary under ``dist`` directory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbachry%2Fexxo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmbachry%2Fexxo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbachry%2Fexxo/lists"}