{"id":13460346,"url":"https://github.com/mityax/rustimport","last_synced_at":"2025-05-15T12:03:02.756Z","repository":{"id":42566712,"uuid":"472256235","full_name":"mityax/rustimport","owner":"mityax","description":"Import Rust source files directly from Python!","archived":false,"fork":false,"pushed_at":"2025-05-09T13:53:36.000Z","size":132,"stargazers_count":253,"open_issues_count":5,"forks_count":14,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-09T14:51:38.658Z","etag":null,"topics":["pyo3","python","python-rust","python3","rust","rust-bindings","rust-python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mityax.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-21T08:54:26.000Z","updated_at":"2025-05-09T13:53:39.000Z","dependencies_parsed_at":"2024-12-21T05:04:00.005Z","dependency_job_id":"7e115004-c3a3-4f7f-ba97-1b662cb86937","html_url":"https://github.com/mityax/rustimport","commit_stats":{"total_commits":75,"total_committers":10,"mean_commits":7.5,"dds":"0.33333333333333337","last_synced_commit":"fd9c30b32819089f44afbe9f9b0a172274b6d959"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mityax%2Frustimport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mityax%2Frustimport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mityax%2Frustimport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mityax%2Frustimport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mityax","download_url":"https://codeload.github.com/mityax/rustimport/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pyo3","python","python-rust","python3","rust","rust-bindings","rust-python"],"created_at":"2024-07-31T10:00:40.110Z","updated_at":"2025-05-15T12:03:02.719Z","avatar_url":"https://github.com/mityax.png","language":"Python","readme":"# rustimport - Import Rust directly from Python! \n\n\u003cp align=center\u003e\n    \u003ca target=\"_blank\" href=\"https://www.python.org/downloads/\" title=\"Python version\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-%3E=_3.8-green.svg\" /\u003e\u003c/a\u003e\n    \u003ca target=\"_blank\" href=\"https://pypi.org/project/rustimport/\" title=\"PyPI version\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/rustimport?logo=pypi\" /\u003e\u003c/a\u003e\n    \u003ca target=\"_blank\" href=\"https://pypi.org/project/rustimport/\" title=\"PyPI version\"\u003e\u003cimg src=\"https://img.shields.io/pypi/dm/rustimport\" /\u003e\u003c/a\u003e\n    \u003ca target=\"_blank\" href=\"LICENSE\" title=\"License: MIT\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-blue.svg\" /\u003e\u003c/a\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nArguably the simplest way to use rust from python – no manual compilation step, setup work or binding code required. rustimport provides a simple CLI, makes sure the compiled extension stays up to date with your source code and supports IPython/Jupyter notebooks.\n\n\u003ci\u003erustimport was heavily inspired by and is partly based upon [cppimport](https://github.com/tbenthompson/cppimport). Check it out if you're interested in similar functionality for C and C++!\u003c/i\u003e\n\n## Installation\n\nInstall with `pip install rustimport`.\n\n## A quick example\n\nSave the Rust code below as `somecode.rs`.\n```rust\n// rustimport:pyo3\n\nuse pyo3::prelude::*;\n\n#[pyfunction]\nfn square(x: i32) -\u003e i32 {\n    x * x\n}\n```\n\nThen open a Python interpreter and import the Rust extension:\n\n```python\n\u003e\u003e\u003e import rustimport.import_hook\n\u003e\u003e\u003e import somecode  # This will pause for a moment to compile the module\n\u003e\u003e\u003e somecode.square(9)\n81\n```\n\nHurray, you've called some Rust code from Python using a combination of `rustimport` and [`pyo3`](https://github.com/PyO3/pyo3)\n\nThis workflow enables you to edit both Rust files and Python and recompilation happens automatically and transparently! It's also handy for quickly whipping together an optimized version of a slow Python function.\n\n## Using the CLI\n\nTo easily create a new single-file extension (like above), or a complete crate, use the provided tool:\n```bash\n$ python3 -m rustimport new my_single_file_extension.rs\n# or create a full rust crate:\n$ python3 -m rustimport new my_crate\n```\n\nAnd import it from Python:\n```python\n\u003e\u003e\u003e import rustimport.import_hook\n\u003e\u003e\u003e import my_single_file_extension, my_crate\n\u003e\u003e\u003e my_single_file_extension.say_hello()\nHello from my_single_file_extension, implemented in Rust!\n\u003e\u003e\u003e my_crate.say_hello()\nHello from my_crate, implemented in Rust!\n```\n\nSmooth!\n\nThe CLI can also be used to pre-compile your rust code for production so you don't have to have the rust toolchain on your production environments – see [below](#usage-in-production) for more. \n\n## An explanation \n\nOkay, now that I've hopefully convinced you on how exciting this is, let's get into the details of how to do this yourself. First, the comment at top is essential to opt in to rustimport. Don't forget this! (See below for an explanation of why this is necessary.)\n```rust\n// rustimport:pyo3\n```\n\nThe bulk of the file is a generic, simple [pyo3](https://github.com/PyO3/pyo3) extension. We use the `pyo3` crate, then define a simple function that squares `x`, and rustimport takes care of exporting that function as part of a Python extension called `somecode`.\n\n## Templating \u0026 Pre-Processing\n\nrustimport offers several layers of customization. This is archieved through a simple pre-processor and templates (well, the only existing template at the moment is `pyo3` - pull requests welcome :D).\n\n### What rustimport did for you in the background\nThe first example in this Readme is the simplest possible form of using rustimport. You just tell rustimport to use the `pyo3` template by writing `rustimport:pyo3` in the first line, and define a function annotated with `pyo3`'s `#[pyfunction]` macro. In the background, rustimport handled a lot of stuff for you:\n\n1. It set up a minimalistic folder structure for a rust crate with your source code in a temporary location.\n2. It generated a Cargo.toml file with the basic configuration for pyo3 and your extension:\n```toml\n[package]\nname = \"somecode\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[lib]\nname = \"somecode\"\ncrate-type = [ \"cdylib\",]\n\n[dependencies.pyo3]\nversion = \"0.16.2\"\nfeatures = [ \"extension-module\",]\n```\n3. It generated a code block exporting your method and appended it to the end of your file:\n```rust\n#[pymodule]\nfn somecode(_py: Python, m: \u0026Bound\u003c'_, PyModule\u003e) -\u003e PyResult\u003c()\u003e {\n  m.add_function(wrap_pyfunction!(square, m)?)?;\n  Ok(())\n}\n```\n\n### Customizing an extension\nYou can do all the above yourself. rustimport will detect that and only fill in the missing parts to make your extension work.\n\n#### 1. Extending `Cargo.toml`\nFor example, to add additional contents to the generated `Cargo.toml` file, use the special `//:` comment syntax at the top of your `.rs` file:\n```rust\n// rustimport:pyo3\n\n// Set the library's version and add a dependency:\n//: [package]\n//: version = \"1.2.3\"\n//:\n//: [dependencies]\n//: rand = \"0.8.5\"\n\nuse rand::Rng;\n\n#[pyfunction]\nfn myfunc() {\n    println!(\"{}\", rand::thread_rng().gen_range(0..10))\n}\n```\n\n#### 2.Tracking additional source files\nTo track additional files for changes, use the special `//d:` comment syntax:\n```rust\n//d: ../other-folder/somefile.rs\n//d: ../*.rs\n//d: ./my-directory/**/*.json\n\n// --snip--\n```\nrustimport will now track files matching these patterns too and re-compiles your extension if any of them changes.\n\n#### 3. Full customization for more control\nIf you write a more complex extension, it's preferrable to just create a normal Rust crate:\n```bash\n$ python3 -m rustimport new my_crate\n$ tree my_crate\nmy_crate\n├── Cargo.toml\n├── .rustimport\n└── src\n    └── lib.rs\n```\n\nThe crate contains all necessary configuration to be directly be imported by rustimport and also some additional explanations on how to configure manually.\n\n## Using environment variables\nrustimport exposed most of it's settings via environment variables. This can be pretty handy in development but also in production environments.\n\nFor example, to force recompilation, just run your script like this:\n```commandline\nRUSTIMPORT_FORCE_REBUILD=true python my_script.py\n```\n\nOr to instruct the compiler to optimize binaries (for example to examine the real performance boost rust gives you) run it like this:\n```commandline\nRUSTIMPORT_RELEASE_BINARIES=true python my_script.py\n```\n\nTake a look at [settings.py](./rustimport/settings.py) for all available environment variables.\n\n## Using in Jupyter\n\n`rustimport` supports compiling code in a Jupyter notebook. To enable this feature load the `rustimport`\nextension from within the Jupyter notebook:\n\n```python\n%load_ext rustimport\n```\n\nThen, prefix a cell with the `%%rustimport` marker to compile it:\n\n```rust\n%%rustimport\nuse pyo3::prelude::*;\n\n#[pyfunction]\nfn square(x: i32) -\u003e i32 {\n    x * x\n}\n```\n\n`%%rustimport` supports release mode and force compilation with flags:\n\n```rust\n%%rustimport --release --force\n...\n```\n\n## Usage in production\n### 1. Building release binaries\nIn production deployments you usually don't want to include the Rust toolchain, all the sources and compile at runtime. Therefore, a simple cli utility for pre-compiling all source files is provided. This utility may, for example, be used in CI/CD pipelines. \n\nUsage is as simple as\n\n```commandline\npython -m rustimport build --release\n```\n\nThis will build a release-optimized binary for all `*.rs` files and Rust crates in the current directory (and it's subdirectories) if they are eligible to be imported (i.e. contain the `// rustimport` comment in the first line or a `.rustimport` file in case of a crate).\n\nAlternatively, you may specifiy one or more root directories or source files to be built:\n\n```commandline\npython -m rustimport build --release ./my/root/folder/ ./my/single/file.rs ./my/crate/\n```\n_Note: When specifying a path to a file, the header check (`// rustimport`) is skipped for that file._\n\n### 2. Toggling release mode on\nTo further improve startup performance for production builds, you can opt-in to skip the checksum and compiled binary existence checks during importing by either setting the environment variable `RUSTIMPORT_RELEASE_MODE` to `true` or setting the configuration from within Python:\n```python\nrustimport.settings.release_mode = True\n```\nThis essentially just disables the import hook and uses the standard python utilities to import the pre-compiled binary.\n\n**Warning:** Make sure to have all binaries pre-compiled with when in release mode, as importing any missing ones will cause exceptions.\n\nIn case you would, for whatever reason, like the binaries to be checked and built in production too, set `rustimport.settings.compile_release_binaries` to `True` to use release-optimized binaries.\n\n## Frequently asked questions\n\n### What's actually going on?\n\nSometimes Python just isn't fast enough. Or you have existing code in a Rust crate. So, you write a Python *extension module*, a library of compiled code. I recommend [pyo3](https://github.com/PyO3/pyo3) for Rust to Python bindings.\n\nI discovered that my productivity is slower when my development process goes from *Edit -\u003e Test* in just Python to *Edit -\u003e Compile -\u003e Test* in Python plus Rust. So, `rustimport` combines the process of compiling and importing an extension in Python so that you can just run `import foobar` and not have to worry about multiple steps. Internally, `rustimport` looks for a file `foobar.rs` or a Rust crate (discovered through `foobar/Cargo.toml`). Assuming one is found, the comments at it's beginning are parsed for either a template (`rustimport:pyo3`) or a cargo manifest, then it's compiled and loaded as an extension module.\n\n### Does rustimport recompile every time a module is imported? \nNo! Compilation should only happen the first time the module is imported. The Rust source is compared with a checksum on each import to determine if any relevant file has changed. Additional dependencies can be tracked by adding to the header comments:\n\n```rust\n//d: ../myothercrate/**/*.rs\n//d: ../myothercrate/Cargo.toml\n```\n\nBy default, rustimport tracks all `*.rs` files as well as `Cargo.toml` and `Cargo.lock` for crates and no additional dependencies for single-file Rust extensions.\n\n### rustimport isn't doing what I want, can I get more verbose output?\n`rustimport` uses the standard Python logging tools. Thus, you can enable logging like this:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.DEBUG)  # or logging.INFO for a bit less verbosity\n# ... do some rustimport stuff here\n```\n\n### It's fast, but can it get even faster?\nTo create release-optimized binaries, set\n\n```python\nrustimport.settings.compile_release_binaries = True\n```\nOr set the environment variable `RUSTIMPORT_RELEASE_BINARIES` to `true`\n\nCompilation might be a little bit slower now due to rust's optimization mechanisms, but at runtime the extension is significantly faster in most cases.\n\n### How can I force a rebuild even when the checksum matches?\n\nSet:\n```python\nrustimport.settings.force_rebuild = True\n```\nOr set the environment variable `RUSTIMPORT_FORCE_REBUILD` to `true`\n\nAnd if this is a common occurrence, I would love to hear your use case and why the normal dependency tracking is insufficient!\n\n### Can I use something else than `pyo3`?\nSure! Though I recommend using `pyo3` due to it's simplicity, you're completely free to use any other library, for example [`rust-cpython`](https://github.com/dgrunwald/rust-cpython).\n\nThere is an example using `rust-cpython` in [examples/cpython_doublecount.rs](./examples/cpython_doublecount.rs)\n\n### How can I make compilation faster? \n\nCompilation happens incrementally by default. That is, the first compilation might take a bit, but subsequent ones are usually much faster.\n\nrustimport uses a temporary directory for caching, which is deleted after a reboot on most systems. Therefore it might be beneficial for you to set a custom cache directory to have a more permanent cache:\n\n```python\nrustimport.settings.cache_dir = os.path.realpath(\"./.rust-cache\")\n```\nOr - you guessed it - use the `RUSTIMPORT_CACHE_DIR` environment variable.\n\nIf this directory doesn't exist, it will be created automatically by rustimport.\n\n### Apple, huh?\nYes, macOS is supported. No additional config should be necessary for pyo3 as the [required linker args](https://pyo3.rs/v0.17.1/building_and_distribution.html#macos) are set automatically by rustimport.\n\n### Why does the import hook need \"rustimport\" on the first line of the .rs file?\nModifying the Python import system is a global modification and thus affects all imports from any other package. As a result, when `cppimport` was first implemented, other packages (e.g. `scipy`) suddenly started breaking because import statements internal to those packages were importing C or C++ files instead of the modules they were intended to import. To avoid this failure mode, the import hook uses an \"opt in\" system where C and C++ files can specify they are meant to be used with cppimport by having a comment on the first line that includes the text \"cppimport\". \n\nrustimport has adopted from this and follows the same pattern. Since rustimport also supports importing whole crates, an additional mechanism was necessary to make that work in the same fashion: You can either create a `.rustimport` file in the crate's root folder (next to `Cargo.toml`) or, alternatively, add a `# rustimport` comment to `Cargo.toml`s first line.\n\nAs an alternative to the import hook, you can use `imp` or `imp_from_path`. The `rustimport.imp` and `rustimport.imp_from_path` performs exactly the same operation as the import hook but in a slightly more explicit way:\n```python\nfoobar = rustimport.imp(\"foobar\")\nfoobar = rustimport.imp_from_path(\"./some/path/foobar.rs\")\nmycrate = rustimport.imp_from_path(\"./mycrate/\")\n```\nBy default, these explicit function do not require the \"rustimport\" keyword on the first line of the .rs source file or the according marker in the crate. \n\n## Contributing and architecture\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for details on the internals of `rustimport` and how to get involved in development.\n\n## License\nrustimport uses the MIT License.\n","funding_links":[],"categories":["语言资源库","Build Tools \u0026 Frameworks"],"sub_categories":["rust"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmityax%2Frustimport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmityax%2Frustimport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmityax%2Frustimport/lists"}