{"id":13550166,"url":"https://github.com/apache/arrow-rs","last_synced_at":"2025-05-12T03:42:38.895Z","repository":{"id":36969193,"uuid":"358917250","full_name":"apache/arrow-rs","owner":"apache","description":"Official Rust implementation of Apache Arrow","archived":false,"fork":false,"pushed_at":"2025-05-01T00:55:47.000Z","size":44891,"stargazers_count":2902,"open_issues_count":437,"forks_count":915,"subscribers_count":50,"default_branch":"main","last_synced_at":"2025-05-01T10:07:16.905Z","etag":null,"topics":["arrow","parquet","rust"],"latest_commit_sha":null,"homepage":"https://arrow.apache.org/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG-old.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-04-17T15:40:05.000Z","updated_at":"2025-05-01T00:53:24.000Z","dependencies_parsed_at":"2023-09-27T15:36:08.568Z","dependency_job_id":"9e0ebe65-d0ad-45e5-89de-6053635098fe","html_url":"https://github.com/apache/arrow-rs","commit_stats":{"total_commits":5506,"total_committers":520,"mean_commits":"10.588461538461539","dds":0.8425354159099164,"last_synced_commit":"b25fe56b8fbcd5e30805967cb598c2df6266c067"},"previous_names":[],"tags_count":119,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Farrow-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Farrow-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Farrow-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Farrow-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/arrow-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251857023,"owners_count":21655121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arrow","parquet","rust"],"created_at":"2024-08-01T12:01:29.705Z","updated_at":"2025-05-01T10:07:24.452Z","avatar_url":"https://github.com/apache.png","language":"Rust","readme":"\u003c!---\n  Licensed to the Apache Software Foundation (ASF) under one\n  or more contributor license agreements.  See the NOTICE file\n  distributed with this work for additional information\n  regarding copyright ownership.  The ASF licenses this file\n  to you under the Apache License, Version 2.0 (the\n  \"License\"); you may not use this file except in compliance\n  with the License.  You may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing,\n  software distributed under the License is distributed on an\n  \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n  KIND, either express or implied.  See the License for the\n  specific language governing permissions and limitations\n  under the License.\n--\u003e\n\n# Native Rust implementation of Apache Arrow and Apache Parquet\n\nWelcome to the [Rust][rust] implementation of [Apache Arrow], the popular in-memory columnar format.\n\nThis repository contains the following crates:\n\n| Crate              | Description                                                                  | Latest API Docs                                  | README                            |\n| ------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------- |\n| [`arrow`]          | Core functionality (memory layout, arrays, low level computations)           | [docs.rs](https://docs.rs/arrow/latest)          | [(README)][arrow-readme]          |\n| [`arrow-flight`]   | Support for Arrow-Flight IPC protocol                                        | [docs.rs](https://docs.rs/arrow-flight/latest)   | [(README)][flight-readme]         |\n| [`parquet`]        | Support for Parquet columnar file format                                     | [docs.rs](https://docs.rs/parquet/latest)        | [(README)][parquet-readme]        |\n| [`parquet_derive`] | A crate for deriving RecordWriter/RecordReader for arbitrary, simple structs | [docs.rs](https://docs.rs/parquet-derive/latest) | [(README)][parquet-derive-readme] |\n\nThe current development version the API documentation in this repo can be found [here](https://arrow.apache.org/rust).\n\nNote: previously the [`object_store`] crate was also part of this repository,\nbut it has been moved to the [arrow-rs-object-store repository]\n\n[apache arrow]: https://arrow.apache.org/\n[`arrow`]: https://crates.io/crates/arrow\n[`parquet`]: https://crates.io/crates/parquet\n[`parquet_derive`]: https://crates.io/crates/parquet-derive\n[`arrow-flight`]: https://crates.io/crates/arrow-flight\n[arrow-rs-object-store repository]: https://github.com/apache/arrow-rs-object-store\n\n## Release Versioning and Schedule\n\nThe Arrow Rust project releases approximately monthly and follows [Semantic\nVersioning].\n\nDue to available maintainer and testing bandwidth, [`arrow`] crates ([`arrow`],\n[`arrow-flight`], etc.) are released on the same schedule with the same versions\nas the [`parquet`] and [`parquet-derive`] crates.\n\nThis crate releases every month. We release new major versions (with potentially\nbreaking API changes) at most once a quarter, and release incremental minor\nversions in the intervening months. See [ticket #5368] for more details.\n\nTo keep our maintenance burden down, we do regularly scheduled releases (major\nand minor) from the `main` branch. How we handle PRs with breaking API changes\nis described in the [contributing] guide.\n\n[contributing]: CONTRIBUTING.md#breaking-changes\n\nPlanned Release Schedule\n\n| Approximate Date | Version    | Notes                                   |\n| ---------------- | ---------- | --------------------------------------- |\n| Apr 2025         | [`55.0.0`] | Major, potentially breaking API changes |\n| May 2025         | [`55.1.0`] | Minor, NO breaking API changes          |\n| June 2025        | [`55.2.0`] | Minor, NO breaking API changes          |\n| July 2025        | [`56.0.0`] | Major, potentially breaking API changes |\n\n[`55.0.0`]: https://github.com/apache/arrow-rs/issues/7084\n[`55.1.0`]: https://github.com/apache/arrow-rs/issues/7393\n[`55.2.0`]: https://github.com/apache/arrow-rs/issues/7394\n[`56.0.0`]: https://github.com/apache/arrow-rs/issues/7395\n[ticket #5368]: https://github.com/apache/arrow-rs/issues/5368\n[semantic versioning]: https://semver.org/\n\n### Rust Version Compatibility Policy\n\narrow-rs, parquet and object_store are built and tested with stable Rust, and will keep a rolling MSRV (minimum supported Rust version) that can only be updated in major releases on a need by basis (e.g. project dependencies bump their MSRV or a particular Rust feature is useful for us etc.). The new MSRV if selected will be at least 6 months old. The minor releases are guaranteed to have the same MSRV.\n\nNote: If a Rust hotfix is released for the current MSRV, the MSRV will be updated to the specific minor version that includes all applicable hotfixes preceding other policies.\n\nE.g.\n\nin Apr 2025 we will release version 55.0.0 which might have a version bump. But the Rust version selected in this case will be at most version 1.81.\n\n### Guidelines for `panic` vs `Result`\n\nIn general, use panics for bad states that are unreachable, unrecoverable or harmful.\nFor those caused by invalid user input, however, we prefer to report that invalidity\ngracefully as an error result instead of panicking. In general, invalid input should result\nin an `Error` as soon as possible. It _is_ ok for code paths after validation to assume\nvalidation has already occurred and panic if not. See [ticket #6737] for more nuances.\n\n[ticket #6737]: https://github.com/apache/arrow-rs/issues/6737\n\n### Deprecation Guidelines\n\nMinor releases may deprecate, but not remove APIs. Deprecating APIs allows\ndownstream Rust programs to still compile, but generate compiler warnings. This\ngives downstream crates time to migrate prior to API removal.\n\nTo deprecate an API:\n\n- Mark the API as deprecated using `#[deprecated]` and specify the exact arrow-rs version in which it was deprecated\n- Concisely describe the preferred API to help the user transition\n\nThe deprecated version is the next version which will be released (please\nconsult the list above). To mark the API as deprecated, use the\n`#[deprecated(since = \"...\", note = \"...\")]` attribute.\n\nFoe example\n\n```rust\n#[deprecated(since = \"51.0.0\", note = \"Use `date_part` instead\")]\n```\n\nIn general, deprecated APIs will remain in the codebase for at least two major releases after\nthey were deprecated (typically between 6 - 9 months later). For example, an API\ndeprecated in `51.3.0` can be removed in `54.0.0` (or later). Deprecated APIs\nmay be removed earlier or later than these guidelines at the discretion of the\nmaintainers.\n\n## Related Projects\n\nThere are several related crates in different repositories\n\n| Crate               | Description                                                  | Documentation                      |\n| ------------------- | ------------------------------------------------------------ | ---------------------------------- |\n| [`object_store`]    | Object Storage (aws, azure, gcp, local, in-memory) interface | [(README)](object_store-readme)    |\n| [`datafusion`]      | In-memory query engine with SQL support                      | [(README)][datafusion-readme]      |\n| [`ballista`]        | Distributed query execution                                  | [(README)][ballista-readme]        |\n| [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO                     | [(README)][parquet_opendal-readme] |\n\n[`datafusion`]: https://crates.io/crates/datafusion\n[`ballista`]: https://crates.io/crates/ballista\n[`parquet_opendal`]: https://crates.io/crates/parquet_opendal\n[parquet_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/parquet/README.md\n[object_store-readme]: https://github.com/apache/arrow-rs-object-store/blob/main/README.md\n\nCollectively, these crates support a wider array of functionality for analytic computations in Rust.\n\nFor example, you can write SQL queries or a `DataFrame` (using the\n[`datafusion`] crate) to read a parquet file (using the [`parquet`] crate),\nevaluate it in-memory using Arrow's columnar format (using the [`arrow`] crate),\nand send to another process (using the [`arrow-flight`] crate).\n\nGenerally speaking, the [`arrow`] crate offers functionality for using Arrow\narrays, and [`datafusion`] offers most operations typically found in SQL,\nincluding `join`s and window functions.\n\nYou can find more details about each crate in their respective READMEs.\n\n## Arrow Rust Community\n\nThe `dev@arrow.apache.org` mailing list serves as the core communication channel for the Arrow community. Instructions for signing up and links to the archives can be found on the [Arrow Community](https://arrow.apache.org/community/) page. All major announcements and communications happen there.\n\nThe Rust Arrow community also uses the official [ASF Slack](https://s.apache.org/slack-invite) for informal discussions and coordination. This is\na great place to meet other contributors and get guidance on where to contribute. Join us in the `#arrow-rust` channel and feel free to ask for an invite via:\n\n1. the `dev@arrow.apache.org` mailing list\n2. the [GitHub Discussions][discussions]\n3. the [Discord channel](https://discord.gg/YAb2TdazKQ)\n\nThe Rust implementation uses [GitHub issues][issues] as the system of record for new features and bug fixes and\nthis plays a critical role in the release process.\n\nFor design discussions we generally use GitHub issues.\n\nThere is more information in the [contributing] guide.\n\n[rust]: https://www.rust-lang.org/\n[`object_store`]: https://crates.io/crates/object-store\n[arrow-readme]: arrow/README.md\n[contributing]: CONTRIBUTING.md\n[parquet-readme]: parquet/README.md\n[flight-readme]: arrow-flight/README.md\n[datafusion-readme]: https://github.com/apache/datafusion/blob/main/README.md\n[ballista-readme]: https://github.com/apache/datafusion-ballista/blob/main/README.md\n[parquet-derive-readme]: parquet_derive/README.md\n[issues]: https://github.com/apache/arrow-rs/issues\n[discussions]: https://github.com/apache/arrow-rs/discussions\n","funding_links":[],"categories":["Rust","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Farrow-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Farrow-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Farrow-rs/lists"}