{"id":16507591,"url":"https://github.com/chmp/serde_arrow","last_synced_at":"2026-05-03T13:09:46.346Z","repository":{"id":37851422,"uuid":"391627283","full_name":"chmp/serde_arrow","owner":"chmp","description":"Convert sequences of Rust objects to Arrow tables","archived":false,"fork":false,"pushed_at":"2026-03-09T21:43:52.000Z","size":3299,"stargazers_count":99,"open_issues_count":19,"forks_count":28,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-03-10T01:43:23.278Z","etag":null,"topics":["arrow"],"latest_commit_sha":null,"homepage":"https://docs.rs/serde_arrow/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chmp.png","metadata":{"files":{"readme":"Readme.md","changelog":"Changes.md","contributing":"Contributing.md","funding":null,"license":"License.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-08-01T12:56:48.000Z","updated_at":"2026-03-09T21:43:49.000Z","dependencies_parsed_at":"2024-01-27T11:56:16.116Z","dependency_job_id":"9dc098c2-82ca-42fe-b55e-e4b5e752db88","html_url":"https://github.com/chmp/serde_arrow","commit_stats":{"total_commits":253,"total_committers":6,"mean_commits":"42.166666666666664","dds":"0.055335968379446654","last_synced_commit":"186f50d9f2640f6de28d13a6d4e6c98be6dee4f5"},"previous_names":[],"tags_count":50,"template":false,"template_full_name":null,"purl":"pkg:github/chmp/serde_arrow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chmp%2Fserde_arrow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chmp%2Fserde_arrow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chmp%2Fserde_arrow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chmp%2Fserde_arrow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chmp","download_url":"https://codeload.github.com/chmp/serde_arrow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chmp%2Fserde_arrow/sbom","scorecard":{"id":278464,"data":{"date":"2025-08-11","repo":{"name":"github.com/chmp/serde_arrow","commit":"13d4766453f0c07d69ad7454d38dbc7d9c7a08c8"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5,"checks":[{"name":"Code-Review","score":3,"reason":"Found 3/9 approved changesets -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"16 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/release.yml:1","Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: License.md:0","Info: FSF or OSI recognized license: MIT License: License.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/chmp/serde_arrow/release.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/chmp/serde_arrow/test.yml/main?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":3,"reason":"branch protection is not maximal on development and all release branches","details":["Info: 'allow deletion' disabled on branch 'main'","Info: 'force pushes' disabled on branch 'main'","Info: 'branch protection settings apply to administrators' is required to merge on branch 'main'","Warn: 'stale review dismissal' is disabled on branch 'main'","Warn: branch 'main' does not require approvers","Warn: codeowners review is not required on branch 'main'","Warn: 'last push approval' is disabled on branch 'main'","Info: status check found to merge onto on branch 'main'","Info: PRs are required in order to make changes on branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/release.yml:14"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-wv8j-m3hx-924j","Warn: Project is vulnerable to: RUSTSEC-2025-0038","Warn: Project is vulnerable to: RUSTSEC-2024-0370"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-17T15:07:02.209Z","repository_id":37851422,"created_at":"2025-08-17T15:07:02.210Z","updated_at":"2025-08-17T15:07:02.210Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32569821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arrow"],"created_at":"2024-10-11T15:29:12.573Z","updated_at":"2026-05-03T13:09:46.340Z","avatar_url":"https://github.com/chmp.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `serde_arrow` - convert sequences of Rust objects to Arrow arrays and back again\n\n[Crate info](https://crates.io/crates/serde_arrow)\n| [API docs](https://docs.rs/serde_arrow/latest/serde_arrow/)\n| [Example](#example)\n| [Related packages \u0026 performance](#related-packages--performance)\n| [Status](serde_arrow/Status.md)\n| [License](#license)\n| [Changes](Changes.md)\n| [Contributing](Contributing.md)\n\nThe arrow in-memory format is a powerful way to work with data frame like\nstructures. The surrounding ecosystem includes a rich set of libraries, ranging\nfrom data frames such as [Polars][polars] to query engines such as\n[DataFusion][datafusion]. However, the API of the underlying Rust crates can be\nat times cumbersome to use due to the statically typed nature of Rust.\n\n`serde_arrow`, offers a simple way to convert Rust objects into Arrow arrays and\nback.  `serde_arrow` relies on the [Serde](https://serde.rs) package to\ninterpret Rust objects. Therefore, adding support for `serde_arrow` to custom\ntypes is as easy as using Serde's derive macros.\n\nIn the Rust ecosystem there are two competing implementations of the arrow\nin-memory format. `serde_arrow` supports both [`arrow`][arrow] and\n[`arrow2`][arrow2] for schema tracing, serialization from Rust structs to\narrays, and deserialization from arrays to Rust structs.\n\n[arrow]: https://docs.rs/arrow/latest/arrow/\n[arrow2]: https://docs.rs/arrow2/latest/arrow2/\n[polars]: https://github.com/pola-rs/polars\n[datafusion]: https://github.com/apache/arrow-datafusion/\n\n## Example\n\nThe following examples assume that `serde_arrow` is added to the `Cargo.toml`\nfile and its features are configured. `serde_arrow` supports different `arrow`\nand `arrow2` versions. The relevant one can be selected by specifying the\ncorrect feature (e.g., `arrow-51` to support `arrow=51`). See\n[here][feature-docs] for more details.\n\n[feature-docs]: https://docs.rs/serde_arrow/latest/serde_arrow/#features\n\nThe following examples use the following Rust structure and example records\n\n```rust\n#[derive(Serialize, Deserialize)]\nstruct Record {\n    a: f32,\n    b: i32,\n}\n\nlet records = vec![\n    Record { a: 1.0, b: 1 },\n    Record { a: 2.0, b: 2 },\n    Record { a: 3.0, b: 3 },\n];\n```\n\n### Serialize to `arrow` `RecordBatch`\n\n```rust\nuse arrow::datatypes::FieldRef;\nuse serde_arrow::schema::{SchemaLike, TracingOptions};\n\n// Determine Arrow schema\nlet fields = Vec::\u003cFieldRef\u003e::from_type::\u003cRecord\u003e(TracingOptions::default())?;\n\n// Build a record batch\nlet batch = serde_arrow::to_record_batch(\u0026fields, \u0026records)?;\n```\n\nThis `RecordBatch` can now be written to disk using [ArrowWriter] from the [parquet] crate.\n\n[ArrowWriter]: https://docs.rs/parquet/latest/parquet/arrow/arrow_writer/struct.ArrowWriter.html\n[parquet]: https://docs.rs/parquet/latest/parquet/\n\n\n```rust\nuse use parquet::arrow::ArrowWriter;\n\nlet file = File::create(\"example.pq\")?;\nlet mut writer = ArrowWriter::try_new(file, batch.schema(), None)?;\nwriter.write(\u0026batch)?;\nwriter.close()?;\n```\n\n### Usage from python\n\nThe written files can be read in Python via\n\n```python\n# using polars\n\u003e\u003e\u003e import polars as pl\n\u003e\u003e\u003e pl.read_parquet(\"example.pq\")\nshape: (3, 2)\n┌─────┬─────┐\n│ a   ┆ b   │\n│ --- ┆ --- │\n│ f32 ┆ i32 │\n╞═════╪═════╡\n│ 1.0 ┆ 1   │\n│ 2.0 ┆ 2   │\n│ 3.0 ┆ 3   │\n└─────┴─────┘\n\n# using pandas\n\u003e\u003e\u003e import pandas as pd\n\u003e\u003e\u003e pd.read_parquet(\"example.pq\")\n     a  b\n0  1.0  1\n1  2.0  2\n2  3.0  3\n```\n\n[arrow2-guide]: https://jorgecarleitao.github.io/arrow2\n\n## Related packages \u0026 Performance\n\n- [`arrow`][arrow]: the JSON component of the official Arrow package supports\n   serializing objects that support serialize via the [Decoder][serde-decoder]\n   object. It supports primitives types, structs and lists\n- [`arrow2-convert`][arrow2-convert]: adds derive macros to convert objects from\n  and to arrow2 arrays. It supports primitive types, structs, lists, and\n  chrono's date time types. Enum support is experimental according to the\n  Readme. If performance is the main objective, `arrow2-convert` is a good\n  choice as it has no or minimal overhead over building the arrays manually.\n- [`arrow-convert`][arrow-convert]: a port of `arrow2-convert` to `arrow-rs`\n- [`typed-arrow`][typed-arrow]: derive based converter of Rust structs to arrow\n\n[serde-decoder]: https://docs.rs/arrow-json/latest/arrow_json/reader/struct.Decoder.html\n[arrow-convert]: https://github.com/Swoorup/arrow-convert\n[arrow2-convert]: https://github.com/DataEngineeringLabs/arrow2-convert\n[typed-arrow]: https://github.com/tonbo-io/typed-arrow\n\nThe different implementation have the following performance differences, when\ncompared to arrow2-convert:\n\n![Time ](timings.png)\n\nThe detailed runtimes of the [benchmarks](./serde_arrow/benches/groups/) are listed below.\n\n\u003c!-- start:benchmarks --\u003e\n### complex_common_serialize(100000)\n\n| label                        | time [ms] | arrow2_convert: | serde_arrow::to | serde_arrow::to | arrow_json::Rea |\n|------------------------------|-----------|-----------------|-----------------|-----------------|-----------------|\n| arrow2_convert::TryIntoArrow |     54.01 |            1.00 |            0.31 |            0.30 |            0.14 |\n| serde_arrow::to_arrow2       |    173.84 |            3.22 |            1.00 |            0.98 |            0.46 |\n| serde_arrow::to_arrow        |    177.92 |            3.29 |            1.02 |            1.00 |            0.47 |\n| arrow_json::ReaderBuilder    |    378.48 |            7.01 |            2.18 |            2.13 |            1.00 |\n\n### complex_common_serialize(1000000)\n\n| label                        | time [ms] | arrow2_convert: | serde_arrow::to | serde_arrow::to | arrow_json::Rea |\n|------------------------------|-----------|-----------------|-----------------|-----------------|-----------------|\n| arrow2_convert::TryIntoArrow |    576.81 |            1.00 |            0.34 |            0.33 |            0.16 |\n| serde_arrow::to_arrow2       |   1701.46 |            2.95 |            1.00 |            0.97 |            0.46 |\n| serde_arrow::to_arrow        |   1748.89 |            3.03 |            1.03 |            1.00 |            0.48 |\n| arrow_json::ReaderBuilder    |   3676.51 |            6.37 |            2.16 |            2.10 |            1.00 |\n\n### primitives_serialize(100000)\n\n| label                        | time [ms] | arrow2_convert: | serde_arrow::to | serde_arrow::to | arrow_json::Rea |\n|------------------------------|-----------|-----------------|-----------------|-----------------|-----------------|\n| arrow2_convert::TryIntoArrow |     15.83 |            1.00 |            0.51 |            0.36 |            0.12 |\n| serde_arrow::to_arrow2       |     30.90 |            1.95 |            1.00 |            0.70 |            0.23 |\n| serde_arrow::to_arrow        |     43.96 |            2.78 |            1.42 |            1.00 |            0.33 |\n| arrow_json::ReaderBuilder    |    133.97 |            8.46 |            4.34 |            3.05 |            1.00 |\n\n### primitives_serialize(1000000)\n\n| label                        | time [ms] | arrow2_convert: | serde_arrow::to | serde_arrow::to | arrow_json::Rea |\n|------------------------------|-----------|-----------------|-----------------|-----------------|-----------------|\n| arrow2_convert::TryIntoArrow |    153.07 |            1.00 |            0.47 |            0.35 |            0.11 |\n| serde_arrow::to_arrow2       |    327.32 |            2.14 |            1.00 |            0.74 |            0.23 |\n| serde_arrow::to_arrow        |    440.39 |            2.88 |            1.35 |            1.00 |            0.31 |\n| arrow_json::ReaderBuilder    |   1429.31 |            9.34 |            4.37 |            3.25 |            1.00 |\n\n\u003c!-- end:benchmarks --\u003e\n\n## License\n\n```text\nCopyright (c) 2021 - 2024 Christopher Prohm and contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchmp%2Fserde_arrow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchmp%2Fserde_arrow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchmp%2Fserde_arrow/lists"}