{"id":28561277,"url":"https://github.com/duneanalytics/arrow_struct","last_synced_at":"2026-02-18T19:31:03.074Z","repository":{"id":268169403,"uuid":"852344582","full_name":"duneanalytics/arrow_struct","owner":"duneanalytics","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-14T20:48:47.000Z","size":38,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-06-10T10:53:12.522Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/duneanalytics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-04T16:39:42.000Z","updated_at":"2024-12-14T20:48:51.000Z","dependencies_parsed_at":"2024-12-14T21:29:45.445Z","dependency_job_id":"b1ac924c-8f82-4fe4-b3ae-e9cc752993c1","html_url":"https://github.com/duneanalytics/arrow_struct","commit_stats":null,"previous_names":["duneanalytics/arrow_struct"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/duneanalytics/arrow_struct","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duneanalytics%2Farrow_struct","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duneanalytics%2Farrow_struct/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duneanalytics%2Farrow_struct/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duneanalytics%2Farrow_struct/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/duneanalytics","download_url":"https://codeload.github.com/duneanalytics/arrow_struct/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duneanalytics%2Farrow_struct/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29591878,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T18:54:29.675Z","status":"ssl_error","status_checked_at":"2026-02-18T18:50:50.517Z","response_time":162,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-10T10:41:51.115Z","updated_at":"2026-02-18T19:31:02.857Z","avatar_url":"https://github.com/duneanalytics.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TODO\n* Benchmark\n    * serde_arrow\n    * arrow2-construct\n* Configurable column cases with attributes\n* Pick a better name\n* Add more convenient interface for converting record batches\n\n# Usage\n\n## RecordBatch vs. StructArray\n\n## Option vs non-Option\nUnless you have a lot of trust in your data, prefer to use `Option` for all struct fields (i.e., `struct Struct { field: Option\u003ci32\u003e }` over `struct Struct { field: i32 }`),\nexcept for nested structs. Arrow does not enforce not-null constraints in RecordBatches. That is, the schema can claim that it's not-null, while in fact the data is null.\n\nWe will panic if we encounter a null field for a not-Option column.\n\n# Performance tips for deserialization\n\n## Zero-copy\n\nIf you can, you should prefer to use references for non-primitive types (i.e., `\u0026str` instead of `String`, `\u0026[u8]` instead of `Bytes`).\nThis avoids clones.\n\n## Avoid Arrow lists\n\nIf you can, you should prefer to avoid using Arrow lists.\nEven if we are careful when deserializing lists, we create a vector for every row with a non-null list.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fduneanalytics%2Farrow_struct","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fduneanalytics%2Farrow_struct","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fduneanalytics%2Farrow_struct/lists"}