{"id":13671486,"url":"https://github.com/heremaps/flatdata","last_synced_at":"2025-04-12T22:37:04.892Z","repository":{"id":28687687,"uuid":"118035076","full_name":"heremaps/flatdata","owner":"heremaps","description":"Write-once, read-many, minimal overhead binary structured file format.","archived":false,"fork":false,"pushed_at":"2024-04-24T14:26:33.000Z","size":1890,"stargazers_count":176,"open_issues_count":25,"forks_count":27,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-04-25T02:41:08.040Z","etag":null,"topics":["cpp","memory-mapped","python3","rust","serialization","storage"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/heremaps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-18T20:34:58.000Z","updated_at":"2024-06-17T12:55:52.336Z","dependencies_parsed_at":"2023-11-10T11:36:23.340Z","dependency_job_id":"0d8de2f7-3ea1-4bd2-9d7f-7f652ba5fbde","html_url":"https://github.com/heremaps/flatdata","commit_stats":{"total_commits":355,"total_committers":18,"mean_commits":19.72222222222222,"dds":0.5408450704225352,"last_synced_commit":"494f7731fa3cd6780d767fc5d218c783714acdee"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heremaps%2Fflatdata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heremaps%2Fflatdata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heremaps%2Fflatdata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heremaps%2Fflatdata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/heremaps","download_url":"https://codeload.github.com/heremaps/flatdata/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248643006,"owners_count":21138353,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","memory-mapped","python3","rust","serialization","storage"],"created_at":"2024-08-02T09:01:10.788Z","updated_at":"2025-04-12T22:37:04.871Z","avatar_url":"https://github.com/heremaps.png","language":"Python","readme":"# \u003cimg align=\"left\" src=\"https://github.com/heremaps/flatdata/blob/master/docs/logo.svg\" width=\"48\" height=\"48\"\u003e flatdata [![Build Status](https://api.travis-ci.com/heremaps/flatdata.svg?branch=master)](https://travis-ci.com/heremaps/flatdata/)\n\n_Write-once, read-many, minimal overhead binary structured file format._\n\nFlatdata is a library providing data structures for convenient creation, storage and access of packed memory-mappable structures with minimal overhead.\n\nWith `flatdata`, the user defines a schema of the data format using a very simple schema language that supports plain structs, vectors and multivectors. The schema is then used to generate builders and readers for serialization and deserialization of the data to an archive of files on disk.\n\nThe data is serialized in a portable way which allows zero-overhead random access to it by using memory mapped storage: the operating system facilities are used for loading, caching and paging of the data, and most important, accessing it as if it were in memory.\n\n## Why `flatdata`\n\nFlatdata helps creating efficient _read-only_ datasets:\n\n* Zero overhead random access\n* Support for bit and byte packing\n* Structuring data using a schema definition\n* Optimized for large read-only datasets\n* Portable, with support for multiple languages\n\nFlatdata _doesn't_ provide:\n\n* Backwards compatible schema evolution\n* Support for mutable datasets\n* Portable floating point serialization\n\nFor more details read [why flatdata](docs/why-flatdata.md).\n\n## Using `flatdata`\n\n### Creating a schema\n\nDefine a flatdata archive:\n\n```cpp\nnamespace loc {\n    struct Point {\n        x : u32 : 32;\n        y : u32 : 32;\n    }\n    archive Locations {\n        pois : vector\u003c Point \u003e;\n    }\n}\n```\n\nThe full schema documentation can be found [here](docs/schema-language.md).\n\n### Generating a module\n\nFlatdata relies on a generator that takes a flatdata schema file as an input and\ngenerates a module for one of the supported languages.\n\nThe following languages are supported:\n\n* First-class citizen implementations:\n  * **[C++](./flatdata-cpp)** - used extensively, tested excessively normally receives features first\n  * **[Rust](./flatdata-rs)** - the newest addition to the family\n* Read-only implementations:\n  * **[Python](./flatdata-py)** - used mostly for inspecting the data\n  * **[Dot](./flatdata-dot)** - used to generate diagrams of the schema\n  * **[Go](./flatdata-go)** - beta implementation\n\n### Generate code\n\nSee the [generator's README](./flatdata-py/generator/README.md#usage) for instructions.\n\n## License\n\nCopyright (c) 2017-2020 HERE Europe B.V.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n   http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in this document by you, as defined in the Apache-2.0 license,\nwithout any additional terms or conditions.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheremaps%2Fflatdata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fheremaps%2Fflatdata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheremaps%2Fflatdata/lists"}