{"id":37078691,"url":"https://github.com/kun-fang/avro-data-model","last_synced_at":"2026-01-14T09:12:42.409Z","repository":{"id":57413085,"uuid":"149920819","full_name":"kun-fang/avro-data-model","owner":"kun-fang","description":"A Python ORM for Avro Schemas","archived":false,"fork":false,"pushed_at":"2021-05-14T02:57:25.000Z","size":26,"stargazers_count":8,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-27T03:46:41.440Z","etag":null,"topics":["avro-data","avro-schema","data-models","orm-library","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kun-fang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-22T21:15:37.000Z","updated_at":"2022-05-23T20:28:24.000Z","dependencies_parsed_at":"2022-08-29T15:21:43.244Z","dependency_job_id":null,"html_url":"https://github.com/kun-fang/avro-data-model","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/kun-fang/avro-data-model","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kun-fang%2Favro-data-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kun-fang%2Favro-data-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kun-fang%2Favro-data-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kun-fang%2Favro-data-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kun-fang","download_url":"https://codeload.github.com/kun-fang/avro-data-model/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kun-fang%2Favro-data-model/sbom","scorecard":{"id":573572,"data":{"date":"2025-08-11","repo":{"name":"github.com/kun-fang/avro-data-model","commit":"1a657e20e666b534d0196888ae580ad7caddadeb"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Code-Review","score":0,"reason":"Found 0/20 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: pipCommand not pinned by hash: init.sh:9","Warn: pipCommand not pinned by hash: init.sh:10","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.md:0","Info: FSF or OSI recognized license: MIT License: LICENSE.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-20T17:01:28.675Z","repository_id":57413085,"created_at":"2025-08-20T17:01:28.675Z","updated_at":"2025-08-20T17:01:28.675Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414924,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro-data","avro-schema","data-models","orm-library","python3"],"created_at":"2026-01-14T09:12:41.738Z","updated_at":"2026-01-14T09:12:42.402Z","avatar_url":"https://github.com/kun-fang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Avro Data Model\n=====\n\n## Introduction\n[Apache Avro](http://avro.apache.org/) is a data serialization framework. It is used in data serialization (especially in Hadoop ecosystem) and RPC protocols. It has libraries to support many languages. The library supports code generation with static languages like Java, while for dynamic languages for example python, code generation is not necessary.\n\nWhen avro data is deserialized in Python environment, it was stored as a dictionary in memory. As a dictionary, it looses all the interesting features provided by the avro schema. For example, you can modify an integer field with a string without getting any errors. As a dictionary, it also doesn't provide any nice features from a normal class, for example, if an avro schema has `firstName` and `lastName` fields, it is not easy to define a `fullName` function to generate the full name.\n\n## Use Cases of the Library\nIn stream processing and RPC protocols, strict data types are required to make sure the system runs correctly. In Python, avro data is converted to a dictionary, which doesn't guarantee types and also doesn't provide a custom class hierarchy. I am looking to develop a way so that a class can be build on top of an avro schema, so that it can keep correct data type and also has a class structure.\n\nMy solution is similar to what [SQLAlchemy ORM](https://www.sqlalchemy.org) does. You need to manually create classes corresponding to avro schemas. However, fields of the avro schemas are all extracted from `avsc` file instead of being manually defined like SQLAlchemy. The classes allow defining methods to introduce new properties or new validations. Please check the following examples for how to use the library.\n\nThe purpose of the library is to bridge the gap between dynamical typed python and the use cases that requires strong types. This library should be restricted to places where static types are required. Otherwise, you will loose all the happiness playing with Python if applying this library everywhere.\n\n\n## Example\n### A Simple Example\n**User.avsc**\n```\n{\n  \"type\": \"record\",\n  \"name\": \"User\",\n  \"fields\": [\n    {\n      \"name\": \"lastName\",\n      \"type\": \"string\"\n    },\n    {\n      \"name\": \"firstName\",\n      \"type\": \"string\"\n    }\n  ]\n}\n```\nThe following code defined a User class associated with the schema\n```\n@avro_schema(AvroDataNames(default_namespace=\"example.avro\"), schema_file=\"User.avsc\")\nclass User(object):\n  def fullname(self):\n    return \"{} {}\".format(self.firstName, self.lastName)\n```\nWith this class definition, the full name can be obtained with the function call.\n```\nuser = User({\"firstName\": \"Alyssa\", \"lastName\": \"Yssa\"})\nprint(user.fullname())\n# Alyssa Yssa\n```\n\n### Avro Schema with Extra Validation\nIn some use cases, some extra validations are required, for example:\n**Date.avsc**\n```\n{\n  \"name\": \"Date\",\n  \"type\": \"record\",\n  \"fields\": [\n    {\n      \"name\": \"year\",\n      \"type\": \"int\"\n    },\n    {\n      \"name\": \"month\",\n      \"type\": \"int\"\n    },\n    {\n      \"name\": \"day\",\n      \"type\": \"int\"\n    }\n  ]\n}\n```\nThe `month` and `day` of a date cannot be arbitrary integers. A extra validation can be done as following:\n```\n@avro_schema(AvroDataNames(default_namespace=\"example.avro\"), schema_file=\"Date.avsc\")\nclass Date(object):\n  def __init__(self, value):\n    if isinstance(value, datetime.date):\n      value = {\n          'year': value.year,\n          'month': value.month,\n          'day': value.day\n      }\n    super().__init__(value)\n\n  def date(self):\n    return datetime.date(self.year, self.month, self.day)\n\n  def validate(self, data):\n    return super().validate(data) \\\n        and datetime.date(data['year'], data['month'], data['day'])\n```\nThe `Date` class can validate the input before assign it to then underlying avro schema\n```\ndate = Date({\"year\": 2018, \"month\": 12, \"date\": 99})\n# ValueError: day is out of range for month\ndate = Date(datetime.date(2018, 12, 12))\n# No Error\n```\n\n### Extract an avro schema defined in an outer schema\nSometimes an avro schema is defined in another schema\n**Employee.avsc**\n```\n{\n  \"type\": \"record\",\n  \"name\": \"Employee\",\n  \"namespace\": \"com.test\",\n  \"fields\": [\n    {\n      \"name\": \"id\"\n      \"type\": \"string\"\n    },\n    {\n      \"name\": \"name\",\n      \"type\": {\n        \"type\": \"record\",\n        \"name\": \"Name\",\n        \"namespace\": \"com.test\",\n        \"fields\": [\n          {\n            \"name\": \"lastName\",\n            \"type\": \"string\"\n          },\n          {\n            \"name\": \"firstName\",\n            \"type\": \"string\"\n          }\n        ]\n      }\n    }\n  ]\n}\n```\nThe schema `com.test.Name` is defined in `com.test.Employee`. There is no `Name.avsc`, but you can still define a class for it the schema:\n```\n# Parent schema must be define first.\n@avro_schema(\n    EXAMPLE_NAMES,\n    schema_file=os.path.join(DIRNAME, \"Employee.avsc\"))\nclass Employee(object):\n    pass\n\n\n# Full name is required\n@avro_schema(EXAMPLE_NAMES, full_name=\"com.test.Name\")\nclass Name(object):\n    pass\n\n\nname = Name({{\"firstName\": \"Alyssa\", \"lastName\": \"Yssa\"})\nprint(name)\n# {'firstName': 'Alyssa', 'lastName': 'Yssa'}\n```\n\n## Contributing\nAfter cloning/forking the repo, navigate to the directory and run\n```\nsource init.sh\n```\nThe python environment should be ready for you.\n\n## Authors\n\n* **Kun Fang** - (https://github.com/kun-fang)\n\nSee also the list of [contributors](https://github.com/your/project/contributors) who participated in this project.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkun-fang%2Favro-data-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkun-fang%2Favro-data-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkun-fang%2Favro-data-model/lists"}