{"id":13494775,"url":"https://github.com/pennsignals/aptos","last_synced_at":"2025-03-28T14:31:53.983Z","repository":{"id":57411164,"uuid":"99624258","full_name":"pennsignals/aptos","owner":"pennsignals","description":":sunny: A tool for validating data using JSON Schema and converting JSON Schema documents into different data-interchange formats","archived":true,"fork":false,"pushed_at":"2020-10-09T22:24:27.000Z","size":74,"stargazers_count":150,"open_issues_count":6,"forks_count":24,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-04-16T01:37:50.800Z","etag":null,"topics":["avro","avro-schema","cli","command-line-tool","data-interchange","json-schema","python3","schema-conversion","validation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pennsignals.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":"contributing.json","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-07T22:01:12.000Z","updated_at":"2024-01-04T16:16:18.000Z","dependencies_parsed_at":"2022-09-09T16:02:08.328Z","dependency_job_id":null,"html_url":"https://github.com/pennsignals/aptos","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pennsignals%2Faptos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pennsignals%2Faptos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pennsignals%2Faptos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pennsignals%2Faptos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pennsignals","download_url":"https://codeload.github.com/pennsignals/aptos/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246046107,"owners_count":20714907,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","avro-schema","cli","command-line-tool","data-interchange","json-schema","python3","schema-conversion","validation"],"created_at":"2024-07-31T19:01:28.021Z","updated_at":"2025-03-28T14:31:53.476Z","avatar_url":"https://github.com/pennsignals.png","language":"Python","funding_links":[],"categories":["Python","JSON Schema Tools"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/pennsignals/aptos\"\u003e\u003cimg src=\"https://rawgit.com/rightlag/d97e8d74d5bbb5b2e0cbe2938c40802d/raw/c957e9af39b1e6d94c1b5e823685084e7beba6ad/aptos.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://travis-ci.org/pennsignals/aptos\"\u003e\u003cimg src=\"https://img.shields.io/travis/pennsignals/aptos.svg?style=flat-square\" alt=\"Build Status\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://coveralls.io/github/pennsignals/aptos\"\u003e\u003cimg src=\"https://img.shields.io/coveralls/pennsignals/aptos.svg?style=flat-square\" alt=\"Coverage Status\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/aptos\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/seven.svg?style=flat-square\" alt=\"PyPI\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://gitter.im/aptos-io/Lobby\"\u003e\u003cimg src=\"https://img.shields.io/gitter/room/aptos/aptos-io.svg?style=flat-square\" alt=\"Gitter\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003e Validate client-submitted data using [JSON Schema](http://json-schema.org/) documents and convert JSON Schema documents into different data-interchange formats.\n\n## Contents\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Data Validation](#data-validation)\n- [Data Validation CLI](#data-validation-cli)\n- [Data Validation API](#data-validation-api)\n- [Structured Messaged Generation](#structured-message-generation)\n- [Supported Data-Interchange Formats](#supported-data-interchange-formats)\n- [Avro](#avro)\n- [Data-Interchange CLI](#data-interchange-cli)\n- [Data-Interchange API](#data-interchange-api)\n- [Testing](#testing)\n- [Additional Resources](#additional-resources)\n- [Future Considerations](#future-considerations)\n- [Maintainers](#maintainers)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Why aptos?\n\n- Validate client-submitted data\n- Convert JSON Schema documents into different data-interchange formats\n- Simple syntax\n- CLI support for data validation and JSON Schema conversion\n- [Stop Being a \"Janitorial\" Data Scientist](https://medium.com/@rightlag/stop-being-a-janitorial-data-scientist-5959cccbeac)\n\n## Installation\n\n**via pip**\n\n    $ pip install aptos\n\n**via git**\n\n    $ git clone https://github.com/pennsignals/aptos.git \u0026\u0026 cd aptos\n    $ python setup.py install\n\n## Usage\n\n`aptos` supports the following capabilities:\n\n - **Data Validation:** Validate client-submitted data using [validation keywords](http://json-schema.org/latest/json-schema-validation.html#rfc.section.6) described in the JSON Schema specification.\n - **Schema Conversion:** Convert JSON Schema documents into different data-interchange formats. See the list of [supported data-interchange formats](#supported-data-interchange-formats) for more information.\n\n```\nusage: aptos [arguments] SCHEMA\n\naptos is a tool for validating client-submitted data using the JSON Schema\nvocabulary and converts JSON Schema documents into different data-interchange\nformats.\n\npositional arguments:\n  schema              JSON document containing the description\n\noptional arguments:\n  -h, --help          show this help message and exit\n\nArguments:\n  {validate,convert}\n    validate          Validate a JSON instance\n    convert           Convert a JSON Schema into a different data-interchange\n                      format\n\nMore information on JSON Schema: http://json-schema.org/\n\n```\n\n## Data Validation\n\nHere is a basic example of a JSON Schema:\n\n```json\n{\n    \"title\": \"Person\",\n    \"type\": \"object\",\n    \"properties\": {\n        \"firstName\": {\n            \"type\": \"string\"\n        },\n        \"lastName\": {\n            \"type\": \"string\"\n        },\n        \"age\": {\n            \"description\": \"Age in years\",\n            \"type\": \"integer\",\n            \"minimum\": 0\n        }\n    },\n    \"required\": [\"firstName\", \"lastName\"]\n}\n```\n\nGiven a JSON Schema, `aptos` can validate client-submitted data to ensure that it satisfies a certain number of criteria.\n\nJSON Schema [Validation keywords](http://json-schema.org/latest/json-schema-validation.html#rfc.section.6) such as `minimum` and `required` can be used to impose requirements for successful validation of an instance. In the JSON Schema above, both the `firstName` and `lastName` properties are required, and the `age` property *MUST* have a value greater than or equal to 0.\n\n| Valid Instance :heavy_check_mark:                     | Invalid Instance :heavy_multiplication_x:                                                                               |\n|-------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|\n| `{\"firstName\": \"John\", \"lastName\": \"Doe\", \"age\": 42}` | `{\"firstName\": \"John\", \"age\": -15}` (missing required property `lastName` and `age` is not greater than or equal to 0)  |\n\n`aptos` can validate client-submitted data using either the CLI or the API:\n\n### Data Validation CLI\n\n    $ aptos validate -instance INSTANCE SCHEMA\n\n**Arguments:**\n\n - **INSTANCE:** JSON document being validated\n - **SCHEMA:** JSON document containing the description\n\n**Example - macOS:**\n\n    $ aptos validate -instance '{\"firstName\": \"John\"}' person.json\n\n**Example - Windows:**\n\n    \u003e aptos validate -instance \"{\\\"firstName\\\": \\\"John\\\"}\" person.json\n\n| Successful Validation :heavy_check_mark:                                                                 | Unsuccessful Validation :heavy_multiplication_x:                                                         |\n|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|\n| ![](https://user-images.githubusercontent.com/2184329/29053486-5c787966-7bbe-11e7-8fd3-4cb51d87d7d9.png) | ![](https://user-images.githubusercontent.com/2184329/29053538-afcce9c6-7bbe-11e7-8be5-61ac1d876fc1.png) |\n\n### Data Validation API\n\n```python\nimport json\n\nfrom aptos.parser import SchemaParser\nfrom aptos.visitor import ValidationVisitor\n\n\nwith open('/path/to/schema') as fp:\n    schema = json.load(fp)\ncomponent = SchemaParser.parse(schema)\n# Invalid client-submitted data (instance)\ninstance = {\n  'firstName': 'John'\n}\ntry:\n    component.accept(ValidationVisitor(instance))\nexcept AssertionError as e:\n    print(e)  # instance {'firstName': 'John'} is missing required property 'lastName'\n```\n\n## Structured Message Generation\n\nGiven a JSON Schema, `aptos` can generate different structured messages.\n\n:warning: **Note:** The JSON Schema being converted *MUST* be a valid [JSON Object](https://spacetelescope.github.io/understanding-json-schema/reference/object.html).\n\n## Supported Data-Interchange Formats\n\n| Format                                                              |         Supported        | Notes                       |\n|---------------------------------------------------------------------|:------------------------:|-----------------------------|\n| [Apache Avro](https://avro.apache.org/)                             |    :heavy_check_mark:    |                             |\n| [Protocol Buffers](https://developers.google.com/protocol-buffers/) | :heavy_multiplication_x: | Planned for future releases |\n| [Apache Thrift](https://thrift.apache.org/)                         | :heavy_multiplication_x: | Planned for future releases |\n| [Apache Parquet](https://parquet.apache.org/)                       | :heavy_multiplication_x: | Planned for future releases |\n\n### Avro\n\nUsing the `Person` schema in the previous example, `aptos` can convert the schema into the Avro data-interchange format using either the CLI or the API.\n\n`aptos` maps the following JSON schema types to Avro types:\n\n| JSON Schema Type | Avro Type |\n|------------------|-----------|\n| `string`         | `string`  |\n| `boolean`        | `boolean` |\n| `null`           | `null`    |\n| `integer`        | `long`    |\n| `number`         | `double`  |\n| `object`         | `record`  |\n| `array`          | `array`   |\n\n\u003e JSON Schema documents containing the `enum` validation keyword are mapped to Avro [`enum`](http://avro.apache.org/docs/current/spec.html#Enums) `symbols` attribute.\n\n\u003e JSON Schema documents with the `type` keyword as an array are mapped to Avro [Union](http://avro.apache.org/docs/current/spec.html#Unions) types.\n\n## Data-Interchange CLI\n\n    $ aptos convert -format FORMAT SCHEMA\n\n**Arguments:**\n\n - **FORMAT:** Data-interchange format\n - **SCHEMA:** JSON document containing the description\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://user-images.githubusercontent.com/2184329/29071365-3d6e7504-7c11-11e7-959e-abcfe15f5e96.png\" width=\"600\"\u003e\n\u003c/p\u003e\n\n## Data-Interchange API\n\n```python\nimport json\n\nfrom aptos.parser import SchemaParser\nfrom aptos.schema.visitor import AvroSchemaVisitor\n\n\nwith open('/path/to/schema') as fp:\n    schema = json.load(fp)\ncomponent = SchemaParser.parse(schema)\nrecord = component.accept(AvroSchemaVisitor())\nprint(json.dumps(record, indent=2))\n```\n\nThe above code generates the following Avro schema:\n\n```json\n{\n  \"type\": \"record\",\n  \"fields\": [\n    {\n      \"doc\": \"\",\n      \"type\": \"string\",\n      \"name\": \"lastName\"\n    },\n    {\n      \"doc\": \"\",\n      \"type\": \"string\",\n      \"name\": \"firstName\"\n    },\n    {\n      \"doc\": \"Age in years\",\n      \"type\": \"long\",\n      \"name\": \"age\"\n    }\n  ],\n  \"name\": \"Person\"\n}\n```\n\n## Testing\n\nAll unit tests exist in the [tests](tests) directory.\n\nTo run tests, execute the following command:\n\n    $ python setup.py test\n\n## Additional Resources\n\n - [Stop Being a \"Janitorial\" Data Scientist](https://medium.com/@rightlag/stop-being-a-janitorial-data-scientist-5959cccbeac) - *A blog post explaining why aptos was created*\n - [Understanding JSON Schema](https://spacetelescope.github.io/understanding-json-schema/) - *An excellent guide for schema authors, from the [Space Telescope Science Institute](http://www.stsci.edu/portal/)*\n\n## Future Considerations\n\n- [Swagger](https://swagger.io/) support\n- Additional [data-interchange](#supported-data-interchange-formats) formats\n\n## Maintainers\n\n| ![Jason Walsh](https://avatars3.githubusercontent.com/u/2184329?v=3\u0026s=128) |\n|:--------------------------------------------------------------------------:|\n|                 [Jason Walsh](https://twitter.com/rightlag)                |\n\n## Contributing\n\nContributions welcome! Please read the [`contributing.json`](contributing.json) file first.\n\nJoin our [Slack](https://aptos-io.slack.com) channel!\n\n## License\n\n[Apache 2.0](LICENSE) \u0026copy; [Penn Signals](https://github.com/pennsignals)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpennsignals%2Faptos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpennsignals%2Faptos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpennsignals%2Faptos/lists"}