{"id":18797787,"url":"https://github.com/networktocode/schema-enforcer","last_synced_at":"2025-04-04T19:15:21.556Z","repository":{"id":37086200,"uuid":"254103222","full_name":"networktocode/schema-enforcer","owner":"networktocode","description":"Schema Enforcer provides a framework for testing structured data against schema definitions.","archived":false,"fork":false,"pushed_at":"2025-02-18T05:47:07.000Z","size":1074,"stargazers_count":48,"open_issues_count":34,"forks_count":9,"subscribers_count":34,"default_branch":"develop","last_synced_at":"2025-03-28T18:12:22.606Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/networktocode.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-08T13:57:49.000Z","updated_at":"2025-02-21T12:09:48.000Z","dependencies_parsed_at":"2024-02-09T02:24:09.112Z","dependency_job_id":"8abafad2-9492-4362-a083-1f0a279cced0","html_url":"https://github.com/networktocode/schema-enforcer","commit_stats":{"total_commits":350,"total_committers":17,"mean_commits":20.58823529411765,"dds":0.6685714285714286,"last_synced_commit":"8a1a8d7e471d10ebc09e9fd494ecacc94d0c89b0"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fschema-enforcer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fschema-enforcer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fschema-enforcer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fschema-enforcer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/networktocode","download_url":"https://codeload.github.com/networktocode/schema-enforcer/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247234930,"owners_count":20905854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T22:09:33.579Z","updated_at":"2025-04-04T19:15:21.527Z","avatar_url":"https://github.com/networktocode.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Schema Enforcer\n\nSchema Enforcer provides a framework for testing structured data against schema definitions using [JSONSchema](https://json-schema.org/understanding-json-schema/index.html).\n\n## Getting Started\n\n### Install\n\nSchema Enforcer is a python library which is available on PyPi. It requires a python version of 3.8 or greater. Once a supported version of python is installed on your machine, pip can be used to install the tool by using the command `python -m pip install schema-enforcer`.\n\n```cli\npython -m pip install schema-enforcer\n```\n\n### Overview\n\nSchema Enforcer requires that two different elements be defined by the user:\n\n- Schema Definition Files: These are files which define the schema to which a given set of data should adhere.\n- Structured Data Files: These are files which contain data that should adhere to the schema defined in one (or multiple) of the schema definition files.\n\n\u003e Note: Data which needs to be validated against a schema definition can come in the form of Structured Data Files or Ansible host vars. Ansible is not installed by default when schema-enforcer is installed. In order to use Ansible features, ansible must already be available or must be declared as an optional dependency when schema-enforcer upon installation. In the interest of brevity and simplicity, this README.md contains discussion only of Structured Data Files -- for more information on how to use `schema-enforcer` with ansible host vars, see [the ansible_command README](docs/ansible_command.md)\n\nWhen `schema-enforcer` runs, it assumes directory hierarchy which should be in place from the folder in which the tool is run.\n\n- `schema-enforcer` will search for **schema definition files** nested inside of `./schema/schemas/` which end in `.yml`, `.yaml`, or `.json`.\n- `schema-enforcer` will do a recursive search for **structured data files** starting in the current working diretory (`./`). It does this by searching all directories (including the current working directory) for files ending in `.yml`, `.yaml`, or `.json`. The `schema` folder and it's subdirectories are excluded from this search by default.\n\n```cli\nbash$ cd examples/example1\nbash$ tree\n.\n├── chi-beijing-rt1\n│   ├── dns.yml\n│   └── syslog.yml\n├── eng-london-rt1\n│   ├── dns.yml\n│   └── ntp.yml\n└── schema\n    └── schemas\n        ├── dns.yml\n        ├── ntp.yml\n        └── syslog.yml\n\n4 directories, 7 files\n```\n\nIn the above example, `chi-beijing-rt1` is a directory with structured data files containing some configuration for a router named `chi-beijing-rt1`. There are two structured data files inside of this folder, `dns.yml` and `syslog.yml`. Similarly, the `eng-london-rt1` directory contains definition files for a router named `eng-london-rt1` -- `dns.yml` and `ntp.yml`.\n\nThe file `chi-beijing-rt1/dns.yml` defines the DNS servers `chi-beijing.rt1` should use. The data in this file includes a simple hash-type data structure with a key of `dns_servers` and a value of an array. Each element in this array is a hash-type object with a key of `address` and a value which is the string of an IP address.\n\n```yaml\nbash$ cat chi-beijing-rt1/dns.yml\n# jsonschema: schemas/dns_servers\n---\ndns_servers:\n  - address: \"10.1.1.1\"\n  - address: \"10.2.2.2\"\n```\n\u003e Note: The line `# jsonschema: schemas/dns_servers` tells `schema-enforcer` the ID of the schema which the structured data defined in the file should be validated against. The schema ID is defined by the `$id` top level key in a schema definition. More information on how the structured data is mapped to a schema ID to which it should adhere can be found in the [mapping_schemas README](./docs/mapping_schemas.md)\n\nThe file `schema/schemas/dns.yml` is a schema definition file. It contains a schema definition for ntp servers written in JSONSchema. The data in `chi-beijing-rt1/dns.yml` and `eng-london-rt1/dns.yml` should adhere to the schema defined in this schema definition file.\n\n```yaml\nbash$ cat schema/schemas/dns.yml\n---\n$schema: \"http://json-schema.org/draft-07/schema#\"\n$id: \"schemas/dns_servers\"\ndescription: \"DNS Server Configuration schema.\"\ntype: \"object\"\nproperties:\n  dns_servers:\n    type: \"array\"\n    items:\n      type: \"object\"\n      properties:\n        name:\n          type: \"string\"\n        address:\n          type: \"string\"\n          format: \"ipv4\"\n        vrf:\n          type: \"string\"\n      required:\n        - \"address\"\n      uniqueItems: true\nrequired:\n  - \"dns_servers\"\n```\n\n\u003e Note: The cat of the schema definition file may be a little scary if you haven't seen JSONSchema before. Don't worry too much if it is difficult to parse right now. The important thing to note is that this file contains a schema definition to which the structured data in the files `chi-beijing-rt1/dns.yml` and `eng-london-rt1/dns.yml` should adhere.\n\n### Basic usage\n\nOnce schema-enforcer has been installed, the `schema-enforcer validate` command can be used run schema validations of YAML/JSON instance files against the defined schema.\n\n```shell\nbash$ schema-enforcer --help\nUsage: schema-enforcer [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  ansible        Validate the hostvar for all hosts within an Ansible...\n  schema         Manage your schemas\n  validate       Validates instance files against defined schema\n```\n\nTo run the schema validations, the command `schema-enforcer validate` can be run.\n\n```shell\nbash$ schema-enforcer validate\nschema-enforcer validate\nALL SCHEMA VALIDATION CHECKS PASSED\n```\n\nTo acquire more context regarding what files specifically passed schema validation, the `--show-pass` flag can be passed in.\n\n```shell\nbash$ schema-enforcer validate --show-pass\nPASS [FILE] ./eng-london-rt1/ntp.yml\nPASS [FILE] ./eng-london-rt1/dns.yml\nPASS [FILE] ./chi-beijing-rt1/syslog.yml\nPASS [FILE] ./chi-beijing-rt1/dns.yml\nALL SCHEMA VALIDATION CHECKS PASSED\n```\n\nIf we modify one of the addresses in the `chi-beijing-rt1/dns.yml` file so that it's value is the boolean `true` instead of an IP address string, then run the `schema-enforcer` tool, the validation will fail with an error message.\n\n```yaml\nbash$ cat chi-beijing-rt1/dns.yml\n# jsonschema: schemas/dns_servers\n---\ndns_servers:\n  - address: true\n  - address: \"10.2.2.2\"\n```\n```shell\nbash$ test-schema validate\nFAIL | [ERROR] True is not of type 'string' [FILE] ./chi-beijing-rt1/dns.yml [PROPERTY] dns_servers:0:address\nbash$ echo $?\n1\n```\n\nWhen a structured data file fails schema validation, `schema-enforcer` exits with a code of 1.\n\n### Configuration Settings\n\nSchema enforcer will work with default settings, however, a `pyproject.toml` file can be placed at the root of the path in which `schema-enforcer` is run in order to override default settings or declare configuration for more advanced features. Inside of this `pyproject.toml` file, `tool.schema_enforcer` sections can be used to declare settings for schema enforcer. Take for example the `pyproject.toml` file in example 2.\n\n```shell\nbash$ cd examples/example2 \u0026\u0026 tree -L 2\n.\n├── README.md\n├── hostvars\n│   ├── chi-beijing-rt1\n│   ├── eng-london-rt1\n│   └── ger-berlin-rt1\n├── invalid\n├── pyproject.toml\n└── schema\n    ├── definitions\n    └── schemas\n\n8 directories, 2 files\n```\n\nIn this toml file, a schema mapping is declared which tells schema enforcer which structured data files should be checked by which schema IDs.\n\n\n```shell\nbash$ cat pyproject.toml\n[tool.schema_enforcer.schema_mapping]\n# Map structured data filename to schema IDs\n'dns_v1.yml' = ['schemas/dns_servers']\n'dns_v2.yml' = ['schemas/dns_servers_v2']\n'syslog.yml' = ['schemas/syslog_servers']\n```\n\n\u003e More information on available configuration settings can be found in the [configuration README](docs/configuration.md)\n\n### Supported Formats\n\nBy default, schema enforcer installs the jsonschema `format_nongpl` extra (in version \u003c1.2.0) or `format-nongpl` (in versions \u003e=1.2.0). This extra allows the use of formats that can be used in schema definitions (e.g. ipv4, hostname...etc). The `format_nongpl` or `format-nongpl` extra only installs transitive dependencies that are not licensed under GPL. The `iri` and `iri-reference` formats are defined by the `rfc3987` transitive dependency which is licensed under GPL. As such, `iri` and `iri-reference` formats are *not* supported by `format-nongpl`/`format_nongpl`. If you have a need to use `iri` and/or `iri-reference` formats, you can do so by running the following pip command (or it's poetry equivalent):\n\n```\npip install 'jsonschema[rfc3987]'\n```\n\nSee the \"Validating Formats\" section in the [jsonschema documentation](https://github.com/python-jsonschema/jsonschema/blob/main/docs/validate.rst) for more information.\n\n### Where To Go Next\n\nDetailed documentation can be found in the README.md files inside of the `docs/` directory.\n- [\"Introducing Schema Enforcer\" blog post](https://blog.networktocode.com/post/introducing_schema_enforcer/)\n- [Using a pyproject.toml file for configuration](docs/configuration.md)\n- [Mapping Structured Data Files to Schema Files](docs/mapping_data_files_to_schemas.md)\n- [The `ansible` command](docs/ansible_command.md)\n- [The `validate` command](docs/validate_command.md)\n- [The `schema` command](docs/schema_command.md)\n- [Implementing custom validators](docs/custom_validators.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworktocode%2Fschema-enforcer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetworktocode%2Fschema-enforcer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworktocode%2Fschema-enforcer/lists"}