{"id":22353513,"url":"https://github.com/dp6/penguin-datalayer-core","last_synced_at":"2025-07-26T16:39:38.015Z","repository":{"id":37965549,"uuid":"342373198","full_name":"DP6/penguin-datalayer-core","owner":"DP6","description":"Validation core engine for the data layer of the Raft Suite ecosystem.","archived":false,"fork":false,"pushed_at":"2023-02-27T12:58:54.000Z","size":400,"stargazers_count":6,"open_issues_count":14,"forks_count":3,"subscribers_count":23,"default_branch":"master","last_synced_at":"2024-11-28T23:32:29.558Z","etag":null,"topics":["camada-de-dados","data-quality","data-quality-checks","datalayer","dp6","gtm","hacktoberfest","json-schema","marketing-automation","penguin-datalayer"],"latest_commit_sha":null,"homepage":"https://dp6.github.io/penguin-datalayer-core","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DP6.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-25T20:40:43.000Z","updated_at":"2022-03-14T13:32:07.000Z","dependencies_parsed_at":"2023-02-16T07:15:22.248Z","dependency_job_id":null,"html_url":"https://github.com/DP6/penguin-datalayer-core","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DP6%2Fpenguin-datalayer-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DP6%2Fpenguin-datalayer-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DP6%2Fpenguin-datalayer-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DP6%2Fpenguin-datalayer-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DP6","download_url":"https://codeload.github.com/DP6/penguin-datalayer-core/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228114910,"owners_count":17871742,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["camada-de-dados","data-quality","data-quality-checks","datalayer","dp6","gtm","hacktoberfest","json-schema","marketing-automation","penguin-datalayer"],"created_at":"2024-12-04T13:08:50.785Z","updated_at":"2024-12-04T13:08:51.724Z","avatar_url":"https://github.com/DP6.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# penguin-datalayer-core\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/DP6/templates-centro-de-inovacoes/main/public/images/centro_de_inovacao_dp6.png\" height=\"100px\" /\u003e\n\n\u003c/div\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#badge\"\u003e\n    \u003cimg alt=\"semantic-release\" src=\"https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@dp6/penguin-datalayer-core\"\u003e\n    \u003cimg alt=\"npm latest version\" src=\"https://img.shields.io/npm/v/@dp6/penguin-datalayer-core/latest.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.codacy.com/gh/DP6/penguin-datalayer-core/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=DP6/penguin-datalayer-core\u0026amp;utm_campaign=Badge_Coverage\"\u003e\u003cimg alt=\"Coverage\" src=\"https://app.codacy.com/project/badge/Coverage/ac10d2fd82a0471889b151b14e560f20\"/\u003e\u003c/a\u003e\n  \u003ca href=\"#badge\"\u003e\n    \u003cimg alt=\"Test\" src=\"https://github.com/dp6/penguin-datalayer-core/actions/workflows/test.yml/badge.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.codacy.com/gh/DP6/penguin-datalayer-core/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=DP6/penguin-datalayer-core\u0026amp;utm_campaign=Badge_Grade\"\u003e\n    \u003cimg alt=\"Code Quality\" src=\"https://app.codacy.com/project/badge/Grade/ac10d2fd82a0471889b151b14e560f20\"/\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n### Available Languages\n\n- [Read this page in English](https://github.com/DP6/penguin-datalayer-core/blob/master/README.md)\n- [Leia esta página em Português](https://github.com/DP6/penguin-datalayer-core/blob/master/README_pt-BR.md)\n\n---\n\nThe penguin-datalayer-core is the validation engine for the penguin-datalayer modules that belongs to raft-suite ecosystem created by DP6 in order to guarantee the [Data Quality](https://en.wikipedia.org/wiki/Data_quality) of the Data Engineering projects implemented on our clients, which is achieved through monitoring and automated data pipelines to guarantee credibility, consistency and availability in the planning, collection and data maintenance steps.\n\n## 1. Requirements for usage\n\n### 1.1 JSON Schema\n\nThe JSON Schema is a structure that allow the **validation** of JSON documents. This structure is used in the project because it allows the declaration of expected data formats in the data layer.\n\n#### Supported Data Types\n\nThe following Data Types are currently supported within this module:\n\n- String\n- Number\n- Boolean\n- Object\n- Array\n\n#### Validation rules\n\nThe following validation rules are accepted:\n\n- **Enum (Equals)**: Should be used to validate the **parity** between the schema value _versus_ the value sent to the data layer;\n- **Pattern (Regex - String)**: Create regular expressions to validate the values of each key;\n- **minItems (Array)**: Validates the minimum itens contained in an array;\n- **Required**: Should be used when a given key is required in the data layer\n\n#### JSON Schema Structure\n\nThe following structure is a JSON Schema example:\n\n```json\n{\n  \"$schema\": \"\",\n  \"title\": \"Schema example\",\n  \"array\": {\n    \"$id\": \"#/properties/schema\",\n    \"type\": \"array\",\n    \"items\": [\n      {\n        \"type\": \"object\",\n        \"properties\": {\n          \"event\": {\n            \"type\": \"string\",\n            \"enum\": [\"test\"]\n          },\n          \"key1\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"key1_sub1\": {\n                \"type\": \"number\",\n                \"enum\": [10]\n              },\n              \"key1_sub2\": {\n                \"type\": \"string\",\n                \"pattern\": \"teste|test|.*\"\n              },\n              \"key1_sub3\": {\n                \"type\": \"string\",\n                \"enum\": [\"production\"]\n              },\n              \"key1_sub4\": {\n                \"type\": \"boolean\",\n                \"enum\": \"desktop|mobile|msite\"\n              }\n            },\n            \"required\": [\"key1_sub1\", \"key1_sub2\", \"key1_sub3\", \"key1_sub4\"]\n          }\n        },\n        \"required\": [\"event\"]\n      }\n    ]\n  }\n}\n```\n\n### 1.2 Ludwig (schema generator)\n\nDP6 owns an automation that generates a structured validation schema.\n\n### Api Docs\n\n- [Index.js](https://github.com/dp6/penguin-datalayer-core/blob/master/docs/index.md)\n- [datalayer-validation-core.js](https://github.com/dp6/penguin-datalayer-core/blob/master/docs/atalayer-validation-core.md)\n- [schema-parser.js](https://github.com/dp6/penguin-datalayer-core/blob/master/docs/atalayer-validation-core.md)\n\n## How to contribute\n\nPull requests welcome! We would love some help to evolve this module. Feel free to search for _open issues_. If there's a new _feature_ or _bug_, please open a new _issue_, so our team can follow up.\n\n### Prerequirements\n\nIt will only be accepted contributions that follows the below requirements:\n\n- [Padrão de commit](https://www.conventionalcommits.org/en/v1.0.0/)\n\n## Support:\n\n**DP6 Koopa-troopa Team**\n\n_e-mail: \u003cmailto:koopas@dp6.com.br\u003e_\n\n\u003cimg src=\"https://raw.githubusercontent.com/DP6/templates-centro-de-inovacoes/main/public/images/koopa.png\" height=\"100\" /\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdp6%2Fpenguin-datalayer-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdp6%2Fpenguin-datalayer-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdp6%2Fpenguin-datalayer-core/lists"}