{"id":19227472,"url":"https://github.com/datastreamapp/schema","last_synced_at":"2025-04-21T01:31:36.782Z","repository":{"id":38816406,"uuid":"120513580","full_name":"datastreamapp/schema","owner":"datastreamapp","description":"DataStream Schema","archived":false,"fork":false,"pushed_at":"2025-04-15T22:36:45.000Z","size":5322,"stargazers_count":13,"open_issues_count":1,"forks_count":5,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-15T23:26:38.997Z","etag":null,"topics":["json-schema","wqx"],"latest_commit_sha":null,"homepage":"https://datastream.org","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datastreamapp.png","metadata":{"files":{"readme":"docs/README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-06T19:37:25.000Z","updated_at":"2025-04-15T22:36:49.000Z","dependencies_parsed_at":"2024-05-09T15:53:25.634Z","dependency_job_id":"b4322c41-6207-4c0b-a889-3c5daf868e5b","html_url":"https://github.com/datastreamapp/schema","commit_stats":null,"previous_names":[],"tags_count":140,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastreamapp%2Fschema","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastreamapp%2Fschema/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastreamapp%2Fschema/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastreamapp%2Fschema/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datastreamapp","download_url":"https://codeload.github.com/datastreamapp/schema/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249982511,"owners_count":21355704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json-schema","wqx"],"created_at":"2024-11-09T15:23:30.949Z","updated_at":"2025-04-21T01:31:35.215Z","avatar_url":"https://github.com/datastreamapp.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/datastreamapp/schema/main/docs/images/datastream.svg?sanitize=true\" alt=\"DataStream Logo\" width=\"400\"\u003e\n  \u003cbr/\u003e\n  DataStream Open Data Schema for Water Quality Data\n  \u003cbr/\u003e\n  \u003cbr/\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003eDownload the latest version in \u003ca href=\"http://datastream.org/schema#\" target=\"_blank\"\u003eJSON Schema\u003c/a\u003e, \u003ca href=\"https://docs.google.com/spreadsheets/d/1OwGkUTyVC3tZ9N_we8uX1kpsPejZnSsgcaTjrlGBxoc\" target=\"_blank\"\u003eGoogle Sheets\u003c/a\u003e, \u003ca href=\"https://datastreamorg.sharepoint.com/:x:/s/Datastream/EaqcNGHom7BFlRi6bRY4VDoBy6ECq6v3bbUyeb0B3S3HGg?e=75aBTl\" target=\"_blank\"\u003eExcel\u003c/a\u003e template formats.\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003c!--\u003ca href=\"https://github.com/datastreamapp/schema\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/datastreamapp/schema.svg?style=social\u0026label=Stars\" alt=\"Stars\" /\u003e\u003c/a\u003e--\u003e\n  \u003c!--\u003ca href=\"https://www.npmjs.com/package/schema\"\u003e\u003cimg src=\"https://img.shields.io/npm/v/schema.svg\" alt=\"npm version\"\u003e\u003c/a\u003e--\u003e\n  \u003c!--\u003ca href=\"https://www.npmjs.com/package/schema\"\u003e\u003cimg src=\"https://img.shields.io/npm/dm/schema.svg\" alt=\"npm downloads\"\u003e\u003c/a\u003e--\u003e\n  \u003c!--\u003ca href=\"https://www.npmjs.com/package/schema\"\u003e\u003cimg src=\"https://img.shields.io/npm/l/schema.svg\" alt=\"npm license\" /\u003e\u003c/a\u003e--\u003e\n\u003c/p\u003e\n\n## DataStream\n\nDataStream ([DataStream.org](https://datastream.org)) is an online open-access platform for sharing water quality and sediment quality data. Data is uploaded, stored and shared in DataStream’s Open Data Schema -- a model based on the WQX standard for the Exchange of Water Quality Data. DataStream is free to use and allows users to query, visualize, and download data in this standardized format. Water monitoring groups across Canada — including communities, researchers, Indigenous nations and governments at all levels — are using DataStream to publish their data.\n\n\u003c!--\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"http://gordonfoundation.ca\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/datastreamapp/schema/master/docs/images/the-gordon-foundation.svg\" alt=\"The Gordon Foundation Logo\" width=\"200\"\u003e\u003c/a\u003e\n\u003c/div\u003e--\u003e\n\nDataStream was developed by [The Gordon Foundation](https://gordonfoundation.ca) and is carried out in collaboration with regional partners and monitoring networks. Data contributors maintain ownership of their data which are published under open data licenses.\n\n## DataStream Open Data Schema (DS-WQX)\nTo ensure consistent formatting of water data and to avoid ambiguous or missing information, we developed an observation-level data schema based on the WQX standard for the Exchange of Water Quality Data. \n\nThe WQX schema was developed by the US Environmental Protection Agency (EPA) and the US Geological Society (USGS) and is an implementation of the ESAR (Environmental Sampling, Analysis and Results) data standard. It was designed to enable multiple monitoring entities to share results in a common format. In the US, the WQX schema is used on the US EPA’s Water Quality Portal to share over 340 million water quality data records data from 400 federal, state, tribal and other partners. \n\nIn 2018, The Gordon Foundation led a comprehensive review process to determine how this model could be adapted to best meet the needs of diverse water monitoring initiatives in Canada. DataStream’s science and technical advisory team, regional partners and collaborators, data contributors, government representatives as well as other members of the water community were engaged in this process. \n\nDataStream’s open data schema (DS-WQX v1.0) was fully implemented across all regional DataStream platforms in 2019. \n\n## Changelog\nThe DataStream open data schema will continue to evolve to meet user needs and therefore is subject to various updates over time (e.g. addition of new allowed values). To view the most recent version number and record of changes please see the [DataStream Upload Template](https://datastreamorg.sharepoint.com/:x:/s/Datastream/EaqcNGHom7BFlRi6bRY4VDoBy6ECq6v3bbUyeb0B3S3HGg?e=75aBTl).\n\n## Dataset Metadata\nIn addition to the observation-level information in DS-WQX, DataStream uses dataset level metadata. [DataStream Metadata](https://github.com/datastreamapp/schema/tree/main/schemas/meta)\n\n## Special Case Tests\nIn addition to our schema enforcing allowed values the column conditional logic; we have included additional check for common errors to the frontend flavour of our schema.\n\n- `Dissolved oxygen (DO)` should not be in `%`\n- `Dissolved oxygen saturation` should not be less than `0%`\n- `Hardness` should not be less than or equal to `0`\n- `pH` should be within `0` and `14`\n- `Temperature` should be within `-100 degC` and `100 degC`\n\n## Testing\nWe aim to have our test be as robust as possible. This is accomplished by having acceptance and rejection tests.\n\n## Install\nYou can download the compiled DS-WQX schema from above.\n\nAlternately, you can build it from the source to include in your project.\n\n```bash\n# Public\n$ npm i\n$ npm run build\n$ cat primary.json\n\n# Private\n$ npm i @datastreamapp/schema\n```\n\n## Use\n### CSV Template\nThe `csv` template follows `R` import/export best practices.\n\n### JavaScript\n```javascript\nimport validate from '@datastreamapp/schema'\nconst data = {}; // Single row of data\nconst valid = validate(data);\nif (!valid) console.error(validate.errors)\n```\n\n### Schema Flavours\nSupports JSON Schema Draft 2019-09 Specification in non-strict mode. Strict mode removes `if`, `then`, `additionalProperties` from the schema.\n\n- `primary`: This includes only JSON schema specification supported parameters\n- `frontend`: Includes value coercion and supplementary conditional checks\n- `backend`: Includes all possible allowed values, value coercion and no conditional logic.\n\n## Publish\n```bash\n# change version in package.json\nnpm test\ncd dist\nnpm publish\n```\n\n## Contributing\n\n### Commenting on the Schema\n\nIf you wish to comment on the schema please [open an issue](https://github.com/datastreamapp/schema/issues).\n\n* For information on opening an issue review github's [creating an issue](https://help.github.com/en/github/managing-your-work-on-github/creating-an-issue) document\n\nYou can also email us at \u003cteam@datastream.org\u003e or visit us at:\n\n* [DataStream](https://datastream.org)\n* [Mackenzie DataStream](https://mackenziedatastream.ca)\n* [Atlantic DataStream](https://atlanticdatastream.ca)\n* [Lake Winnipeg DataStream](https://lakewinnipegdatastream.ca)\n* [Great Lakes DataStream](https://greatlakesdatastream.ca)\n* [Pacific DataStream](https://pacificdatastream.ca)\n\n### Development\n```bash\nbrew install nvm\nnvm get 12\nnpm i\n```\n\n### Publishing\n```bash\n# update version in `package.json`\nnpm run test\ncd dist\nnpm publish\n```\n\n## References\n- [WQX](https://github.com/datastreamapp/wqx)\n- [R Import/Export](https://cran.r-project.org/doc/manuals/r-release/R-data.html)\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"http://gordonfoundation.ca\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/datastreamapp/schema/main/docs/images/the-gordon-foundation.svg?sanitize=true\" alt=\"The Gordon Foundation Logo\" width=\"200\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n## Citation\n\nIf you use this schema, please cite it as below.\n\nDataStream Initiative. ([Year]). DataStream WQX Schema ([current version]). GitHub. https://github.com/datastreamapp/schema\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastreamapp%2Fschema","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatastreamapp%2Fschema","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastreamapp%2Fschema/lists"}