Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/frictionlessdata/example-continuous-data-integration
[DEPRECATED] Please use https://goodtables.io
https://github.com/frictionlessdata/example-continuous-data-integration
Last synced: 5 days ago
JSON representation
[DEPRECATED] Please use https://goodtables.io
- Host: GitHub
- URL: https://github.com/frictionlessdata/example-continuous-data-integration
- Owner: frictionlessdata
- Archived: true
- Created: 2016-05-01T14:58:42.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2016-09-15T22:18:25.000Z (about 8 years ago)
- Last Synced: 2024-08-02T12:47:57.118Z (3 months ago)
- Language: Python
- Homepage:
- Size: 7.81 KB
- Stars: 12
- Watchers: 5
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-starred - frictionlessdata/example-continuous-data-integration - [DEPRECATED] Please use https://goodtables.io (others)
README
Continuous Data Integration!
[](https://travis-ci.org/frictionlessdata/example-continuous-data-integration)
It goes like this:
1. You have a data package - like the one here.
2. You install a small script for testing the data package - see `scripts/test_data.py`
3. Configure travis CI - see `.travis.yml`
4. Hey presto - your data will be checked on every commit or pull request
## Examples
Here's some example of actual builds:
* TODO
We also list the sample outputs below. You can configure error output to be in JSON
or CSV rather than TXT.
### Bad Structure
This "bad" CSV (won't get any dinner):
```
Date,Country,Number,
2015-01-01,3,20.3
2015-02-01,United States,23.5,,
2015-02,United States,x23.5,,,
,,
```
Results in:
```
+----------------+---------------+-------------+-------------------------------------------------------+
| result_name | result_id | row_index | result_message |
+================+===============+=============+=======================================================+
| Missing Header | structure_001 | 0 | Headers column is empty. |
+----------------+---------------+-------------+-------------------------------------------------------+
| Defective Row | structure_003 | 0 | The row dimensions are incorrect compared to headers. |
+----------------+---------------+-------------+-------------------------------------------------------+
| Defective Row | structure_003 | 1 | The row dimensions are incorrect compared to headers. |
+----------------+---------------+-------------+-------------------------------------------------------+
| Defective Row | structure_003 | 2 | The row dimensions are incorrect compared to headers. |
+----------------+---------------+-------------+-------------------------------------------------------+
| Empty Row | structure_005 | 3 | Row is empty. |
+----------------+---------------+-------------+-------------------------------------------------------+
| Defective Row | structure_003 | 3 | The row dimensions are incorrect compared to headers. |
+----------------+---------------+-------------+-------------------------------------------------------+
```
### Bad Schema
This "bad" CSV (must be punished):
```
Date,Country,Number
2015-01-01,3,20.3
2015-02-01,United States,23.5
2015-02,United States,x23.5
```
Results in:
```
+----------------+----------------+-------------+-------------------------------------------------------------+
| result_name | column_index | row_index | result_message |
+================+================+=============+=============================================================+
| Incorrect Type | 0 | 2 | The value "2015-02" in column "Date" is not a valid Date. |
+----------------+----------------+-------------+-------------------------------------------------------------+
| Incorrect Type | 2 | 2 | The value "x23.5" in column "Number" is not a valid Number. |
+----------------+----------------+-------------+-------------------------------------------------------------+
```