https://github.com/frictionlessdata/datapackage-pipelines-elasticsearch
Datapackage-pipelines integration with Elasticsearch
https://github.com/frictionlessdata/datapackage-pipelines-elasticsearch
Last synced: 7 months ago
JSON representation
Datapackage-pipelines integration with Elasticsearch
- Host: GitHub
- URL: https://github.com/frictionlessdata/datapackage-pipelines-elasticsearch
- Owner: frictionlessdata
- License: mit
- Created: 2017-08-03T20:50:00.000Z (over 8 years ago)
- Default Branch: main
- Last Pushed: 2021-06-01T12:55:10.000Z (almost 5 years ago)
- Last Synced: 2025-08-30T15:42:48.603Z (8 months ago)
- Language: Python
- Homepage:
- Size: 16.6 KB
- Stars: 4
- Watchers: 6
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# ElasticSearch Extensions for datapackage-pipelines
## Install
```
# use pip install
pip install datapackage-pipelines-elasticsearch
# OR clone the repo and install it with pip
git clone https://github.com/frictionlessdata/datapackage-pipelines-elasticsearch.git
pip install -e .
```
## Usage
You can use datapackage-pipelines-elasticsearch as a plugin for (dpp)[https://github.com/frictionlessdata/datapackage-pipelines#datapackage-pipelines]. In pipeline-spec.yaml it will look like this
```yaml
...
- run: elasticseach.dump.to_index
```
### ***`dump.to_index`***
Saves the datapackage to an ElasticSearch instance.
_Parameters_:
- `engine` - Connection string for connecting to the ElasticSearch instance (URL syntax)
Also supports `env://`, which indicates that the connection string should be fetched from the indicated environment variable.
If not specified, assumes a default of `env://DPP_ELASTICSEARCH`
Environment variable should take the form of 'host:port' or a fully-qualified url (e.g. 'https://user:pass@host:port' or 'https://host:port' etc.)
- `indexes` - Mapping between resources and indexes. Keys are index names, value is a list of objects with the following attributes:
- `resource-name` - name of the resource that should be dumped to the table
- `doc-type` - The document type to use when indexing docuemtns