https://github.com/cjsaylor/datamnom
Generic data ingestion for Elasticsearch to be visualized by Kibana.
https://github.com/cjsaylor/datamnom
elasticsearch ingestion kibana visualizations
Last synced: about 1 month ago
JSON representation
Generic data ingestion for Elasticsearch to be visualized by Kibana.
- Host: GitHub
- URL: https://github.com/cjsaylor/datamnom
- Owner: cjsaylor
- Created: 2015-06-04T18:00:42.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2021-07-29T10:14:34.000Z (over 3 years ago)
- Last Synced: 2025-01-22T10:36:30.030Z (3 months ago)
- Topics: elasticsearch, ingestion, kibana, visualizations
- Language: JavaScript
- Homepage:
- Size: 460 KB
- Stars: 3
- Watchers: 6
- Forks: 1
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Datamnom
tl;dr: Generic data ingestion for [Elasticsearch](https://www.elastic.co/products/elasticsearch) to be visualized by [Kibana](https://www.elastic.co/products/kibana).
Originally created for a local [Hack for Change event](https://medium.com/zumba-tech/zumbatech-takes-on-hackforchange-f8e8ebdc14d7#.2vfjxtk03), this project aims to get quick visualizations via Kibana that look like this:
> Dashboard above produced in Kibana from an import of [Florida Vendor data](https://github.com/cjsaylor/datamnom/wiki/Florida-Vendor-Data) from 2014 that was open sourced by FL state government.
## Why this method, why not make an API
Elasticsearch serves as both a generic search tool, but it also functions as an API through
Elasticsearch's rest interface while simultaneously supplying an aggregation and visualization framework
through Kibana (among other tools).
## Requirements
* NodeJS `10+`
## Install
* `docker-compose up -d`
* `npm install`
## Usage
1. `docker-compose up -d` (if you don't already have elasticsearch running locally)
2. Create a configuration file (see [examples](examples/)).
3. Run `node bin/datamnom.js --config=/path/to/config/from/step/1 --files=/glob/path/to/files`
4. Open http://localhost:5601 to see your data imported