https://github.com/txn2/xn2
pull, synchronize and package data on interval.
https://github.com/txn2/xn2
Last synced: 11 months ago
JSON representation
pull, synchronize and package data on interval.
- Host: GitHub
- URL: https://github.com/txn2/xn2
- Owner: txn2
- License: apache-2.0
- Created: 2018-11-13T00:08:14.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-06-06T21:22:27.000Z (about 7 years ago)
- Last Synced: 2025-06-02T07:13:25.033Z (about 1 year ago)
- Language: Go
- Homepage:
- Size: 2.73 MB
- Stars: 5
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# xn2
**WIP**: Pull, synchronize and package data on interval. Xn2 is a data scraper / muxer, combining data from multiple URLs defined by a set.
See the example configuration used for testing with `docker-compose up`: [simple.yml](./config/xn2/simple.yml)
Build and test **xn2** with `docker-compose up`. `docker-compose` starts a fake api server and configures **xn2** poll metrics from it. **xn2** and all other services are monitored with **prometheus**.
### Demo / Tutorial
Demonstrating **Xn2** with `docker-compose up` will stand up a fairly common data pipeline with three options for visualizing collected data:
- **Grafana**: UI exposed at: http://localhost:3000
- **Kibana**: UI exposed at: http://localhost:5601
- **Prometheus**: Graph UI exposed at: http://localhost:9090/graph
Observe raw metrics collected by Xn2 at: http://localhost:8888/metrics
### Development
Run demo from Xn2 source code:
```bash
docker-compose up --build
```
Run source with local config and no destination:
```bash
go run ./cmd/xn2.go --port 8282 --config ./config/xn2/local.yml
```
While running check **prometheus** metrics at https://localhost:8282/metrics
### Release Management
Test release:
```bash
goreleaser --skip-publish --rm-dist --skip-validate
```
Release
```bash
GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist
```