Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/theatlantic/go-pensieve
Parse Krux data feed files and import into a Pilosa DB
https://github.com/theatlantic/go-pensieve
Last synced: 7 days ago
JSON representation
Parse Krux data feed files and import into a Pilosa DB
- Host: GitHub
- URL: https://github.com/theatlantic/go-pensieve
- Owner: theatlantic
- License: mit
- Created: 2017-08-07T17:48:47.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-08-21T15:12:29.000Z (about 6 years ago)
- Last Synced: 2024-10-11T04:34:19.350Z (about 1 month ago)
- Language: Go
- Size: 5.86 KB
- Stars: 3
- Watchers: 29
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Pensieve
This is an example project to show the feasibility of calculating Krux segment overlap with [Pilosa](https://www.pilosa.com/).
## Setup
Run these setup commands first:
```shell
> brew install pilosa golang python
> go get github.com/pilosa/go-pilosa
> git clone [email protected]:theatlantic/go-pensieve.git && cd go-pensieve
> go build pensieve.go
> pip install git+ssh://[email protected]/theatlantic/python-pilosa.git@fix-top-n
```
In a separate window, run the Pilosa server:
```shell
> pilosa server
```
Make sure all of the krux output files (all .gz parts) are in a directory (say, `./files`). Then, create the index and import all of the files:
```shell
> curl localhost:10101/index/segmentation -X POST
> ./pensieve -dir ./files
```
## Calculate Overlaps
```shell
> ./getintersections [, ...]
```