https://github.com/elodina/stockpile

stockpile is a mesos executor for go_kafka_client scheduler that mirros data from Apache Kafka to Apache Cassandra
https://github.com/elodina/stockpile

Last synced: 1 day ago
JSON representation

stockpile is a mesos executor for go_kafka_client scheduler that mirros data from Apache Kafka to Apache Cassandra

• Host: GitHub
• URL: https://github.com/elodina/stockpile
• Owner: elodina
• License: apache-2.0
• Created: 2015-09-21T13:04:04.000Z (about 9 years ago)
• Default Branch: master
• Last Pushed: 2016-04-29T14:23:15.000Z (over 8 years ago)
• Last Synced: 2024-08-04T04:06:08.119Z (3 months ago)
• Language: Go
• Homepage: http://www.elodina.net
• Size: 22.9 MB
• Stars: 3
• Watchers: 12
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-mesos - Kafaka -> Cassandra mirroring

README

        
# stockpile

**stockpile** is a mesos executor for go_kafka_client scheduler that mirros data from Apache Kafka to Apache Cassandra

## Build

```bash

$ git clone [email protected]:elodina/stockpile.git

$ cd stockpile

$ export GO15VENDOREXPERIMENT=1

$ go build

```

## Prepare Cassandra

Create table with schema for cassandra producer.

## Run

To run stockpile executor, copy it's binary to go-kafka-client-mesos directory, add new consumer task using cli and start it.

Options:

```

  brokers

        Kafka broker list (default "localhost:9092")

  cassandra

        Cassandra cluster (default "localhost:9042")

  keyspace

        Cassandra keyspace (required)

  partitions

        Kafka partitions list (required)

  schema

        Schema registry URL (required)

  topics

        Kafka topic list (required)

```

Example:

```bash

$ ./cli add consumer 1 --executor=stockpile

$ ./cli update 1 --mem 64 --cpu 0.1

$ ./cli update 1 --options="topics=test;partitions=0;keyspace=test;schema=localhost:8081"

$ ./cli start 1

```