Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/aredee/accumulo-mesos

Last synced: 3 months ago
JSON representation

Host: GitHub
URL: https://github.com/aredee/accumulo-mesos
Owner: aredee
License: apache-2.0
Created: 2015-04-08T21:10:36.000Z (about 9 years ago)
Default Branch: master
Last Pushed: 2015-10-27T13:17:25.000Z (over 8 years ago)
Last Synced: 2024-01-18T18:31:00.992Z (5 months ago)
Language: JavaScript
Size: 1.48 MB
Stars: 17
Watchers: 11
Forks: 2
Open Issues: 4
Metadata Files:
- Readme: README.md
- License: LICENSE.txt

Lists

awesome-mesos - Apache Accumulo

README

Accumulo Mesos Framework
=========================
Initialize and run Accumulo clusters as a Mesos framework.

------------

**DISCLAIMER**
_This is a very early version of accumulo-mesos framework. This
document, code behavior, and anything else may change without notice and/or break older installations._

------------

# Design
The accumulo-mesos framework launches accumulo server processes on mesos client machines using
the `$ACCUMULO_HOME/bin/accumulo ` script. It automatically configures Java and Accumulo
memory settings based on the mesos offer. The framework doesn't depend on Accumulo directly so
it should be able to support the more recent versions of Accumulo. It is tested with 1.7.0

# Current Status

### Implemented
* Framework no longer depends on accumulo! Accumulo is uploaded to HDFS. There are assumptions about
where somethings will be within the accumulo tarball when extracted, but this has been stable.
* Accumulo init is separate step from running the framework. Currently this requires a local copy of
Accumulo somewhere.

### Near Term Tasks
* Docker?
* Run accumulo init from a mesos client just like the accumulo servers.
* Flesh out the framework webservice.
* Reconnect framework to a running cluster.

# Running the Framework
First you have to upload artifacts to HDFS (accumulo tarball, framework tarball, native libs .so)
and copy that into your config JSON structures. (see `dev/config`)

Running the framework is then a two step process. First you must initialize the framework.
See `dev/init_framework.sh` Then you can run the framework see `dev\start_framework.sh`

```
java -jar /vagrant/dev/dist/accumulo-mesos-dist-0.2.0-SNAPSHOT/accumulo-mesos-framework-0.2.0-SNAPSHOT-jar-with-dependencies.jar -h
usage: accumulo-mesos [-b ] [-cc ] [-fc ] [-h] [-i] [-m
] [-n ] [-P ] [-t ] [-v] [-z ]
-b,--bind-address IP address of interface to bind HTTP interface
to
-cc,--cluster JSON file containing cluster configuration
-fc,--framework JSON file of entire framework configuration
-h,--help Print this message and exit
-i,--init If present, initialize new Accumulo instance
-m,--master Location of mesos master to connect to
-n,--name Name of this mesos framework
-P,--port Port number to serve HTTP interface
-t,--tarball URI of framework tarball
-v,--version Show version number
-z,--zookeepers List of Zookeeper servers
```

## Configuration
See config examples in `dev/config`

## Testing
A multi-vm Vagrantfile is provided along with many provisioning scripts to setup
the VMs for testing the framework. See `/dev` directory for more info.

# Thanks
Thanks to the cassandra-mesos project. I stole a lot of the project setup and framework design ideas from there.