Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/wavii/darner
simple, lightweight message queue
https://github.com/wavii/darner
Last synced: 1 day ago
JSON representation
simple, lightweight message queue
- Host: GitHub
- URL: https://github.com/wavii/darner
- Owner: wavii
- License: other
- Archived: true
- Created: 2012-07-28T23:26:19.000Z (over 12 years ago)
- Default Branch: master
- Last Pushed: 2017-11-23T01:18:09.000Z (almost 7 years ago)
- Last Synced: 2024-08-02T05:06:24.730Z (3 months ago)
- Language: C++
- Homepage:
- Size: 743 KB
- Stars: 865
- Watchers: 90
- Forks: 80
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Changelog: ChangeLog
- License: LICENSE
Awesome Lists containing this project
README
# Darner
Darner is a very simple message queue server. Unlike in-memory servers such as [redis](http://redis.io/), Darner is
designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as
[RabbitMQ](http://www.rabbitmq.com/), Darner keeps all messages **out of process**, relying instead on the kernel's
virtual memory manager via [log-structured storage](https://code.google.com/p/leveldb/).
The result is a durable queue server that uses a small amount of in-resident memory regardless of queue size, while
still achieving [remarkable performance](/docs/benchmarks.md).
Darner is based on Robey Pointer's [Kestrel](/robey/kestrel) simple, distributed message queue. Like Kestrel, Darner
follows the "No talking! Shhh!" approach to distributed queues: A single Darner server has a set of queues identified
by name. Each queue is a strictly-ordered FIFO, and querying from a fleet of Darner servers provides a loosely-ordered
queue. Darner also supports Kestrel's two-phase reliable fetch: if a client disconnects before confirming it handled
a message, the message will be handed to the next client.
Compared to Kestrel, Darner boasts much higher throughput, better concurrency, an order of magnitude better tp99, and
uses an order of magnitude less memory. But Darner has less configuration, and far fewer features than Kestrel. Check
out the [benchmarks](/docs/benchmarks.md)!
Darner is used at [Wavii](http://wavii.com/), and is written and maintained by [Erik Frey](/erikfrey).
## Installing
### OSX
Darner installs on a Mac via [Homebrew](/mxcl/homebrew):
```bash
brew install darner
```
Remember to update your Homebrew install if the formula is not available.
### Ubuntu
You'll need build tools, [CMake](http://www.cmake.org/), [Boost](http://www.boost.org/), and
[LevelDB](https://code.google.com/p/leveldb/)/[snappy](https://code.google.com/p/snappy/) to build Darner. The
following works on Ubuntu 12.04:
```bash
sudo apt-get install -y build-essential cmake libboost-all-dev libsnappy-dev libleveldb-dev
```
Then fetch and install Darner:
```bash
git clone git://github.com/wavii/darner.git
cd darner
cmake . && make && sudo make install
```
## Running
Make a directory for Darner to store its queues, say `/var/spool/darner/`, then run Darner like so.
```bash
user@home:~$ darner -d /var/spool/darner/
[INFO] 2012-Aug-13 03:59:41.047739: darner: queue server
[INFO] 2012-Aug-13 03:59:41.048051: build: Aug 26 2012 (15:47:48) v0.1.2 (c) Wavii, Inc.
[INFO] 2012-Aug-13 03:59:41.048132: listening on port: 22133
[INFO] 2012-Aug-13 03:59:41.048507: data dir: /var/spool/darner/
[INFO] 2012-Aug-13 03:59:41.048798: starting up
```
Voila! By default, Darner listens on port 22133.
## Protocol
Darner follows the same protocol as [Kestrel](/robey/kestrel/blob/master/docs/guide.md#memcache), which is the memcache
protocol.
Currently missing from the Darner implementation but TODO: some stats.