https://github.com/sunng87/slacker-cluster
Clojure Micro-Service framework based on Slacker RPC
https://github.com/sunng87/slacker-cluster
Last synced: 5 months ago
JSON representation
Clojure Micro-Service framework based on Slacker RPC
- Host: GitHub
- URL: https://github.com/sunng87/slacker-cluster
- Owner: sunng87
- License: epl-1.0
- Created: 2012-04-13T13:03:51.000Z (almost 14 years ago)
- Default Branch: master
- Last Pushed: 2019-03-15T15:50:47.000Z (almost 7 years ago)
- Last Synced: 2025-07-05T00:30:57.216Z (6 months ago)
- Language: Clojure
- Homepage:
- Size: 322 KB
- Stars: 77
- Watchers: 8
- Forks: 4
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Slacker Cluster
This is the cluster module for
[slacker](https://github.com/sunng87/slacker), a clojure RPC
framework.
[](https://travis-ci.org/sunng87/slacker-cluster)
[](https://clojars.org/slacker/slacker-cluster)
[](https://github.com/sunng87/slacker-cluster/blob/master/LICENSE)
Base on service discovery by ZooKeeper, slacker has a solution for
high availability and load balancing. You can have several slacker
servers in a cluster serving namespaces of functions. The
cluster-enabled slacker client will choose one (or several, based on your
grouping function) of these servers to call. Once a server is added
to or removed from the cluster, the client will receive a notification
from zookeeper and establish/destroy the connection to the server.
To create such a slacker cluster, you have to deploy at least one zookeeper
instance in your topology.
## Leiningen
## Cluster Enabled Slacker Server
On the server side, using the server starter function from
`slacker.server.cluster`, add an option `:cluster` and provide your
cluster information.
``` clojure
(use 'slacker.server.cluster)
(start-slacker-server ...
:cluster {:name "cluster-name"
:zk "127.0.0.1:2181"})
```
Cluster information here:
* `:name` the cluster name (*required*)
* `:zk` zookeeper server address (*required*)
* `:node` server IP (*optional*, if you don't provide the server IP
here, slacker will try to detect server IP by connecting to zk,
on which we assume that your slacker server are bound on the same
network with zookeeper)
* `:server-data` allows you to set some small amount, server specific
data that will be available to client grouping function to choose
server.
## Cluster Enabled Slacker Client
On the client side, you have to specify the zookeeper address instead
of a particular slacker server. Use the `clustered-slackerc`:
``` clojure
(use 'slacker.client.cluster)
(def sc (clustered-slackerc "cluster-name" "127.0.0.1:2181"))
(use-remote 'sc 'slapi)
```
*Important*: You should make sure to use the `use-remote` and
`defn-remote` from `slacker.client.cluster` instead of
`slacker.client`.
## Client Grouping
New in 0.10.1. By default, slacker cluster client randomly pick a
server to run your invocation. But at some situation, you may like to
override this behavior, for example, run a function on all servers.
The grouping option gives you total control of this behavior.
```clojure
(defn my-grouping [ns-name fn-name params slacker-client servers]
...)
(clustered-slackerc "cluster-name" "127.0.0.1:2181" :grouping my-grouping)
```
The first three arguments represent your current invocation. And you
can return one or more servers based on the information and your
business logic. Server specific data is available from `(server-data
client "server-addr")`.
You can also return constant value:
* `:all` call on all servers available
* `:random` pick a random server to call
* `:first` always call the first server in client cached server list
* `:leader` always pick the leader node in cluster (new in 0.12)
* `:least-in-flight` call the server with least pending requests
### Grouping results
When you are using :grouping function, you might get multiple return
values from different servers. A new option `:grouping-results` is to
define how many values to return for the call. Possible values are:
* `:single` default, only return a single value, behavior same as the
function call
* `:vector` returns values from different servers as a vector
* `:map` returns values from different servers as a map, with server's
host:port as key
* `:nil` always ignore the result and return nil
* `(fn [ns-name fn-name params])` a function that returns values
above. You can use different grouping-results policy according to
function you call.
Note that `:vector` and `:map` would break default behavior of the
function call, and change return type.
### Grouping exceptions
You can decide whether the client will throw an exception when some of
remote functions throwing exceptions.
* `:all` default, only throw exception when all remote functions return error
* `:any` will throw exception when any remote server caused an error
If you choose `:all` but some of servers failed to return, we will
only return results from servers succeeded.
### Grouping at function level
You can also define `:grouping` and `:grouping-results` function at
`defn-remote` level.
```clojure
(defn-remote sc slacker.example.api/timestamp
:grouping :all
:grouping-results :single)
```
If there is no server in the cluster provides the function, you can
add `:unavailable-value` option on `defn-remote` for return value on
this situation.
## Server Manager
When the server started with option `:manager true`, a management
API is exposed as a namespace: `slacker.cluster.manager`.
There are functions under this namespace:
* `(offline)` put this server to offline mode then no more requests will
be sent to it. The server is still running and you can call it from
address.
* `(offline-ns nsname)` put a namespace on this server to offline
mode.
* `(online)` put the server to online mode.
* `(online-ns nsname)` put a namespace on this server to online mode.
* `(set-server-data! data)` set the data associated with this server.
* `(server-data)` get the data with this server.
* `(shutdown)` shutdown the server completely.
## Examples
There is a cluster example in the source code. To run the server,
start a zookeeper on your machine (127.0.0.1:2181). We have an example
zookeeper server you can start with:
lein run-example-zk
Start server instance:
lein run-example-server 2104
Open a new terminal, start another server instance:
lein run-example-server 2105
On another terminal, you can run the example client:
lein run-example-client
By checking logs, you can trace the calls on each server instance.
## API doc
[API doc](https://sunng87.github.io/slacker-cluster) is generated
against latest master branch, with lein-codox with Travis CI.
## License
Copyright (C) 2011-2016 Sun Ning
Distributed under the Eclipse Public License, the same as Clojure.