Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/muoncore/photon

Photon is an event store with cold+hot event streaming
https://github.com/muoncore/photon

clojure eventsourcing java microservice reactive reactive-streams

Last synced: 3 months ago
JSON representation

Photon is an event store with cold+hot event streaming

Lists

README 

        
# Photon Event Store



Photon is an event store with cold+hot event streaming.



In microservices, each service is supposed to take responsibility for a single functional capability: a computational process. In this context, these processes will follow the same pattern: gathering sequences of inputs and generating sequences of outputs though transformation functions, also called projections. Usually such functions are deeply coupled with both 1) the implementation of the service and 2) the data consumed to generate a current state and outputs. As a consequence, designs and implementations of the concepts of storage, sequence handling and data transformation have to be repeated over and over again across different services.



Photon is an attempt at avoiding such redundancy by abstracting and encapsulating both the storage and the transformation of data as service methods. It is designed as a black box with an internal database with a projection engine, both generic enough to store free-form data and allow for a wide range of expressivity for data transformation, in real time. In this way, business rules can be implemented and deployed in Photon instances via projections, effectively decoupling business logic and data storage and processing.



## Quickstart



Download the link:https://github.com/muoncore/muon-starter[Muon Starter] repository and run through the instructions and

pre-requisites from there. The latest released version of photon will be started using Docker Compose,

along with a contained RabbitMQ instance for communication and other support microservices.



Install the link:https://github.com/muoncore/muon-cli[Muon CLI] and set up with the URL `amqp://muon:microservices@localhost`



You can then see photon running



```bash

> muon d



┌──────────────────────────────┬──────────────────────────────┬──────────────────────────────┐

│ SERVICE NAME                 │ TAGS                         │ CONTENT/TYPE                 │

├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤

│ molecule                     │                              │ application/json             │

├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤

│ photon                       │ photon,eventstore            │ application/json             │

└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘

```



You can emit an event using the CLI



```bash

> muon event '{ "event-type": "SomethingHappened", "schema": "1","stream-name": "something","payload": {"message": "Hi there!"}}'

```



This will persist the event in the given stream. It can be replayed at any point later on



You can replay the persisted events using the CLI



```bash



> muon replay something



```



All the events stored will then be replayed.



Check the latest http://muoncore.io/submodules/photon/doc/index.html[photon documentation] for more information.



Check the http://muoncore.io/guide/index.html to see patterns for developing with events using Photon



## Developing Photon



### Prerequisites



You will need [Leiningen][] 2.0.0 or above installed.



https://github.com/technomancy/leiningen[leiningen]



### Running



You can run photon directly from the source:



```

lein do cljsbuild once, run

```



Or you can build an uberjar:



```

lein do cljsbuild once, uberjar

java -jar photon-*-standalone.jar

```



### Admin Console



Once running, point your browser to: `http[s]://HOST:PORT/index.html`. HOST and PORT can be configured as seen below.



.Migration of UI

****

This console will be migrated to the Molecule project once the Molecule plugin system is stable.

****



### Startup options



```

Usage: java -jar photon-x.x.x-standalone.jar [-h] [-option value] ... [-option value]

Options:

-microservice.name    : Service ID, especially important for Muon (default = photon)

-rest.host            : The IP or hostname of the web server for frontend and API. Change it for external access (default = localhost)

-rest.port            : The port for the UI frontend and the REST API

-rest.keystore        : If set, the web server will be started in SSL mode using the certificates identified by this path

-rest.keypass         : The password required to open the keystore set in rest.keystore. Not required in not-SSL mode

-admin.user           : The default username for logging in and requesting API tokens (default = admin)

-admin.pass           : The default password for logging in and requesting API tokens (default = p4010n)

-admin.secret         : A secret string that will be used to encode authentication tokens (default is random on launch)

-projections.port     : Port to stream projection updates to (default = 8375)

-events.port          : Port to stream incoming events to (default = 8376)

-muon.url             : AMQP endpoint for Muon-based transport and discovery (default = amqp://localhost)

-parallel.projections : Number of cores assigned for parallel stream processing (default = number of cores on your machine)

-projections.path     : Local folder with projections, in EDN format, to pre-load on start (default = /tmp/photon)

-db.backend           : DB backend plugin to use (default=h2). Depending on the build of photon, this can be one of:

                        h2, cassandra, redis, file, mongo, riak, dummy.

-h2.path              : If using H2, the file prefix for the database file, including path (default = /tmp/photon.h2)

-cassandra.ip         : If using Cassandra, the host of the cluster

-file.path            : If using files as backend, the absolute path to the file

-mongodb.host         : If using MongoDB, the host of the cluster

-riak.default_bucket  : If using Riak, the name of the bucket

-riak.node.X          : If using Riak, the nodes that form the cluster (riak.node.1, riak.node.2, etc.)

```



### Setting up a file for static configuration



Photon can be configured either directly from the command line or from a file, and parameters can be combined from different sources. The order of priority in which the configuration is build is the following:



1. Command-line arguments

2. photon.properties in the working directory

3. resources/photon.properties

4. resources/config.properties



Example of property file:



```

# Microservice identifier (default = photon):

microservice.name=photon

# AMQP endpoint (default = amqp://localhost):

muon.url=amqp://username:password@localhost

# Number of cores assigned for parallel stream processing

# (default = number of cores on your machine):

parallel.projections=8

# Local folder with projections, in EDN format, to pre-load on start

# (default = /tmp/photon):

file.path=/path/to/edn-files/

# DB backend plugin to use, several options currently available:

db.backend={file,mongodb,riak,cassandra}

# Depending on the backend, you'll need to set up the DB plugin:

cassandra.ip=127.0.0.1

file.path=/path/to/file.json

mongodb.host=localhost

riak.default_bucket=photon-eventstore-v1

riak.node.1=riak1.yourdomain.com

riak.node.2=riak2.yourdomain.com

riak.node.3=riak3.yourdomain.com

```



### Muon schemas



The endpoints and expected schemas to interact with `photon` can be found [here](doc/schemas.md).



### Testing



Tests are run by executing



```

lein midje

```



To run the test suite from the REPL:



```bash

lein repl

=> (use 'midje.sweet)

=> (autotest)

```



Alternatively, create a test photon client to interact with photon:



```bash

lein new muon-clojure photon-test-client

```



### Profiling



In order to achieve the best performance and thoughput, photon has been intensively tested and profiled with the [YourKit](https://www.yourkit.com) profiler, with a license kindly provided by their creators as part of their support to the open source community.



image:https://www.yourkit.com/images/yklogo.png[YourKit logo]



YourKit supports open source projects with its full-featured Java Profiler.

YourKit, LLC is the creator of YourKit Java Profiler

and YourKit .NET Profiler,

innovative and intelligent tools for profiling Java and .NET applications.