https://github.com/metov/insight-silverbullet

SilverBullet provides sub-second latency, live optimization of cryptocoin portfolios with distributed architecture.
https://github.com/metov/insight-silverbullet

Last synced: 14 days ago
JSON representation

SilverBullet provides sub-second latency, live optimization of cryptocoin portfolios with distributed architecture.

• Host: GitHub
• URL: https://github.com/metov/insight-silverbullet
• Owner: metov
• License: mit
• Created: 2019-06-07T04:41:11.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2019-07-10T06:14:48.000Z (over 5 years ago)
• Last Synced: 2024-10-04T21:16:17.530Z (about 1 month ago)
• Language: Python
• Homepage:
• Size: 962 KB
• Stars: 1
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# SilverBullet

Take your best shot at crypto markets!

SilverBullet live optimization of cryptocurrency portfolio with very low latency, suited to high-frequency trading applications.

## Problem statement

Split second decisions can mean the difference between profit and ruin in modern markets. To fully exploit the economic potential of the market, a trading system must be able to evaluate market conditions with extremely high latency. If it takes too long to reach a decision, the situation will have changed and even a great decision will lose its value. I wanted to create an architecture that enables such low latency for live analysis of financial price data.

## General solution

Live tick data is read from a financial data provider (currently simulated with a Kafka producer) and ingested with Kafka. A Kafka consumer calculates per-asset summary statistics over a moving time window. Heavy computations, such as the Monte Carlo optimizer, run on Spark and can be readily scaled. Computation results are stored in Cassandra, and provided to the user with Dash (visualizations) and Flask (REST API).

![](stack.png)

## Setup

* Set up Kafka as described in [kafka-setup](setup/kafka/kafka-setup.md)

	* Kafka is a distributed, fault tolerant message queue. Kafka will ingest the price data into a message queue and make it available to other programs.

* Set up Spark as described in [spark-setup](setup/spark/spark-setup.md)

	* Apache Spark is a distributed processing framework. Spark will do the distributed computations.

	* Ensure that `exchange_simulator.py`, `asset_digest.py` and `portfolio_evaluate_spark.py` exist on the Spark master along with their dependencies (`exchange_simulator.py` requires the tick data to be present as well). Fill in the correct IPs and other details in .conf files. Run `exchange_simulator.py` and `asset_digest.py` directly, and `portfolio_evaluate_spark.py` with `spark-submit`.

* Set up Cassandra as described in [cassandra-setup](setup/cassandra/cassandra-setup.md)

	* Cassandra is a distributed database. It will store computation results.

* Set up a Web UI node as described in [webui-setup](setup/webui/webui-setup.md)

The Web UI should now be available on the Web UI node.