Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/exchange-core/exchange-core
Ultra-fast matching engine written in Java based on LMAX Disruptor, Eclipse Collections, Real Logic Agrona, OpenHFT, LZ4 Java, and Adaptive Radix Trees.
https://github.com/exchange-core/exchange-core
bitcoin cryptocurrency cryptocurrency-exchanges exchange exchange-api forex-trading hft hft-trading java lmax-disruptor lock-free low-latency matching-algorithm matching-engine order-book stock-exchange stock-market stock-trading trading-api trading-platform
Last synced: about 1 month ago
JSON representation
Ultra-fast matching engine written in Java based on LMAX Disruptor, Eclipse Collections, Real Logic Agrona, OpenHFT, LZ4 Java, and Adaptive Radix Trees.
- Host: GitHub
- URL: https://github.com/exchange-core/exchange-core
- Owner: exchange-core
- License: apache-2.0
- Created: 2018-08-05T18:25:16.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2023-10-08T18:06:59.000Z (11 months ago)
- Last Synced: 2024-07-29T01:18:38.777Z (about 1 month ago)
- Topics: bitcoin, cryptocurrency, cryptocurrency-exchanges, exchange, exchange-api, forex-trading, hft, hft-trading, java, lmax-disruptor, lock-free, low-latency, matching-algorithm, matching-engine, order-book, stock-exchange, stock-market, stock-trading, trading-api, trading-platform
- Language: Java
- Homepage:
- Size: 1.54 MB
- Stars: 2,028
- Watchers: 111
- Forks: 791
- Open Issues: 61
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# exchange-core
[](https://travis-ci.org/mzheravin/exchange-core)
[](https://www.javadoc.io/doc/exchange.core2/exchange-core)
[](https://lgtm.com/projects/g/mzheravin/exchange-core/context:java)
[![][license img]][license]
Exchange-core is an **open source market exchange core** based on
[LMAX Disruptor](https://github.com/LMAX-Exchange/disruptor),
[Eclipse Collections](https://www.eclipse.org/collections/) (ex. Goldman Sachs GS Collections),
[Real Logic Agrona](https://github.com/real-logic/agrona),
[OpenHFT Chronicle-Wire](https://github.com/OpenHFT/Chronicle-Wire),
[LZ4 Java](https://github.com/lz4/lz4-java),
and [Adaptive Radix Trees](https://db.in.tum.de/~leis/papers/ART.pdf).
Exchange-core includes:
- orders matching engine
- risk control and accounting module
- disk journaling and snapshots module
- trading, admin and reports API
Designed for high scalability and pauseless 24/7 operation under high-load conditions and providing low-latency responses:
- 3M users having 10M accounts in total
- 100K order books (symbols) having 4M pending orders in total
- less than 1ms worst wire-to-wire target latency for 1M+ operations per second throughput
- 150ns per matching for large market orders
Single order book configuration is capable to process 5M operations per second on 10-years old hardware (Intel® Xeon® X5690) with moderate latency degradation:
|rate|50.0%|90.0%|95.0%|99.0%|99.9%|99.99%|worst|
|----|-----|-----|-----|-----|-----|------|-----|
|125K|0.6µs|0.9µs|1.0µs|1.4µs|4µs |24µs |41µs |
|250K|0.6µs|0.9µs|1.0µs|1.4µs|9µs |27µs |41µs |
|500K|0.6µs|0.9µs|1.0µs|1.6µs|14µs |29µs |42µs |
| 1M|0.5µs|0.9µs|1.2µs|4µs |22µs |31µs |45µs |
| 2M|0.5µs|1.2µs|3.9µs|10µs |30µs |39µs |60µs |
| 3M|0.7µs|3.6µs|6.2µs|15µs |36µs |45µs |60µs |
| 4M|1.0µs|6.0µs|9µs |25µs |45µs |55µs |70µs |
| 5M|1.5µs|9.5µs|16µs |42µs |150µs|170µs |190µs|
| 6M|5µs |30µs |45µs |300µs|500µs|520µs |540µs|
| 7M|60µs |1.3ms|1.5ms|1.8ms|1.9ms|1.9ms |1.9ms|
Benchmark configuration:
- Single symbol order book.
- 3,000,000 inbound messages are distributed as follows: 9% GTC orders, 3% IOC orders, 6% cancel commands, 82% move commands. About 6% of all messages are triggering one or more trades.
- 1,000 active user accounts.
- In average ~1,000 limit orders are active, placed in ~750 different price slots.
- Latency results are only for risk processing and orders matching. Other stuff like network interface latency, IPC, journaling is not included.
- Test data is not bursty, meaning constant interval between commands (0.2~8µs depending on target throughput).
- BBO prices are not changing significantly throughout the test. No avalanche orders.
- No coordinated omission effect for latency benchmark. Any processing delay affects measurements for next following messages.
- GC is triggered prior/after running every benchmark cycle (3,000,000 messages).
- RHEL 7.5, network-latency tuned-adm profile, dual X5690 6 cores 3.47GHz, one socket isolated and tickless, spectre/meltdown protection disabled.
- Java version 8u192, newer Java 8 versions can have a [performance bug](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8221355)
### Features
- HFT optimized. Priority is a limit-order-move operation mean latency (currently ~0.5µs). Cancel operation takes ~0.7µs, placing new order ~1.0µs;
- In-memory working state for accounting data and order books.
- Event-sourcing - disk journaling and journal replay support, state snapshots (serialization) and restore operations, LZ4 compression.
- Lock-free and contention-free orders matching and risk control algorithms.
- No floating-point arithmetic, no loss of significance is possible.
- Matching engine and risk control operations are atomic and deterministic.
- Pipelined multi-core processing (based on LMAX Disruptor): each CPU core is responsible for certain processing stage, user accounts shard, or symbol order books shard.
- Two different risk processing modes (specified per symbol): direct-exchange and margin-trade.
- Maker/taker fees (defined in quote currency units).
- Two order books implementations: simple implementation ("Naive") and performance implementation ("Direct").
- Order types: Immediate-or-Cancel (IOC), Good-till-Cancel (GTC), Fill-or-Kill Budget (FOK-B)
- Testing - unit-tests, integration tests, stress tests, integrity/consistency tests.
- Low GC pressure, objects pooling, single ring-buffer.
- Threads affinity (requires JNA).
- User suspend/resume operation (reduces memory consumption).
- Core reports API (user balances, open interest).
### Installation
1. Install library into your Maven's local repository by running `mvn install`
2. Add the following Maven dependency to your project's `pom.xml`:
```
exchange.core2
exchange-core
0.5.3
```
Alternatively, you can clone this repository and run the [example test](https://github.com/mzheravin/exchange-core/tree/master/src/test/java/exchange/core2/tests/examples/ITCoreExample.java).
### Usage examples
Create and start empty exchange core:
```java
// simple async events handler
SimpleEventsProcessor eventsProcessor = new SimpleEventsProcessor(new IEventsHandler() {
@Override
public void tradeEvent(TradeEvent tradeEvent) {
System.out.println("Trade event: " + tradeEvent);
}
@Override
public void reduceEvent(ReduceEvent reduceEvent) {
System.out.println("Reduce event: " + reduceEvent);
}
@Override
public void rejectEvent(RejectEvent rejectEvent) {
System.out.println("Reject event: " + rejectEvent);
}
@Override
public void commandResult(ApiCommandResult commandResult) {
System.out.println("Command result: " + commandResult);
}
@Override
public void orderBook(OrderBook orderBook) {
System.out.println("OrderBook event: " + orderBook);
}
});
// default exchange configuration
ExchangeConfiguration conf = ExchangeConfiguration.defaultBuilder().build();
// no serialization
Supplier serializationProcessorFactory = () -> DummySerializationProcessor.INSTANCE;
// build exchange core
ExchangeCore exchangeCore = ExchangeCore.builder()
.resultsConsumer(eventsProcessor)
.serializationProcessorFactory(serializationProcessorFactory)
.exchangeConfiguration(conf)
.build();
// start up disruptor threads
exchangeCore.startup();
// get exchange API for publishing commands
ExchangeApi api = exchangeCore.getApi();
```
Create new symbol:
```java
// currency code constants
final int currencyCodeXbt = 11;
final int currencyCodeLtc = 15;
// symbol constants
final int symbolXbtLtc = 241;
// create symbol specification and publish it
CoreSymbolSpecification symbolSpecXbtLtc = CoreSymbolSpecification.builder()
.symbolId(symbolXbtLtc) // symbol id
.type(SymbolType.CURRENCY_EXCHANGE_PAIR)
.baseCurrency(currencyCodeXbt) // base = satoshi (1E-8)
.quoteCurrency(currencyCodeLtc) // quote = litoshi (1E-8)
.baseScaleK(1_000_000L) // 1 lot = 1M satoshi (0.01 BTC)
.quoteScaleK(10_000L) // 1 price step = 10K litoshi
.takerFee(1900L) // taker fee 1900 litoshi per 1 lot
.makerFee(700L) // maker fee 700 litoshi per 1 lot
.build();
future = api.submitBinaryDataAsync(new BatchAddSymbolsCommand(symbolSpecXbtLtc));
```
Create new users:
```java
// create user uid=301
future = api.submitCommandAsync(ApiAddUser.builder()
.uid(301L)
.build());
// create user uid=302
future = api.submitCommandAsync(ApiAddUser.builder()
.uid(302L)
.build());
```
Perform deposits:
```java
// first user deposits 20 LTC
future = api.submitCommandAsync(ApiAdjustUserBalance.builder()
.uid(301L)
.currency(currencyCodeLtc)
.amount(2_000_000_000L)
.transactionId(1L)
.build());
// second user deposits 0.10 BTC
future = api.submitCommandAsync(ApiAdjustUserBalance.builder()
.uid(302L)
.currency(currencyCodeXbt)
.amount(10_000_000L)
.transactionId(2L)
.build());
```
Place orders:
```java
// first user places Good-till-Cancel Bid order
// he assumes BTCLTC exchange rate 154 LTC for 1 BTC
// bid price for 1 lot (0.01BTC) is 1.54 LTC => 1_5400_0000 litoshi => 10K * 15_400 (in price steps)
future = api.submitCommandAsync(ApiPlaceOrder.builder()
.uid(301L)
.orderId(5001L)
.price(15_400L)
.reservePrice(15_600L) // can move bid order up to the 1.56 LTC, without replacing it
.size(12L) // order size is 12 lots
.action(OrderAction.BID)
.orderType(OrderType.GTC) // Good-till-Cancel
.symbol(symbolXbtLtc)
.build());
// second user places Immediate-or-Cancel Ask (Sell) order
// he assumes wost rate to sell 152.5 LTC for 1 BTC
future = api.submitCommandAsync(ApiPlaceOrder.builder()
.uid(302L)
.orderId(5002L)
.price(15_250L)
.size(10L) // order size is 10 lots
.action(OrderAction.ASK)
.orderType(OrderType.IOC) // Immediate-or-Cancel
.symbol(symbolXbtLtc)
.build());
```
Request order book:
```java
future = api.requestOrderBookAsync(symbolXbtLtc, 10);
```
GtC orders manipulations:
```java
// first user moves remaining order to price 1.53 LTC
future = api.submitCommandAsync(ApiMoveOrder.builder()
.uid(301L)
.orderId(5001L)
.newPrice(15_300L)
.symbol(symbolXbtLtc)
.build());
// first user cancel remaining order
future = api.submitCommandAsync(ApiCancelOrder.builder()
.uid(301L)
.orderId(5001L)
.symbol(symbolXbtLtc)
.build());
```
Check user balance and GtC orders:
```java
Future report = api.processReport(new SingleUserReportQuery(301), 0);
```
Check system balance:
```java
// check fees collected
Future totalsReport = api.processReport(new TotalCurrencyBalanceReportQuery(), 0);
System.out.println("LTC fees collected: " + totalsReport.get().getFees().get(currencyCodeLtc));
```
### Testing
- latency test: mvn -Dtest=PerfLatency#testLatencyMargin test
- throughput test: mvn -Dtest=PerfThroughput#testThroughputMargin test
- hiccups test: mvn -Dtest=PerfHiccups#testHiccups test
- serialization test: mvn -Dtest=PerfPersistence#testPersistenceMargin test
### TODOs
- market data feeds (full order log, L2 market data, BBO, trades)
- clearing and settlement
- reporting
- clustering
- FIX and REST API gateways
- cryptocurrency payment gateway
- more tests and benchmarks
- NUMA-aware and CPU layout custom configuration
### Contributing
Exchange-core is an open-source project and contributions are welcome!
### Support
- [Discussion group in Telegram (t.me/exchangecoretalks)](https://t.me/exchangecoretalks)
- [News channel in Telegram (t.me/exchangecore)](https://t.me/exchangecore)
[license]:LICENSE.txt
[license img]:https://img.shields.io/badge/License-Apache%202-blue.svg