Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/alpacahq/pymarketstore

Python driver for MarketStore
https://github.com/alpacahq/pymarketstore

alpaca analytics marketstore pandas pandas-dataframe pip python3 quantitative-finance timeseries

Last synced: about 23 hours ago
JSON representation

Python driver for MarketStore

Awesome Lists containing this project

README 

        
# pymarketstore

Python driver for MarketStore



Build Status: ![build status](https://circleci.com/gh/alpacahq/pymarketstore/tree/master.png?971fa5b1079e8af0568db6caf772132c54f04dc2)



Pymarketstore can query and write financial timeseries data from [MarketStore](https://github.com/alpacahq/marketstore)



Tested with 3.3+



## How to install



```

$ pip install pymarketstore

```



## Examples



```

In [1]: import pymarketstore as pymkts



## query data



In [2]: param = pymkts.Params('BTC', '1Min', 'OHLCV', limit=10)



In [3]: cli = pymkts.Client()



In [4]: reply = cli.query(param)



In [5]: reply.first().df()

Out[5]:

                               Open      High       Low     Close     Volume

Epoch

2018-01-17 17:19:00+00:00  10400.00  10400.25  10315.00  10337.25   7.772154

2018-01-17 17:20:00+00:00  10328.22  10359.00  10328.22  10337.00  14.206040

2018-01-17 17:21:00+00:00  10337.01  10337.01  10180.01  10192.15   7.906481

2018-01-17 17:22:00+00:00  10199.99  10200.00  10129.88  10160.08  28.119562

2018-01-17 17:23:00+00:00  10140.01  10161.00  10115.00  10115.01  11.283704

2018-01-17 17:24:00+00:00  10115.00  10194.99  10102.35  10194.99  10.617131

2018-01-17 17:25:00+00:00  10194.99  10240.00  10194.98  10220.00   8.586766

2018-01-17 17:26:00+00:00  10210.02  10210.02  10101.00  10138.00   6.616969

2018-01-17 17:27:00+00:00  10137.99  10138.00  10108.76  10124.94   9.962978

2018-01-17 17:28:00+00:00  10124.95  10142.39  10124.94  10142.39   2.262249



## write data



In [7]: import numpy as np



In [8]: import pandas as pd



In [9]: data = np.array([(pd.Timestamp('2017-01-01 00:00').value / 10**9, 10.0)], dtype=[('Epoch', 'i8'), ('Ask', 'f4')])



In [10]: cli.write(data, 'TEST/1Min/Tick')

Out[10]: {'responses': None}



In [11]: cli.query(pymkts.Params('TEST', '1Min', 'Tick')).first().df()

Out[11]:

                            Ask

Epoch

2017-01-01 00:00:00+00:00  10.0



```



## Client



`pymkts.Client(endpoint='http://localhost:5993/rpc')`



Construct a client object with endpoint.



## Query



`pymkts.Client#query(symbols, timeframe, attrgroup, start=None, end=None, limit=None, limit_from_start=False)`



You can build parameters using `pymkts.Params`.



- symbols: string for a single symbol or a list of symbol string for multi-symbol query

- timeframe: timeframe string

- attrgroup: attribute group string.  symbols, timeframe and attrgroup compose a bucket key to query in the server

- start: unix epoch second (int), datetime object or timestamp string. The result will include only data timestamped equal to or after this time.

- end: unix epoch second (int), datetime object or timestamp string.  The result will include only data timestamped equal to or before this time.

- limit: the number of records to be returned, counting from either start or end boundary.

- limit_from_start: boolean to indicate `limit` is from the start boundary.  Defaults to False.



Pass one or multiple instances of `Params` to `Client.query()`.  It will return `QueryReply` object which holds internal numpy array data returned from the server.



## Write



`pymkts.Client#write(data, tbk)`



You can write a numpy array to the server via `Client.write()` method.  The data parameter must be numpy's [recarray type](https://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.recarray.html) with

a column named `Epoch` in int64 type at the first column.  `tbk` is the bucket key of the data records.



## List Symbols



`pymkts.Client#list_symbols()`



The list of all symbols stored in the server are returned.



## Server version



`pymkts.Client#server_version()`



Returns a string of Marketstore-Version header from a server response.



## Streaming



If the server supports WebSocket streaming, you can connect to it using

`pymkts.StreamConn` class.  For convenience, you can call `pymkts.Client#stream()` to obtain the instance with the same server

information as REST client.



Once you have this instance, you will set up some event handles by

either `register()` method or `@on()` decorator.  These methods accept

regular expressions to filter which stream to act on.



To actually connect and start receiving the messages from the server,

you will call `run()` with the stream names.  By default, it subscribes

to all by `*/*/*`.



`pymkts.Client#stream()`



Return a `StreamConn` which is a websocket connection to the server.



`pymkts.StreamConn#(endpoint)`



Create a connection instance to the `endpoint` server. The endpoint

string is a full URL with "ws" or "wss" scheme with the port and path.



`pymkts.StreamConn#register(stream_path, func)`

`@pymkts.StreamConn#on(stream_path)`



Add a new message handler to the connection.  The function will be called

with `handler(StreamConn, {"key": "...", "data": {...,}})` if the key

(time bucket key) matches with the `stream_path` regular expression.

The `on` method is a decorator version of `register`.



`pymkts.StreamConn#run([stream1, stream2, ...])`



Start communication with the server and go into an indefinite loop. It

does not return until unhandled exception is raised, in which case the

connection is closed so you need to implement retry.  Also, since this is

a blocking method, you may need to run it in a background thread.



An example code is as follows.



```

import pymarketstore as pymkts



conn = pymkts.StreamConn('ws://localhost:5993/ws')



@conn.on(r'^BTC/')

def on_btc(conn, msg):

    print('received btc', msg['data'])



conn.run(['BTC/*/*'])  # runs until exception



-> received btc {'Open': 4370.0, 'High': 4372.93, 'Low': 4370.0, 'Close': 4371.74, 'Volume': 3.3880948699999993, 'Epoch': 1507299600}

```