https://github.com/filippobovo/betfair_data
Simple script to collect market data from Betfair.
https://github.com/filippobovo/betfair_data
betfair betfair-api collection data python
Last synced: 9 months ago
JSON representation
Simple script to collect market data from Betfair.
- Host: GitHub
- URL: https://github.com/filippobovo/betfair_data
- Owner: FilippoBovo
- Created: 2019-06-29T10:42:58.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-12-08T05:50:14.000Z (about 3 years ago)
- Last Synced: 2025-04-08T16:55:13.900Z (10 months ago)
- Topics: betfair, betfair-api, collection, data, python
- Language: Python
- Size: 22.5 KB
- Stars: 6
- Watchers: 1
- Forks: 2
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Betfair Data
Simple script to download live market ladder data of a single market from Betfair and store it into an SQLite database. In addition, you can convert the downloaded data into a Parquet file, which has a much smaller size and faster loading times.
## Install
Install the Python requirement packages with the following command.
```shell
pip install -r requirements.txt
```
## Run
To run the script, first, export the Betfair credentials details as environment variables.
```shell
export BETFAIR_USERNAME=
export BETFAIR_PASSWORD=
export BETFAIR_APP_KEY=
export BETFAIR_CERT_FILE=
export BETFAIR_CERT_KEY_FILE=
```
In the above code, substitute the placeholders, like `` with your details.
Then, invoke the script using Python.
```shell
python record_market_ladder.py
```
Here, `` is the ID of the market whose live market ladder data you wish to download.
To terminate the program, press Ctrl + C.
For more information on how to use the script, you may use the help flag.
```shell
python record_market_ladder.py --help
```
It is also possible to run several recording tasks in the background by using the following command.
```shell
cat market_ids.txt | xargs -I market_id screen -d -m bash -c "python record_market_ladder.py market_id"
```
Here, `market_ids.txt` is a file containing a list of market IDs, like in the following example.
```
1.160287874
1.160287758
1.160287990
1.160288106
```
There will be one recording task for each market ID in the list.
To convert a zipped SQL file outputed by the `record_market_ladder.py` script to a Parquet file, which is much smaller in size and has faster loading times, you can use the `sql_to_parquet.py` script.
```shell
python sql_to_parquet.py file.zip
```
Here, `file.zip` is the zipped SQL file outputed by the `record_market_ladder.py` script.
For more information on how to use the script, you may use the help flag.
```shell
python sql_to_parquet.py --help
```
To convert multiple files in a folder, you can using the following conventient command.
```shell
find -name "*.zip" -type f -print0 | xargs -0 -I file python sql_to_parquet.py file
```