{"id":20031227,"url":"https://github.com/filippobovo/betfair_data","last_synced_at":"2026-02-27T04:03:50.645Z","repository":{"id":39761708,"uuid":"194394423","full_name":"FilippoBovo/betfair_data","owner":"FilippoBovo","description":"Simple script to collect market data from Betfair.","archived":false,"fork":false,"pushed_at":"2022-12-08T05:50:14.000Z","size":23,"stargazers_count":6,"open_issues_count":4,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-05T04:36:27.713Z","etag":null,"topics":["betfair","betfair-api","collection","data","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FilippoBovo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-29T10:42:58.000Z","updated_at":"2024-08-25T06:06:53.000Z","dependencies_parsed_at":"2023-01-25T03:15:22.118Z","dependency_job_id":null,"html_url":"https://github.com/FilippoBovo/betfair_data","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FilippoBovo/betfair_data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoBovo%2Fbetfair_data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoBovo%2Fbetfair_data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoBovo%2Fbetfair_data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoBovo%2Fbetfair_data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FilippoBovo","download_url":"https://codeload.github.com/FilippoBovo/betfair_data/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoBovo%2Fbetfair_data/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29884515,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T23:51:21.483Z","status":"online","status_checked_at":"2026-02-27T02:00:06.759Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["betfair","betfair-api","collection","data","python"],"created_at":"2024-11-13T09:31:44.519Z","updated_at":"2026-02-27T04:03:50.629Z","avatar_url":"https://github.com/FilippoBovo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Betfair Data\n\nSimple script to download live market ladder data of a single market from Betfair and store it into an SQLite database. In addition, you can convert the downloaded data into a Parquet file, which has a much smaller size and faster loading times.\n\n## Install\n\nInstall the Python requirement packages with the following command.\n\n```shell\npip install -r requirements.txt\n```\n\n## Run\n\nTo run the script, first, export the Betfair credentials details as environment variables.\n\n```shell\nexport BETFAIR_USERNAME=\u003cbetfair_username\u003e\nexport BETFAIR_PASSWORD=\u003cbetfair_password\u003e\nexport BETFAIR_APP_KEY=\u003cbetfair_app_key\u003e\nexport BETFAIR_CERT_FILE=\u003cpath_to_the_betfair_certificate_file\u003e\nexport BETFAIR_CERT_KEY_FILE=\u003cpath_to_the_betfair_certificate_key_file\u003e\n```\n\nIn the above code, substitute the placeholders, like `\u003cbetfair_username\u003e` with your details.\n\nThen, invoke the script using Python.\n\n```shell\npython record_market_ladder.py \u003cmarket_id\u003e\n```\n\nHere, `\u003cmarket_id\u003e` is the ID of the market whose live market ladder data you wish to download.\n\nTo terminate the program, press Ctrl + C.\n\nFor more information on how to use the script, you may use the help flag.\n\n```shell\npython record_market_ladder.py --help\n```\n\nIt is also possible to run several recording tasks in the background by using the following command.\n\n```shell\ncat market_ids.txt | xargs -I market_id screen -d -m bash -c \"python record_market_ladder.py market_id\"\n```\n\nHere, `market_ids.txt` is a file containing a list of market IDs, like in the following example.\n\n```\n1.160287874\n1.160287758\n1.160287990\n1.160288106\n```\n\nThere will be one recording task for each market ID in the list.\n\nTo convert a zipped SQL file outputed by the `record_market_ladder.py` script to a Parquet file, which is much smaller in size and has faster loading times, you can use the `sql_to_parquet.py` script.\n\n```shell\npython sql_to_parquet.py file.zip\n```\n\nHere, `file.zip` is the zipped SQL file outputed by the `record_market_ladder.py` script.\n\nFor more information on how to use the script, you may use the help flag.\n\n```shell\npython sql_to_parquet.py --help\n```\n\nTo convert multiple files in a folder, you can using the following conventient command.\n\n```shell\nfind \u003cfolder_with_zipped_sql_files\u003e -name \"*.zip\" -type f -print0 | xargs -0 -I file python sql_to_parquet.py file\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilippobovo%2Fbetfair_data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilippobovo%2Fbetfair_data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilippobovo%2Fbetfair_data/lists"}