{"id":21407293,"url":"https://github.com/mkst/perch","last_synced_at":"2026-01-03T15:04:13.994Z","repository":{"id":114862643,"uuid":"101380542","full_name":"mkst/perch","owner":"mkst","description":"not quite a trading platform, written in q","archived":false,"fork":false,"pushed_at":"2019-06-15T12:43:21.000Z","size":72,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-23T04:13:16.639Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"q","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkst.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-25T07:56:56.000Z","updated_at":"2023-12-08T11:48:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"8e9ee9ec-e8bc-4538-8bd3-0789a3320668","html_url":"https://github.com/mkst/perch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkst%2Fperch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkst%2Fperch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkst%2Fperch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkst%2Fperch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkst","download_url":"https://codeload.github.com/mkst/perch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243902321,"owners_count":20366262,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T16:45:57.523Z","updated_at":"2026-01-03T15:04:13.921Z","avatar_url":"https://github.com/mkst.png","language":"q","funding_links":[],"categories":[],"sub_categories":[],"readme":"# perch\nNot quite a trading _platform_, written in q/kdb+. Very unfinished.\n\n_\"...Look on my works, ye mighty, and despair!\"_\n\n## TODO\n\n - [x] book builder (convert quotes into books)\n - [x] subscribers reconnect on disconnect (basic)\n - [ ] pool builder (aggregate books into pools)\n - [ ] order manager\n - [ ] HTML5 front-end for order manager\n - [ ] investigate alternative messaging protocol (e.g. zeromq)\n\n## Tutorial\n\n**Compile shared objects**\n\n    $ cd ./code/c/lib/socket/ \u0026\u0026 make\n\n**Start up logger, rdb, bookbuilder, bitstamp gateway \u0026 analytics**\n\n    $ ./scripts/bitstamp.sh\n\n**Roll log to HDB (multi-parted)**\n\n    $ q32 code/kdb/scripts/roll.q -log /tmp/perch/log/20171124 -hdb /tmp/perch/hdb -cov /tmp/perch/cover -multiparted\n\n**Read data from HDB (multi-parted)**\n\n```\nq)\\l /tmp/perch/cover\nq)select from Coverage\ndate       table sym    exch start                         end                           num   path           \n--------------------------------------------------------------------------------------------------------------\n2017.11.24 BBO   BTCUSD BITS 2017.11.24D18:48:56.482520000 2017.11.24D18:50:58.897789000 64    :/tmp/perch/hdb\n2017.11.24 BBO   BTCEUR BITS 2017.11.24D18:48:56.487848000 2017.11.24D18:50:47.955632000 15    :/tmp/perch/hdb\n2017.11.24 BBO   EURUSD BITS 2017.11.24D18:48:56.488533000 2017.11.24D18:50:59.809528000 74    :/tmp/perch/hdb\n2017.11.24 Book  BTCUSD BITS 2017.11.24D18:48:56.482520000 2017.11.24D18:50:58.897789000 106   :/tmp/perch/hdb\n2017.11.24 Book  BTCEUR BITS 2017.11.24D18:48:56.487848000 2017.11.24D18:50:59.725295000 105   :/tmp/perch/hdb\n2017.11.24 Book  EURUSD BITS 2017.11.24D18:48:56.488533000 2017.11.24D18:50:59.809528000 110   :/tmp/perch/hdb\n2017.11.24 Quote BTCUSD BITS 2017.11.24D18:48:56.481505000 2017.11.24D18:50:58.891458000 21040 :/tmp/perch/hdb\n2017.11.24 Quote BTCEUR BITS 2017.11.24D18:48:56.486751000 2017.11.24D18:50:59.717426000 20840 :/tmp/perch/hdb\n2017.11.24 Quote EURUSD BITS 2017.11.24D18:48:56.487226000 2017.11.24D18:50:59.805242000 21840 :/tmp/perch/hdb\n2017.11.24 Trade BTCUSD BITS 2017.11.24D18:48:56.481811000 2017.11.24D18:50:54.522541000 60    :/tmp/perch/hdb\n2017.11.24 Trade BTCEUR BITS 2017.11.24D18:48:56.487072000 2017.11.24D18:50:47.619267000 18    :/tmp/perch/hdb\n2017.11.24 Trade EURUSD BITS 2017.11.24D18:48:56.487448000 2017.11.24D18:50:32.060525000 6     :/tmp/perch/hdb\n\nq)extract select from Coverage where date=2017.11.24, table=`Quote, sym=`BTCUSD\ndate       time                          timeExch                      sym    exch side price   qty        action snapshot\n--------------------------------------------------------------------------------------------------------------------------\n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8265    0.2251282  N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8254.59 1.4952     N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8254.57 1.2062     N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8253.74 19.67719   N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8250    1.131913   N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8248    0.1196997  N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8241.11 2.9988     N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8241.09 0.2284482  N      1       \n2017.11.24 2017.11.24D18:48:56.481505000 2017.11.24D18:48:56.425000000 BTCUSD BITS B    8241.06 13.23785   N      1    \n..\n```\n\n**Roll log to HDB (parted)**\n\n    q32 code/kdb/scripts/roll.q -log /tmp/perch/log/20171124 -hdb /tmp/perch/hdb2\n\n**Read data from HDB (parted)**\n```\nq)\\l /tmp/perch/hdb2\nq)tables[]\n`s#`BBO`Book`Quote`Trade\nq)select from Trade where date=last date, sym=`BTCUSD\ndate       sym    time                          timeExch                      exch side price   qty        id      \n-------------------------------------------------------------------------------------------------------------------\n2017.11.24 BTCUSD 2017.11.24D18:48:56.481811000 2017.11.24D18:48:30.000000000 BITS X    8270.06 0.1718778  27446276\n2017.11.24 BTCUSD 2017.11.24D18:49:10.866386000 2017.11.24D18:49:10.000000000 BITS X    8265    0.1691656  27446306\n2017.11.24 BTCUSD 2017.11.24D18:49:23.661188000 2017.11.24D18:49:23.000000000 BITS X    8269.59 0.00090683 27446324\n2017.11.24 BTCUSD 2017.11.24D18:49:23.779136000 2017.11.24D18:49:23.000000000 BITS X    8269.6  0.00090692 27446326\n2017.11.24 BTCUSD 2017.11.24D18:49:23.846100000 2017.11.24D18:49:23.000000000 BITS X    8269.6  0.00090691 27446328\n2017.11.24 BTCUSD 2017.11.24D18:49:23.941200000 2017.11.24D18:49:23.000000000 BITS X    8269.61 0.00078591 27446329\n2017.11.24 BTCUSD 2017.11.24D18:49:24.025934000 2017.11.24D18:49:23.000000000 BITS X    8269.62 0.02887223 27446330\n..\nq)meta Trade\nc       | t f a\n--------| -----\ndate    | d    \nsym     | s   p\ntime    | p    \ntimeExch| p    \nexch    | s    \nside    | c    \nprice   | f    \nqty     | f    \nid      | j\n```\n**Example query against HDB**\n```\nq)vwap:{$[z\u003elast s:sums x;0n;wavg[deltas z\u0026s;y]]}\nq)/aj all and pick 10 random trades, calculate vwap mid price\nq)10?select date, time, sym, exch, price, qty, mid:vmid, diff:abs(vmid-price) from update vmid:.5*vbid+vask from update vbid:vwap'[bidqty;bidpx;qty], vask:vwap'[askqty;askpx;qty] from aj[`sym`time;select from Trade where date=last date;select from Book where date=last date]\ndate       time                          sym    exch price   qty        mid      diff      \n-------------------------------------------------------------------------------------------\n2017.11.24 2017.11.24D18:50:19.994639000 BTCUSD BITS 8269.41 0.00078594 8262.11  7.299998  \n2017.11.24 2017.11.24D18:49:54.189139000 BTCUSD BITS 8269.59 0.00066486 8262.205 7.385     \n2017.11.24 2017.11.24D18:51:24.348548000 BTCUSD BITS 8254.4  0.0164427  8261.945 7.545     \n2017.11.24 2017.11.24D18:53:39.136744000 BTCEUR BITS 6940.02 0.0021     6943.49  3.47      \n2017.11.24 2017.11.24D18:50:10.856727000 BTCEUR BITS 6949.98 0.114      6944.998 4.98207   \n2017.11.24 2017.11.24D18:49:24.025934000 BTCUSD BITS 8269.62 0.02887223 8267.309 2.311235  \n2017.11.24 2017.11.24D18:50:24.318900000 BTCUSD BITS 8270.04 0.00066528 8270.045 0.005     \n2017.11.24 2017.11.24D18:50:39.365629000 BTCUSD BITS 8270.01 0.05379207 8270.035 0.02462925\n2017.11.24 2017.11.24D18:51:59.204386000 BTCUSD BITS 8253.74 1.687082   8255.707 1.967346  \n2017.11.24 2017.11.24D18:51:19.464555000 BTCUSD BITS 8255.19 0.4278546  8262.4   7.209989  \n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkst%2Fperch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkst%2Fperch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkst%2Fperch/lists"}