{"id":22608925,"url":"https://github.com/bitfinexcom/bitfinex-terminal","last_synced_at":"2025-04-11T06:14:11.358Z","repository":{"id":43552946,"uuid":"290581807","full_name":"bitfinexcom/bitfinex-terminal","owner":"bitfinexcom","description":null,"archived":false,"fork":false,"pushed_at":"2023-09-24T05:20:22.000Z","size":34609,"stargazers_count":68,"open_issues_count":11,"forks_count":21,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-11T06:14:03.541Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitfinexcom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-26T19:04:44.000Z","updated_at":"2024-10-15T06:13:56.000Z","dependencies_parsed_at":"2022-09-05T22:31:00.215Z","dependency_job_id":null,"html_url":"https://github.com/bitfinexcom/bitfinex-terminal","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbitfinex-terminal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbitfinex-terminal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbitfinex-terminal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbitfinex-terminal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitfinexcom","download_url":"https://codeload.github.com/bitfinexcom/bitfinex-terminal/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248351393,"owners_count":21089270,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-08T15:10:06.743Z","updated_at":"2025-04-11T06:14:11.298Z","avatar_url":"https://github.com/bitfinexcom.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bitfinex Terminal\n\nBitfinex Terminal offers market data as live data streams and is built on top of [Dazaar](https://github.com/bitfinexcom/dazaar). It offers first class support for Algo Traders by offering market data over a distributed database to everyone.\n\nThere are free and paid market data streams available. They give algo traders and data scientists easy and fast access to our historical trading data. Trading data is easy to download and replicate. It is shared on a P2P network - like BitTorrent - but with a database interface. Also, unlike with BitTorrent, you won't have to download everything until you can use it, streaming time ranges is supported. And if you decide to download more data, e.g. from an earlier timeframe, it will of course preserve the data ordered. Free data comes in a nice B-Tree. Order and funding books are in compressed append-only logs, both guarantee fast access.\n\nOur paid streams give access to historical orderbook data and historical funding book data.\nEvery five seconds we record the data of the funding and order books, which gives an insight into important characterisitcs of the market, i.e. liquidity over time.\nThe data is compressed using an effective algorithm, which allows for fast data transfer and small storage footprints.\n\n## Table of Contents\n\n  1. [How to use it?](#howtouse)\n  1. [Support for Algo Traders](#firstclass)\n  1. [Example: Get a Full Snapshot of the Free BTCUSD trades, and keep following new updates](#example-trades)\n  1. [Example: Query Free Candles Data](#example-candles)\n  1. [Example: Access the Paid Orderbooks with USDT](#example-orderbooks-usdt)\n  1. [Example: Access the Paid Orderbooks with LND](#example-orderbooks-lnd)\n  1. [Example: Access Historical Funding Book Data with LND](#example-fundingbooks-lnd)\n  1. [Example: Query Local Orderbook Data](#example-orderbooks-local)\n  1. [Example: Download and Query Historical Funding Stats](#example-funding-stats)\n  1. [Tutorial: Backtest your Trading Strategies with Bitfinex Terminal \u0026 Honey Framework](./articles/backtesting-with-hf.md)\n  1. [Tutorial: Execute your Trading Strategy with the Honey Framework and Bitfinex Terminal](./articles/execute-strategy-hf.md)\n  1. [Article: Learn more about Dazaar](https://blog.dazaar.com/2020/09/12/introducing-dazaar/)\n  1. [Video: Screencast for Free Data Streams](./img/video-free-streams.gif)\n\n\u003ca id=\"howtouse\" /\u003e\n\n## How to use it?\n\nEvery data stream in Bitfinex Terminal and also Dazaar has a unique `id` on the network. Imagine it like the url of the data stream. You can directly use ids or you can use Dazaar Cards, which are [available for download](./cards). We have prepared a few examples that will show how to use it. Bitfinex Terminal currently offers a mix of free and paid data streams:\n\n - Historical Candles Data (free)\n - Historical Trades Data (free)\n - Historical Funding Stats Data (free)\n - Historical Orderbooks Data (paid)\n - Historical Fundingbooks Data (paid)\n\nPaid streams will require you to send a specific amount of crypto currency to a defined address. Dazaar will start replicating the whole dataset in the background unless sparse-mode is enabled. That means your database query is resolved with a high priority, while it additionally downloads the whole dataset in the background. For a guided example, [see this short how-to guide.](#example-orderbooks)\n\n\u003ca id=\"firstclass\" /\u003e\n\n## First class support for Algo Traders\n\nBitfinex Terminal is also supported by the [Honey Framework](https://honey.bitfinex.com/). You can easily backtest your Honey Framework Strategies on Bitfinex Terminal with the [Honey Framework backtesting tools](./articles/backtesting-with-hf.md). And you can also decide to trade on your own or to sell the trading signals on the Dazaar network with [bfx-hf-strategy-dazaar](https://github.com/bitfinexcom/bfx-hf-strategy-dazaar).\n\n## Examples\n\n\u003ca id=\"example-trades\" /\u003e\n\nAlso make sure to check our [tutorial material](#tutorials) :)\n\n### Example: Get a Full Snapshot of the BTCUSD trades, and keep following new updates\n\nIn our first example we will prepare a full copy of the Bitfinex trades data. We also want to keep following new updates that come in. We will walk through a small set of code, as if we were writing it. The full example can be found at [examples/trades-full-copy.js](examples/trades-full-copy.js).\n\nAs a first step we have to require our dependencies, one of it are the terms of service that we can read at [https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use):\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\n\nconst Hyperbee = require('hyperbee')\nconst keyEncoding = require('bitfinex-terminal-key-encoding')\nconst terms = require('bitfinex-terminal-terms-of-use')\n```\n\nAnd we create a Dazaar market. Our data will be stored at `dbs/full-trades`\n\n```js\nconst market = dazaar('dbs/full-trades')\n```\n\nThen we download the [Dazaar Card for the stream](./cards/free/trades/bitfinex.terminal.btcusd.trades.json) and load it. The option `live: true` keeps the connection open, so we keep listening for updates after the data is fully synced. If we would set `sparse` to `true`, we would just download requested data that we request in a query. This way, we make a full copy. We also accept the Bitfinex Terminal terms of use by loading them into Dazaar, after we have read them:\n\n```js\nconst card = require('../cards/free/trades/bitfinex.terminal.btcusd.trades.json')\nconst buyer = dmarket.buy(card, { live: true, sparse: false, terms })\n```\n\nIn the next step a lot happens. We register an event listener for the `feed` event. When it is emitted, we know that the feed is ready for consuming. We also set up a `Hyperbee` instance, which will provide us a nice interface to access the data. We pass the `db` instance to a function called `doQuery` which we will highlight next.\n\n```js\nbuyer.on('feed', function () {\n  console.log('got feed')\n\n  const db = new Hyperbee(buyer.feed, {\n    keyEncoding,\n    valueEncoding: 'json'\n  })\n\n  doQuery(db)\n})\n```\n\nThe function `doQuery` will make a request for us. It will request the required data prioritized and print it to our console. We select all trades with a timestamp larger than October 2018, 9:00 UTC and `2019`. We also limit the results to `10`. The SQL equivalent to this query would be:\n\n```\nSELECT * from trades\nWHERE timestamp \u003e= 2018-10-10T09:00:00.000Z\n  AND timestamp \u003c= 2019-10-10T09:00:00.000Z\nLIMIT 10\n```\n\nThis is our query:\n\n```js\nfunction doQuery (db) {\n  db.createReadStream({\n    gte: { timestamp: new Date('2018-10-10T09:00:00.000Z') },\n    lte: { timestamp: new Date('2019-10-10T09:00:00.000Z') },\n    limit: 10\n  }).on('data', (d) =\u003e {\n    console.log(d)\n  })\n}\n```\n\n\nTo start everything, we have to join the P2P swarm:\n\n```js\nswarm(buyer)\n```\n\nAnd for demonstration purposes we also log the downloaded elements, so we can see the progress:\n\n```js\nsetInterval(() =\u003e {\n  if (!buyer.feed) return\n\n  console.log('data feed length is', buyer.feed.length, 'elements')\n}, 1000)\n```\n\nThis example code will download the whole BTCUSD trades dataset from Bitfinex Terminal. If the connection is reset, it will just resume where it stopped. We also make a request for a timerange and print it to the console. The timerange is prioritized and downloaded first. When all downloading is finished, we keep the socket open to receive the latest updates. That is a lot that is done for us behind the scenes.\n\n\u003ca id=\"example-candles\" /\u003e\n\n### Example: Query Candles Data\n\nQuerying Candle data is as easy as getting historical trade data. All candle data is consolidated into one database, and we just have to select our candle type.\n\nThe full example can be found at [examples/candles-sparse-select.js](examples/candles-sparse-select.js). Our setup is similar to the trades example, we require our dependencies and set up a database that is stored on disk:\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\n\nconst Hyperbee = require('hyperbee')\nconst keyEncoding = require('bitfinex-terminal-key-encoding')\n\nconst market = dazaar('dbs/sparse-candles')\n```\n\nThis time we load a [Dazaar Card for candles](./cards/free/candles/bitfinex.terminal.btcusd.candles.json). We also enable the `sparse` mode, that means, just the data we directly request is downloaded. We also accept the Bitfinex Terminal terms of use by loading module for it into Dazaar:\n\n```js\nconst card = require('../cards/free/candles/bitfinex.terminal.btcusd.candles.json')\nconst terms = require('bitfinex-terminal-terms-of-use')\nconst buyer = dmarket.buy(card, { sparse: true, terms })\n```\n\nOur event listener looks exactly the same as the one from the previous example:\n\n```js\nbuyer.on('feed', function () {\n  console.log('got feed')\n\n  const db = new Hyperbee(buyer.feed, {\n    keyEncoding,\n    valueEncoding: 'json'\n  })\n\n  doQuery(db)\n})\n```\n\nFor our query we define a timeframe and the candle type. We also reverse the results, so we should get the newest entries first. The equivalent SQL query is:\n\n```\nSELECT * from candles\nWHERE candle = '5m'\n AND timestamp \u003e= 2018-10-10T09:00:00.000Z\n AND timestamp \u003c= 2019-10-10T09:00:00.000Z\nLIMIT 10\nORDER BY timestamp DESC\n```\n\nHere is our query:\n\n```js\nfunction doQuery (db) {\n  db.createReadStream({\n    gte: { candle: '5m', timestamp: new Date('2018-10-10T09:00:00.000Z') },\n    lte: { candle: '5m', timestamp: new Date('2019-10-10T09:00:00.000Z') },\n    limit: 10,\n    reverse: true\n  }).on('data', (d) =\u003e {\n    console.log(d)\n  })\n}\n```\n\nIn this example we did a sparse-enabled select on the data, so just the data we selected is downloaded. The full example can be found in [examples/candles-sparse-select.js](examples/candles-sparse-select.js).\n\n\u003ca id=\"example-orderbooks-usdt\" /\u003e\n\n### Example: Access the Paid Orderbooks with USDT\n\nSimilar to paying for time in other cloud computing services you pay Bitfinex Terminal streams on an hourly basis.\n\nAs a first step we need to download the Dazaar Card for the stream we are interested in. In this how-to-guide we'll fetch data from the BTCUSD orderbook stream. A final note: please follow the code tutorial and code example first before making any payments, as payment time starts counting as soon as the payment is done.\n\n```\nwget https://github.com/bitfinexcom/bitfinex-terminal/tree/master/cards/paid/orderbooks/bitfinex.terminal.btcusd.orderbook.json\n```\n\nTo access the data we need to install several modules:\n\n```\nnpm install dazaar @dazaar/payment-eth \\\n  bitfinex-terminal-order-book bitfinex-terminal-terms-of-use\n```\n\nOne particular module we install is the terms of use module. You can read the terms of use here: [https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use). Later we will pass that module to the dazaar market to accept the terms.\n\nWe create a file called `test-orderbooks.js` and load the modules we just installed with npm:\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\nconst Payment = require('@dazaar/payment-eth')\nconst terms = require('bitfinex-terminal-terms-of-use')\n\nconst Orderbook = require('bitfinex-terminal-order-book')\n```\n\nWe also load the Dazaar Card of of the data stream:\n\n```js\nconst card = require('./bitfinex.terminal.btcusd.orderbook.json')\n```\n\nWhen we open the file `bitfinex.terminal.btcusd.orderbook.json` we can see the Dazaar Card. The `id` field refers to the data stream, and the payment section gives instructions for payment. You can see that it requires 2 cents USDT for one hour of access:\n\n```\n  \"amount\": 0.02,\n  \"unit\": \"hours\",\n  \"interval\": 1\n```\n\n\nSimilar to Free Bitfinex Terminal data we set up a database and load the card. We also accept the terms of use by passing them as an argument:\n\n```js\nconst market = dazaar('./dbs/terminal-orderbook-btcusd')\nconst buyer = market.buy(card, { sparse: false, terms })\n```\n\nWhen our local client is bootstrapped, we print the payment informations to the console:\n\n```js\nbuyer.ready(function () {\n  console.log('Pay to ' + Payment.tweak(buyer.key, card))\n\n  swarm(buyer)\n})\n```\n\nIt will print something like `Pay to 0xaf2333...` to the console. When we pay 2 cents USDT to that address, we get one hour of access to the stream and our replication starts. But for now, we just continue to write our example program.\n\nWhen we made the payment, as soon as our payment is received, the feed event is triggered, and we start replicating the data. In case we have already payed, the `feed` event is triggered, too. For example purposes, we also do a database query with the orderbook query api:\n\n```js\nbuyer.on('feed', function () {\n  console.log('got feed')\n\n  const o = new Orderbook(buyer.feed)\n  doQuery(o)\n})\n```\n\nOur query function will fetch the entries of the last 5 minutes. It is the equivalent to the SQL query `SELECT * from \u003ctable\u003e WHERE timestamp \u003e $TIMESTAMP_5_MINUTES_AGO`. The option `live` keeps the stream open, so each new entry is printed to the console:\n\n```js\nasync function doQuery (o) {\n  const start = new Date().getTime() - (1000 * 60 * 5)\n\n  const s = o.createReadStream({ start, live: true })\n\n  for await (const data of s) {\n    console.log('---\u003e', data)\n  }\n}\n```\n\nWhen we now start or program with `node test-orderbooks.js` it will print an ETH address to the console. When we pay 0.02 USDT to it, the streaming of data will start. Additionally older data is replicated in the background, for later use. When we want to consume more data, we can pay any time. If we pay before the hour ends, it will be added on top of our available time.\n\nThe full code example can be found in [examples/payed-orderbooks.js](examples/payed-orderbooks.js).\n\n\u003ca id=\"example-orderbooks-lnd\" /\u003e\n\n### Example: Access the Paid Orderbooks with Lightning\n\nNext to USDT and LEO, payments with Bitcoin via Lightning are possible. The current fee is 100 satoshi per hour per dataset, as we will see soon. Let's get started.\n\nSimilar to the previous example we download the Dazaar Card of the data stream we are interested in. For our example we'll use the BTCUSD orderbooks Dazzar Card again:\n\n```\nwget https://github.com/bitfinexcom/bitfinex-terminal/tree/master/cards/paid/orderbooks/bitfinex.terminal.btcusd.orderbook.json\n```\n\nWhen we open the file `bitfinex.terminal.btcusd.orderbook.json` we see the different pricings. One section refers to BTC, it mentions an amount of 100 satoshi per hour:\n\n```\n{\n  \"method\": \"Lightning\",\n  \"currency\": \"SATS\",\n  \"amount\": 100,\n  \"unit\": \"hours\",\n  \"interval\": 1,\n  \"minSeconds\": 1\n}\n```\n\nWith that in mind we can start to code and install the required dependencies. To pay with Lighning, we have to use the `@dazaar/payment-lightning` module:\n\n```\nnpm install dazaar @dazaar/payment-lightning  \\\n  bitfinex-terminal-order-book bitfinex-terminal-terms-of-use \\\n  hyperbee\n```\n\nIn our code, we load our dependencies.\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\nconst Payment = require('@dazaar/payment-lightning')\nconst terms = require('bitfinex-terminal-terms-of-use')\nconst Orderbook = require('bitfinex-terminal-order-book')\n```\n\nOne important module is the terms of use module `bitfinex-terminal-terms-of-use`. You can read the terms of use here: [https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use). If you agree with the terms, we can continue to code and pass the module content later to the Dazaar market.\n\nWe also have to load our Dazaar Card:\n\n```js\nconst card = require('./bitfinex.terminal.btcusd.orderbook.json')\n```\n\nSimilar to the previous examples, we set up a database:\n\n```js\nconst market = dazaar('dbs/terminal-orderbook-btcusd-lnd')\n```\n\nAnd we also initiate the Dazaar market. Please note that you have to insert the terms of use here, if you agree with them:\n\n```js\nconst buyer = market.buy(card, { sparse: false, terms })\n```\n\nWhen our market setup from above is finished, the `ready` function is called as a callback. In the next step we request an invoice for `100 satoshi`, which will give us an hour to access and download data. The invoice is printed to the console, and can be paid with any Lightning wallet:\n\n```js\nbuyer.ready(function () {\n  console.log('ready')\n\n  const payment = new Payment(buyer)\n  payment.requestInvoice(100, function (err, invoice) {\n    if (err) console.log(err)\n    console.log(invoice)\n  })\n\n  swarm(buyer)\n})\n```\n\nWhen the payment has arrived its destination, the stream becomes available for Dazaar. On our side a `feed` event is emitted, and we know that we can send our database queries. Of course the `feed` event is also emitted when we have made a payment in the past that is still valid.\n\n```js\nbuyer.on('feed', function () {\n  console.log('got feed')\n\n  const o = new Orderbook(buyer.feed)\n  doQuery(o)\n})\n```\n\nOur query will be very simple, the SQL equivalent is:\n\n```\nSELECT * from orderbooks\nWHERE timestamp \u003e $TIMESTAMP_10_MINUTES_AGO\n```\n\nIn addition, we keep the stream open and listen for ne data coming in with `live: true`. There are other query modifier available for more complex queries, namely `end` and `limit`.\n\n`limit` will limit the result set to the defined amount and with `end` a we can set the upper bound for our timeseries query.\n\n```js\nasync function doQuery (o) {\n  const s = o.createReadStream({\n    start: new Date().getTime() - (1000 * 60 * 10),\n    live: true\n  })\n\n  for await (const data of s) {\n    console.log('---\u003e', data)\n  }\n}\n```\n\nWhen we now run the code, which is also located at [examples/payed-orderbooks-lnd.js](examples/payed-orderbooks-lnd.js), it will print a Lightning invoice.\n\n![invoice](./img/lnd-invoice.png)\n\nOnce we pay the invoice, we get access, the query is exectuted and the results printed to the console. As sparse mode is set to `false`, the db also replicates the whole dataset in the background.\n\nThe full code example can be found in [examples/payed-orderbooks-lnd.js](examples/payed-orderbooks-lnd.js).\n\n\n\u003ca id=\"example-orderbooks-local\" /\u003e\n\n### Example: Query Local Orderbook Data\n\nOf course its possible to query the local copy of the orderbook data you've paid for. In essence, we'll just remove the part where we connect to the servers. That means, we will modify [examples/payed-orderbooks.js](examples/payed-orderbooks.js) a little bit.\n\nAs we don't connect to Dazaar network we need to require less modules:\n\n```js\nconst dazaar = require('dazaar')\nconst terms = require('bitfinex-terminal-terms-of-use')\n\nconst Orderbook = require('bitfinex-terminal-order-book')\n```\n\nAs in the previous example, we require our card and setup Dazaar. Note that the used database path `dbs/terminal-orderbook-btcusd` is equal to the one used in the previous example, where we paid and downloaded data using the ETH payment provider:\n\n```js\nconst card = require('../cards/paid/orderbooks/bitfinex.terminal.btcusd.orderbook.json')\nconst market = dazaar('dbs/terminal-orderbook-btcusd')\nconst buyer = market.buy(card, { sparse: false, terms })\n```\n\nWhen the `buyer.ready` callback is called, we initialise the database query API and run our query:\n\n```js\nbuyer.ready(function () {\n  const o = new Orderbook(buyer.feed)\n  doQuery(o)\n})\n```\n\nOur query is the equivalent of a `SELECT * from \u003ctable\u003e LIMIT 5` query in SQL:\n\n```js\nasync function doQuery (o) {\n  const s = o.createReadStream({ limit: 5 })\n\n  for await (const data of s) {\n    console.log('---\u003e', data)\n  }\n}\n```\n\nLets say we want to do a range query, with a max of 5 results. The equivalent in SQL would be:\n\n```\nSELECT * from \u003ctable\u003e WHERE timestamp \u003e $TIMESTAMP_5_MINUTES_AGO AND timestamp \u003c $NOW LIMIT 5\n```\n\nWe would replace our query in `doQuery` in this way:\n\n```js\nconst start = new Date().getTime() - (1000 * 60 * 5)\nconst end = Date.now()\n\nconst s = o.createReadStream({ limit: 5, start, end  })\n```\n\nThe full example can be found at [examples/payed-orderbook-local.js](examples/payed-orderbook-local.js).\n\n\n### Example: Access Historical Funding Book Data with LND\n\n\u003ca id=\"example-fundingbooks-lnd\" /\u003e\n\nHistorical funding book data gives an insight into the development of the funding market over time. Access to historical funding book data works similar to orderbooks, we have to install required modules first:\n\n```\nnpm install dazaar @dazaar/payment-lightning  \\\n  bitfinex-terminal-funding-book-encoding bitfinex-terminal-terms-of-use \\\n  hyperbee\n```\n\nBefore we can start we have to choose a funding currency. In our example we use `USD`. We download the Dazaar Card for it:\n\n```\nwget https://github.com/bitfinexcom/bitfinex-terminal/tree/master/cards/paid/funding/bitfinex.terminal.funding.books.json\n```\n\nLets do some coding now. To start we need to require our dependencies:\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\nconst Hyperbee = require('hyperbee')\nconst Payment = require('@dazaar/payment-lightning')\nconst { keyEncoding, valueEncoding } = require('bitfinex-terminal-funding-book-encoding')\n\nconst terms = require('bitfinex-terminal-terms-of-use')\n```\n\nOne important module is the module `bitfinex-terminal-terms-of-use`. You can read the terms of use here: [https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use). Just when you agree with the terms you can continue with the tutorial. In order to agree, we have to pass the terms into the Dazaar buy function later.\n\nSimilar to previous examples we require the card and set up the database. We will query the USD funding book. This time we enable sparse mode. That means, just the data we request in a query is downlaoded. Sparse mode gives faster results to ad-hoc queries, but it does not download the other data that is not part of the query in the background for us. Please also note that we are passing the terms of use here to accept them.\n\n```js\nconst card = require('../cards/paid/funding/bitfinex.terminal.funding.books.json')\n\nconst market = dazaar('dbs/terminal-fundingbook-usd-lnd')\nconst buyer = market.buy(card, { sparse: true, terms })\n```\n\nThe next step is to request an invoice for payment. It is printed to the console:\n\n```js\nbuyer.ready(function () {\n  console.log('ready')\n\n  const payment = new Payment(buyer)\n  payment.requestInvoice(100, function (err, invoice) {\n    if (err) console.log(err)\n    console.log(invoice)\n  })\n\n  swarm(buyer)\n})\n```\n\nOnce payment has happened and there is still time left to download and query data, the `feed` event is emitted, and we can query the data:\n\n```js\nbuyer.on('feed', function () {\n  const db = new Hyperbee(buyer.feed, {\n    keyEncoding,\n    valueEncoding\n  })\n\n  doQuery(db)\n})\n```\n\nThe function `doQuery` runs the actual query of the data that also replicates the data that is replicated. As we are in sparse-mode, the data not related to our query is not replicated in the background:\n\n```js\nfunction doQuery (db) {\n  db.sub('USD').createReadStream({\n    limit: 10,\n    reverse: true\n  }).on('data', (d) =\u003e {\n    console.log(JSON.stringify(d, null, 2))\n  })\n}\n```\n\nYou can find the full example in [examples/payed-fundingbooks-lnd.js](examples/payed-fundingbooks-lnd.js).\n\n\n### Example: Download and Query Historical Funding Stats\n\n\u003ca id=\"example-funding-stats\" /\u003e\n\nFunding Stats give you an insight about the development of the funding market available at Bitfinex. The data for all funding pairs is stored in a single database. In this short intro we will replicate the whole database and run a query on it.\n\nFirst we'll have to install the required modules:\n\n```\nnpm install dazaar bitfinex-terminal-terms-of-use \\\n  hyperbee bitfinex-terminal-funding-encoding\n```\n\nSimilar to the other data streams we have to require `dazaar` and `hyperbee`:\n\n```js\nconst dazaar = require('dazaar')\nconst swarm = require('dazaar/swarm')\n\nconst Hyperbee = require('hyperbee')\n```\n\nThe Terms of Service are accepted by requiring the `bitfinex-terminal-terms-of-use` module, and then passing it into Dazaar. For now, we just require it:\n\n```js\nconst terms = require('bitfinex-terminal-terms-of-use')\n```\n\nWe also load the key- and value-encodings for the funding stats database:\n\n```js\nconst { keyEncoding, valueEncoding } = require('bitfinex-terminal-funding-encoding')\n```\n\nIn the next step we set up the Dazaar database:\n\n```js\nconst market = dazaar('dbs/funding-stats')\n```\n\nWe require the Dazaar Card which points to the data stream. In case you wonder, the card is available [here](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/cards/free/funding-stats/bitfinex.terminal.funding.stats.json):\n\n```js\nconst card = require('../cards/free/funding-stats/bitfinex.terminal.funding.stats.json')\n```\n\nTo access the data we have to accept the Terms of Service. You can read them [here](https://github.com/bitfinexcom/bitfinex-terminal/tree/master/terms-of-use). If you agree with the terms, you pass them together with the Dazaar Card as an argument to the `market.buy` function. Please note that we also set `sparse` to `false`, so the whole database will get replicated in the background. In case we would set `sparse` to `true`, just the data resolving from our query would be replicated.\n\n```js\nconst buyer = market.buy(card, { sparse: false, terms })\n```\n\nAs next steps we are going to initiate the database and run our query. When we connect to the remote peers, a `feed` event is emitted. Starting from that point we can query the database using `hyperbee`:\n\n```js\nbuyer.on('feed', function () {\n  console.log('got feed')\n\n  const db = new Hyperbee(buyer.feed, {\n    keyEncoding,\n    valueEncoding\n  })\n\n  doQuery(db)\n})\n```\n\nThe Funding Stats database uses sub-databases. For each funding pair there is a sub-database that can be accessed using the `db.sub()` command. The `doQuery` function will query funding stats for `USD`. We will query the last three entries for the timeframe between the third of January and the fifth of January 2021, 9 AM. The SQL quivalent for the query we run is:\n\n```\nSELECT * from funding_stats\nWHERE f_currency = 'USD'\n  AND datetime \u003e= '2021-01-03T09:00:00.000Z'\n  AND datetime \u003c= '2021-01-05T09:00:00.000Z'\nLIMIT 3\nORDER BY datetime DESC\n```\n\nOur hyperbee query looks like this, it logs the results to the console:\n\n```js\nfunction doQuery (db) {\n  db.sub('USD').createReadStream({\n    gte: new Date('2021-01-03T09:00:00.000Z'),\n    lte: new Date('2021-01-05T09:00:00.000Z'),\n    limit: 3,\n    reverse: true\n  }).on('data', (d) =\u003e {\n    console.log(d)\n  })\n}\n```\n\nThe last step for receiving the funding stats is to connect to the remote peers, and starting to download and query the data:\n\n```js\nswarm(buyer)\n```\n\nThat's it! When you run the code, it will download the whole funding stats database in the background. In addition it runs a query for `USD` funding stats to select a few data entries. You can find the full code in [examples/funding-stats.js](examples/funding-stats.js).\n\n## Tutorials\n\n\u003ca id=\"tutorials\" /\u003e\n\nOur tutorials can be found in the folder [articles](./articles).\n\n - [Tutorial: Backtest your Trading Strategies with Bitfinex Terminal \u0026 Honey Framework](./articles/backtesting-with-hf.md)\n - [Tutorial: Execute your Trading Strategy with the Honey Framework and Bitfinex Terminal](./articles/execute-strategy-hf.md)\n\n## Articles\n\n\u003ca id=\"articles\" /\u003e\n\n - [Introducing Dazaar](https://blog.dazaar.com/2020/09/12/introducing-dazaar/)\n\n## Videos\n\n\u003ca id=\"videos\" /\u003e\n\n - [Video: Screencast for Free Data Streams](./img/video-free-streams.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fbitfinex-terminal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitfinexcom%2Fbitfinex-terminal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fbitfinex-terminal/lists"}