{"id":22609021,"url":"https://github.com/bitfinexcom/hypercore-logs","last_synced_at":"2025-07-28T19:08:03.188Z","repository":{"id":40511247,"uuid":"288678130","full_name":"bitfinexcom/hypercore-logs","owner":"bitfinexcom","description":null,"archived":false,"fork":false,"pushed_at":"2022-10-05T07:44:57.000Z","size":135,"stargazers_count":2,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-11T06:33:25.091Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitfinexcom.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-19T08:36:40.000Z","updated_at":"2024-01-24T16:26:25.000Z","dependencies_parsed_at":"2023-01-19T07:30:27.045Z","dependency_job_id":null,"html_url":"https://github.com/bitfinexcom/hypercore-logs","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/bitfinexcom/hypercore-logs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fhypercore-logs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fhypercore-logs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fhypercore-logs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fhypercore-logs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitfinexcom","download_url":"https://codeload.github.com/bitfinexcom/hypercore-logs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fhypercore-logs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267569484,"owners_count":24109086,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-08T15:10:38.606Z","updated_at":"2025-07-28T19:08:02.866Z","avatar_url":"https://github.com/bitfinexcom.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hypercore-logs\n\nHypercore logs is an API and CLI tool that provides functionality for writing/reading logs through hypercores. Currently it supports the following write formats:\n- `hypercore-logger` - this one is just the base wrapper for hypercore logs and the user could use this one to write it's own custom log transports\n- `hypercore-file-logger` - this one writes tail of the file to the logger feed\n- `hypercore-udp-logger` - this one reads logs though a udp server and writes them to the logger feed\n\nThe logs can be red simply though an transport layer agnostic module known as `hypercore-log-reader`.\n\nBeside providing the API for custom integrations the package contains also it's own cli tool (`hyperlog`) that can we be used for writing and reading logs.\n\n## Setup\n\nYou can simply install the tool by running `npm i -g https://github.com/bitfinexcom/hypercore-logs.git` and then you can use the cli by running `hyperlog` cmd through terminal. For custom code integration install it localy without `global` flag!\n\nAlso if you want to install specific version (preffered way), you can install it with release tag, e.g. `npm i -g https://github.com/bitfinexcom/hypercore-logs.git#v0.3.3`\n\n## Command line docs\n\nThe command line tool provides two basic commands:\n- `read` - read the hypercore log from destination by specifying the public key\n- `write` - write to the hypercore log from tailing file or udp server\n\nThe command can be run like this:\n```console\nhyperlog --help\nhyperlog \u003ccommand\u003e\n\nCommands:\n  hyperlog read       creates a reader for a hypercore log\n  hyperlog dht-read   creates a reader for a hyperswarm log\n  hyperlog write      creates a hypercore log writer\n  hyperlog dht-write  creates a hyperswarm log writer\n\nOptions:\n  --version  Show version number                                       [boolean]\n  --help     Show help                                                 [boolean]\n```\n\nWrite command usage:\n```console\nhyperlog write --help\nhyperlog write\n\ncreates a hypercore log writer\n\nOptions:\n  --version         Show version number                                [boolean]\n  --help            Show help                                          [boolean]\n  --key, -k         feed public key, use either hex string or path to file, if\n                    not specified alongside with 'secret-key' it will generate a\n                    new one                                             [string]\n  --secret-key, -s  feed private key, use either hex string or path to file, if\n                    not specified alongside with 'key' it will generate a new\n                    one                                                 [string]\n  --datadir, -d     feed data directory, if ommited RAM memory will be used\n                                                                        [string]\n  --file, -f        file, dir or glob pattern that will be tailed, use quoted\n                    arg when passing globs! Use either file or port option.\n                                                                        [string]\n  --republish       republish entire file to the stream, used alongside file\n                    option                            [boolean] [default: false]\n  --port, -p        UDP server port, use either file or port option     [number]\n```\n\nDHT write command usage:\n```console\nhyperlog dht-write --help\nhyperlog dht-write\n\ncreates a hyperswarm log writer\n\nOptions:\n      --version  Show version number                                   [boolean]\n      --help     Show help                                             [boolean]\n  -f, --file     file, dir or glob pattern that will be tailed, use quoted arg\n                 when passing globs! Use either file or port option.    [string]\n  -s, --seed     Key pair's seed                        [string] [default: null]\n```\n\nRead command usage:\n```console\nhyperlog read --help\nhyperlog read\n\ncreates a reader for a hypercore log\n\nOptions:\n  --version      Show version number                                   [boolean]\n  --help         Show help                                             [boolean]\n  --key, -k      feed public key, use either hex string or path to file\n                                                             [string] [required]\n  --datadir, -d  feed data directory, if ommited RAM memory will be used[string]\n  --output, -o   log output directory or file, if not provided output will be\n                 logged to console.                                     [string]\n  --remote-prefix, -rp   path prefix to be omitted                              [string]\n  --console, -c  log output to console, if output provided and console ommited\n                 then output would be logged only in file!             [boolean]\n  --tail         tail the log file                                     [boolean]\n  --start        feed read start, ignored in case if tail is specified, if\n                 negative it's considered from feed end                 [number]\n  --end          feed read end, ignored in case if tail is specified, if\n                 negative it's considered from feed end                 [number]\n  --start-date   feed read start by date, ignored in case if start\n                 is specified                                           [string]\n  --end-date     feed read end by date, ignored in case if end\n                 is specified                                           [string]\n  --include      filter logs by Regular expression                      [string]\n  --exclude      exclude logs by Regular expression, can be used along\n                 with \"include\" option                                  [string]\n```\n\nDHT read command usage:\n```console\nhyperlog dht-read --help\nhyperlog dht-read\n\ncreates a reader for a hyperswarm log\n\nOptions:\n      --version              Show version number                             [boolean]\n      --help                 Show help                                       [boolean]\n      -o, --output           log output directory or file, if not provided\n                             output will be logged to console.               [string]\n      --remote-prefix, --rp  path prefix to be omitted                       [string]\n      -c, --console          log output to console, if output provided and\n                             console ommited then output would be logged\n                             only in file!                                   [boolean]\n      --include              filter logs by Regular expression               [string]\n      --exclude              exclude logs by Regular expression, can be used\n                             along with \"include\" option                     [string]\n  -k, --key      feed public key, use either hex string or path to file\n                                                                             [string] [required]\n      --start-date           feed read start by date                         [string]\n      --end-date             feed read end by date                           [string]\n      --start-pattern        start feed on pattern match                     [string]\n      --end-pattern          end feed on pattern match                       [string]\n```\n\n## Examples\n\n### Example - tail file\n\n```console\nhyperlog write --file ../logs/1597852241433.log --datadir ../tmp\nhcore-logger key: 3c4e86d7ffdaf790d61c62bb1a025f2fa73b3881c19178ca3d08767b59c46023 +0ms\nhcore-logger secret-key: c8308cbeb469ee4a0a9add423bfd0197198cc08f61a8a029a0afd47f8930c84b3c4e86d7ffdaf790d61c62bb1a025f2fa73b3881c19178ca3d08767b59c46023 +5ms\nhcore-logger feed started listening for changes on ../logs/1597852241433.log +0ms\n```\n\n### Example - tail dir\n\n```console\nhyperlog write --file ../logs\nhcore-logger key: 482233ad2a7575010f4f9a7a19e013f5d6fa52f83b5f282b7b2c9928c6b334de +0ms\nhcore-logger secret-key: 6601acc4386c805f7a95f675369a6c96d6c5a207162eb0397a89a007fb282221482233ad2a7575010f4f9a7a19e013f5d6fa52f83b5f282b7b2c9928c6b334de +2ms\nhcore-logger feed started listening for changes on /home/vigan/Desktop/bitfinex/logs/test1.log, /home/vigan/Desktop/bitfinex/logs/test2.log +0ms\n```\n\n### Example - tail glob pattern\n\n```console\nhyperlog write --file '../logs/**/*.log'\nhcore-logger key: e686ec27b4699bc3039c344d2fd9a52daaf022fcac986dc345c2d49b3b6174b9 +0ms\nhcore-logger secret-key: 233a0e66b2f6dc46c7bedf0d60c49e47744c0758d5a41ff5ed0ee54364c1e704e686ec27b4699bc3039c344d2fd9a52daaf022fcac986dc345c2d49b3b6174b9 +3ms\nhcore-logger feed started listening for changes on /home/vigan/Desktop/bitfinex/logs/test1.log, /home/vigan/Desktop/bitfinex/logs/test2.log +0ms\n```\n\n### Example - udp server\n\n```console\nhyperlog write --port 7070 \\\n  --key 3c4e86d7ffdaf790d61c62bb1a025f2fa73b3881c19178ca3d08767b59c46023 \\\n  --secret-key c8308cbeb469ee4a0a9add423bfd0197198cc08f61a8a029a0afd47f8930c84b3c4e86d7ffdaf790d61c62bb1a025f2fa73b3881c19178ca3d08767b59c46023\n```\n\nUDP client for sending packages:\n```console\nnc -u 127.0.0.1 7070\nsome test message\nanother test message\n```\n\n### Example - reader\n\n```console\nhyperlog read --key 3c4e86d7ffdaf790d61c62bb1a025f2fa73b3881c19178ca3d08767b59c46023 --tail\n```\n\n### Example - read and store to file\n\n```console\nhyperlog read --key 8be30f022777683321f685125315e5a7a79a1978829e5e4416037e9bb30fef8e --output test3.log --console\n```\n\n### Example - read and store to dir\n\n```console\nhyperlog read --key 8be30f022777683321f685125315e5a7a79a1978829e5e4416037e9bb30fef8e --output test3/ --input-dir /home/dev/logs/\n```\n\n### Example - read and filter logs that contains \"[client]\" at the start and doesn't contain \"fatal\" word\n\n```console\nhyperlog read --key 8be30f022777683321f685125315e5a7a79a1978829e5e4416037e9bb30fef8e --console --include \"\\[server\\]\" --exclude \"warn\"\n```\n\n### Example - read and filter logs by start/end date\n\n```console\nnode ./bin/hyperlog.js read  --key 0db350f414a93274e5613930b79c7226f21270cfc1435b96350422a0abf63dc2 --console --start-date 2022-01-20T12:38:14.973Z --end-date 2022-01-21T12:38:25.977Z\n```\n\n### Example - DHT logger\n```console\nnode ./bin/hyperlog.js dht-write --file ../logs/1597852241433.log --seed 4416037e9bb30fef8e\n```\n\n### Example - DHT log reader\n```console\nnode ./bin/hyperlog.js dht-read --key 8be30f022777683321f685125315e5a7a79a1978829e5e4416037e9bb30fef8e\n```\n\n### Example - code\n\nLog writer:\n```js\nconst cwd = process.cwd()\nconst feedDir = path.join(cwd, '/tmp/data')\n\nconst server = new HyperCoreLogger(feedDir)\n\nconst main = async () =\u003e {\n  await server.start()\n  const pubinterval = setInterval(() =\u003e {\n    server.feed.append('some data')\n  }, 3000)\n\n  setTimeout(async () =\u003e {\n    clearInterval(pubinterval)\n    await server.stop()\n  }, 20000)\n}\n\nmain()\n\n```\n\nLog reader:\n```js\nconst key = '0db350f414a93274e5613930b79c7226f21270cfc1435b96350422a0abf63dc2'\nconst client = new HyperCoreLogReader(() =\u003e ram(), key, null, null, { snapshot: false, tail: true })\n\nconst main = async () =\u003e {\n  client.on('data', (data) =\u003e console.log(data.toString().trim()))\n  await client.start()\n\n  setTimeout(async () =\u003e {\n    await client.stop()\n  }, 60000)\n}\n\nmain()\n\n```\n\nMore examples can be found under examples directory!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fhypercore-logs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitfinexcom%2Fhypercore-logs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fhypercore-logs/lists"}