{"id":22261537,"url":"https://github.com/nimblemarkets/dbn-go","last_synced_at":"2026-03-15T03:21:58.169Z","repository":{"id":232891305,"uuid":"774543819","full_name":"NimbleMarkets/dbn-go","owner":"NimbleMarkets","description":"Golang bindings and tools for DataBento","archived":false,"fork":false,"pushed_at":"2026-02-11T20:14:15.000Z","size":1501,"stargazers_count":28,"open_issues_count":11,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-12T04:39:34.526Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NimbleMarkets.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2024-03-19T18:23:44.000Z","updated_at":"2026-02-11T20:06:39.000Z","dependencies_parsed_at":"2024-04-12T05:09:01.095Z","dependency_job_id":"b3dda62a-54cd-4ec7-9a43-c230cbbc63a1","html_url":"https://github.com/NimbleMarkets/dbn-go","commit_stats":null,"previous_names":["nimblemarkets/dbn-go"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/NimbleMarkets/dbn-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fdbn-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fdbn-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fdbn-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fdbn-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NimbleMarkets","download_url":"https://codeload.github.com/NimbleMarkets/dbn-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fdbn-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29441197,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T07:24:13.446Z","status":"ssl_error","status_checked_at":"2026-02-14T07:23:58.969Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-03T09:13:14.881Z","updated_at":"2026-02-14T09:08:03.054Z","avatar_url":"https://github.com/NimbleMarkets.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dbn-go - Golang bindings to DBN\n\n\u003cp\u003e\n    \u003ca href=\"https://github.com/NimbleMarkets/dbn-go/tags\"\u003e\u003cimg src=\"https://img.shields.io/github/tag/NimbleMarkets/dbn-go.svg\" alt=\"Latest Tag\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pkg.go.dev/github.com/NimbleMarkets/dbn-go\"\u003e\u003cimg src=\"https://pkg.go.dev/badge/github.com/NimbleMarkets/dbn-go.svg\" alt=\"Go Reference\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/NimbleMarkets/dbn-go/blob/main/CODE_OF_CONDUCT.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg\"  alt=\"Code Of Conduct\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n**Golang tooling for Databento's APIs and DBN format**\n\nThis repository contains Golang bindings to [Databento's](https://databento.com) file format [Databento Binary Encoding (DBN)](https://databento.com/docs/knowledge-base/new-users/dbn-encoding), [Historical API](#historical-api), and [Live API](#live-api).  It also includes [tools](./cmd/README.md) to interact with these services.\n\n * [Library Usage](#library-usage)\n * [Reading DBN Files](#reading-dbn-files)\n * [Reading JSON Files](#reading-json-files)\n * [Historical API](#historical-api)\n * [Live API](#live-api)\n * [Tools](#tools)\n   * [Tool Installation](./cmd/README.md#installation)\n\n**NOTE:** This library is **not** affiliated with Databento.  Please be careful with commands which incur billing.  We are not responsible for any charges you incur.\n\n\n## Library Usage\n\nTo use this library, import the following package, optionally with Databento Historical or Live support:\n\n```go\nimport (\n    dbn \"github.com/NimbleMarkets/dbn-go\"\n    dbn_hist \"github.com/NimbleMarkets/dbn-go/hist\"\n    dbn_live \"github.com/NimbleMarkets/dbn-go/live\"\n)\n```\n\nMost `dbn-go` [types](./structs.go) and [enums](./consts.go) parallel Databento's libraries.  Available messages types are:\n\n  * [`Mbp0Msg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Mbp0Msg)\n  * [`MboMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#MboMsg)\n  * [`Mbp1Msg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Mbp1Msg)\n  * [`Cmbp1Msg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Cmbp1Msg)\n  * [`Mbp10Msg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Mbp10Msg)\n  * [`OhlcvMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#OhlcvMsg)\n  * [`ImbalanceMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#ImbalanceMsg)\n  * [`SymbolMappingMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#SymbolMappingMsg)\n  * [`ErrorMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#ErrorMsg)\n  * [`SystemMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#SystemMsg)\n  * [`StatMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#StatMsg)\n  * [`StatusMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#StatusMsg)\n  * [`InstrumentDefMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#InstrumentDefMsg)\n\n\n## Reading DBN Files\n\nIf you want to read a homogeneous array of DBN records from a file, use the [`dbn.ReadDBNToSlice`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#ReadDBNToSlice) generic function. We include an `io.Reader` wrapper, [`dbn.MakeCompressedReader`]https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#MakeCompressedReader), that automatically handles `zstd`-named files.  The generic argument dicates which message type to read.\n\n```go\nfile, closer, _ := dbn.MakeCompressedReader(\"ohlcv-1s.dbn.zstd\", false)\ndefer closer.Close()\nrecords, metadata, err := dbn.ReadDBNToSlice[dbn.OhlcvMsg](file)\n```\n\nAlternatively, you can use the [`DBNScanner`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#DbnScanner) to read records one-by-one.  Each record can be handled directly, or automatically dispatched to the callback method of a struct that implements the [`dbn.Visitor` interface](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Visitor).\n\n```go\ndbnFile, _ := os.Open(\"ohlcv-1s.dbn\")\ndefer dbnFile.Close()\n\ndbnScanner := dbn.NewDbnScanner(dbnFile)\nmetadata, err := dbnScanner.Metadata()\nif err != nil {\n\treturn fmt.Errorf(\"scanner failed to read metadata: %w\", err)\n}\nfor dbnScanner.Next() {\n    header := dbnScanner.GetLastHeader()\n    fmt.Printf(\"rtype: %s  ts: %d\\n\", header.RType, header.TsEvent)\n\n    // you could get the raw bytes and size:\n    lastRawByteArray := dbnScanner.GetLastRecord()\n    lastSize := dbnScanner.GetLastSize()\n\n    // you probably want to use this generic helper to crack the message:\n    ohlcv, err := dbn.DbnScannerDecode[dbn.OhlcvMsg](dbnScanner)\n\n    // or if you are handing multiple message types, dispatch a Visitor:\n    err = dbnScanner.Visit(visitor)\n}\nif err := dbnScanner.Error(); err != nil \u0026\u0026 err != io.EOF {\n    return fmt.Errorf(\"scanner error: %w\", err)\n}\n```\n\n\n## Reading JSON Files\n\nIf you already have DBN-based JSON text files, you can use the generic [`dbn.ReadJsonToSlice`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#ReadJsonToSlice) or [`dbn.JsonScanner`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#JsonScanner) to read them in as `dbn-go` structs.  Similar to the raw DBN, you can handle records manually or use the [`dbn.Visitor` interface](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go#Visitor).\n\n```go\njsonFile, _ := os.Open(\"ohlcv-1s.dbn.json\")\ndefer jsonFile.Close()\nrecords, metadata, err := dbn.ReadJsonToSlice[dbn.OhlcvMsg](jsonFile)\n```\n\n```go\njsonFile, _ := os.Open(\"ohlcv-1s.dbn.json\")\ndefer jsonFile.Close()\njsonScanner := dbn.NewJsonScanner(jsonFile)\nfor jsonScanner.Next() {\n    if err := jsonScanner.Visit(visitor); err != nil {\n        return fmt.Errorf(\"visitor err: %w\", err)\n    }\n}\nif err := jsonScanner.Error(); err != nil {\n    fmt.Errorf(\"scanner err: %w\", err)\n}\n```\n\nMany of the `dbn-go` structs are annotated with `json` tags to facilitate JSON serialization and deserialization using `json.Marshal` and `json.Unmarshal`.  That said, `dbn-go` uses [`valyala/fastjson`](https://github.com/valyala/fastjson) and hand-written extraction code.\n\n\n## Historical API\n\nSupport for the [Databento Historical API](https://databento.com/docs/api-reference-historical) is available in the [`/hist`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/hist) folder.  Every API method is a function that takes an API key and arguments and returns a response struct and error.\n\n```\ndatabentoApiKey := os.Getenv(\"DATABENTO_API_KEY\")\nschemas, err := dbn_hist.ListSchemas(databentoApiKey, \"EQUS.MINI\")\n```\n\nThe source for `dbn-go-hist` illustrates [using this `dbn_hist` module](https://github.com/NimbleMarkets/dbn-go/blob/main/cmd/dbn-go-hist/main.go#L104).\n\n\n## Live API\n\nSupport for the [Databento Live API](https://databento.com/docs/api-reference-live) is available in the [`/live`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live) folder.\n\nThe general model is:\n\n  1. Use [`dbn_live.NewLiveClient`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#NewLiveClient) to create a [`LiveClient`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveClient) based on a [`LiveConfig`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveConfig) and connect to a DBN gateway\n  2. [`client.Authenticate`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveClient.Authenticate) with the gateway.\n  3. [`client.Subscribe`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveClient.Subscribe) to a stream using one or many [`SubscriptionRequestMsg`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#SubscriptionRequestMsg).\n  4. Begin the stream with [`client.Start`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveClient.Start).\n  5. Read the stream using [`client.GetDbnScanner`](https://pkg.go.dev/github.com/NimbleMarkets/dbn-go/live#LiveClient.GetDbnScanner).\n\nThe source for `dbn-go-live` illustrates [using this `dbn_live` module](https://github.com/NimbleMarkets/dbn-go/blob/main/cmd/dbn-go-live/main.go#L111).\n\n\n## Tools\n\nWe include [some tools](./cmd/README.md) to make our lives easier. [Installation instructions](./cmd/README.md#installation)\n\n * [`dbn-go-file`](./cmd/README.md#dbn-go-file): a CLI to process DBN files\n * [`dbn-go-hist`](./cmd/README.md#dbn-go-hist): a CLI to use the Historical API\n * [`dbn-go-live`](./cmd/README.md#dbn-go-live): a simple Live API feed handler\n * [`dbn-go-mcp`](./cmd/README.md#dbn-go-mcp): a LLM Model Context Protocol (MCP) server\n * [`dbn-go-slurp-docs`](./cmd/README.md#dbn-go-slurp-docs): a tool to scrape Databento docs for offline use\n * [`dbn-go-tui`](./cmd/README.md#dbn-go-tui): a TUI for your Databento account\n\n## Open Collaboration\n\nWe welcome contributions and feedback.  Please adhere to our [Code of Conduct](./CODE_OF_CONDUCT.md) when engaging our community.\n\n * [GitHub Issues](https://github.com/NimbleMarkets/dbn-go/issues)\n * [GitHub Pull Requests](https://github.com/NimbleMarkets/dbn-go/pulls)\n\n\n## License\n\nReleased under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0), see [LICENSE.txt](./LICENSE.txt).\n\nPortions adapted from [`databento/dbn`](https://github.com/databento/dbn) [`databendo/databento-rs`](https://github.com/databento/databento-rs) under the same Apache license.\n\nCopyright (c) 2024-2025 [Neomantra Corp](https://www.neomantra.com).   \n\n----\nMade with :heart: and :fire: by the team behind [Nimble.Markets](https://nimble.markets).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnimblemarkets%2Fdbn-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnimblemarkets%2Fdbn-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnimblemarkets%2Fdbn-go/lists"}