{"id":37101834,"url":"https://github.com/alpstable/gidari","last_synced_at":"2026-01-14T12:21:25.912Z","repository":{"id":60591295,"uuid":"517471861","full_name":"alpstable/gidari","owner":"alpstable","description":"Transport web data to local/remote storage using Gidari","archived":false,"fork":false,"pushed_at":"2023-12-18T22:47:32.000Z","size":33358,"stargazers_count":29,"open_issues_count":8,"forks_count":28,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-08T21:17:18.683Z","etag":null,"topics":["api-wrapper","csv","data-science","go","http","mongodb","storage"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alpstable.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-25T00:45:41.000Z","updated_at":"2024-06-17T10:45:50.000Z","dependencies_parsed_at":"2024-06-18T18:23:29.320Z","dependency_job_id":"9299ad8c-db72-41b8-b447-a013a5d35f56","html_url":"https://github.com/alpstable/gidari","commit_stats":{"total_commits":717,"total_committers":37,"mean_commits":19.37837837837838,"dds":"0.28730822873082285","last_synced_commit":"f7707994a72657de533d59c3bf3cb647785263c4"},"previous_names":["alpine-hodler/gidari"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/alpstable/gidari","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpstable%2Fgidari","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpstable%2Fgidari/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpstable%2Fgidari/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpstable%2Fgidari/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alpstable","download_url":"https://codeload.github.com/alpstable/gidari/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpstable%2Fgidari/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28420513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-wrapper","csv","data-science","go","http","mongodb","storage"],"created_at":"2026-01-14T12:21:25.294Z","updated_at":"2026-01-14T12:21:25.904Z","avatar_url":"https://github.com/alpstable.png","language":"Go","readme":"# Gidari\n\n[![PkgGoDev](https://img.shields.io/badge/go.dev-docs-007d9c?logo=go\u0026logoColor=white)](https://pkg.go.dev/github.com/alpstable/gidari)\n![Build Status](https://github.com/alpstable/gidari/actions/workflows/ci.yml/badge.svg)\n[![Go Report Card](https://goreportcard.com/badge/github.com/alpstable/gidari)](https://goreportcard.com/report/github.com/alpstable/gidari)\n[![Discord](https://img.shields.io/discord/987810353767403550)](https://discord.gg/3jGYQz74s7)\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/alpstable/gidari/main/etc/assets/gidari-gopher.png\" width=\"300\"\u003e\u003c/p\u003e\n\nGidari is a library for batch querying data and persisting the results to local storage.\n\n## Installation\n\n```sh\ngo get github.com/alpstable/gidari@latest\n```\n\nFor information on using the CLI, see [here](https://github.com/alpstable/gidari-cli).\n\n## Usage\n\nGidari supports HTTP and network socket services. There are two ways to use an HTTP service:\n\n1. Iterate over [`http.Response`](https://pkg.go.dev/net/http#Response) data, for pre-defined [`http.Request`](https://pkg.go.dev/net/http#Request)s.\n2. Define a writer to concurrently \"write\" response data for pre-defined `http.Request`s.\n\nSee the Go Docs or [Web-to-Storage Examples](#web-to-storage-examples) section for examples.\n\nNetwork sockets involves subscribing to a socket (such a web socket) and continuously iterating over the data via ReadWriter interface. The results would then be sent to a user-defined ListWriter to be stored. See the Go Docs or [Network Socket Examples](#network-socket-examples) section for examples.\n\n### Authenticating HTTP Requests\n\n| Protocol                                                                                   | Parameters                                                                    | Description                                                                                                                                                                                                       |\n|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [Basic](https://www.w3.org/Protocols/HTTP/1.0/spec.html#BasicAA)                           | Requires a username/email and password                                        | The server will authorize the request only if it can validate the user-ID and password for the protection space of the Request-URI                                                                                |\n| [Coinbase](https://docs.cloud.coinbase.com/exchange/docs/authorization-and-authentication) | Requires a key, passphrase and secrete generated in the Coinbase Exchange GUI | The Coinbase API requires a CB-ACCESS-SIGN header that is generated by creating a sha256 HMAC using the base64-decoded secret key on the prehash string from a timestamp + method + requestPath + body combination |\n| [Kraken](https://docs.kraken.com/rest/#section/Authentication) | Requires key and secret generated in the Kraken Pro GUI | The Kraken spot API uses a custom authentication algorithm that is based on a combination of API key, nonce, and message signature. The signature is generated using a hash-based message authentication code (HMAC) with SHA-512 as the hash function. |\n\n\n### Web-to-Storage Examples\n\n\n| Data Type | Writer                                          | Example                                      | Description                                               |\n|-----------|-------------------------------------------------|----------------------------------------------|-------------------------------------------------------------------|\n| [CSV](https://www.loc.gov/preservation/digital/formats/fdd/fdd000323.shtml)       | [csvpb](https://github.com/alpstable/csvpb)     | [examples/csvp](examples/csvpb/main.go)      | Use the HTTPService to write web API data to stdout as CSV        |\n| [MongoDB Document](https://www.mongodb.com/docs/manual/core/document/)  | [mongopb](https://github.com/alpstable/mongopb) | [exmaples/mongopb](examples/mongopb/main.go) | Use the HTTPService to write web API data to a MongoDB Collection |\n\n### Network Socket Examples\n\n| Type | Writer        | Example                                  | Description                                               |\n|------|---------------|------------------------------------------|-----------------------------------------------------------|\n| WebSocket |`fmt.Println` | [examples/websocket](examples/websocket) | Subscribe to the Coinbase WebSocket and print the results |\n\n## Contributing\n\nFollow [this guide](docs/CONTRIBUTING.md) for information on contributing.\n\n## Resources\n\n- Public REST APIs from [Postman Documenter](https://documenter.getpostman.com/view/8854915/Szf7znEe)\n- Go Gopher artwork by [Victoria Trum](https://www.fiverr.com/victoria_trum?source=order_page_user_message_link)\n- The original Go gopher was designed by the awesome [Renee French](http://reneefrench.blogspot.com/)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpstable%2Fgidari","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falpstable%2Fgidari","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpstable%2Fgidari/lists"}