{"id":15325365,"url":"https://github.com/streamdp/ccd","last_synced_at":"2026-02-18T15:01:01.928Z","repository":{"id":104056523,"uuid":"433030882","full_name":"streamdp/ccd","owner":"streamdp","description":"Service for collecting data from a several crypto data providers.","archived":false,"fork":false,"pushed_at":"2025-09-10T09:49:40.000Z","size":7586,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-10T13:06:30.697Z","etag":null,"topics":["cryptocompare","gin-gonic","go-sql-driver-msyql","golang","huobi","postgresql","websocket-client","websocket-server"],"latest_commit_sha":null,"homepage":"https://ccd.oncook.top","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamdp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-11-29T12:19:04.000Z","updated_at":"2025-09-10T09:48:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"36e94719-43fd-4b6e-a56d-6ec8bab87df0","html_url":"https://github.com/streamdp/ccd","commit_stats":{"total_commits":152,"total_committers":4,"mean_commits":38.0,"dds":"0.48684210526315785","last_synced_commit":"9bb0c520a467f674e761566f5d758fe319539f10"},"previous_names":[],"tags_count":74,"template":false,"template_full_name":null,"purl":"pkg:github/streamdp/ccd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdp%2Fccd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdp%2Fccd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdp%2Fccd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdp%2Fccd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamdp","download_url":"https://codeload.github.com/streamdp/ccd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdp%2Fccd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29582786,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T13:56:48.962Z","status":"ssl_error","status_checked_at":"2026-02-18T13:54:34.145Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cryptocompare","gin-gonic","go-sql-driver-msyql","golang","huobi","postgresql","websocket-client","websocket-server"],"created_at":"2024-10-01T09:30:01.534Z","updated_at":"2026-02-18T15:01:01.912Z","avatar_url":"https://github.com/streamdp.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CCD\n[![Website ccd.oncook.top](https://img.shields.io/website-up-down-green-red/https/ccd.oncook.top/healthz.svg)](https://ccd.oncook.top/)\n[![GitHub release](https://img.shields.io/github/release/streamdp/ccd.svg)](https://github.com/streamdp/ccd/releases/)\n\nThis microservice is designed to manage real-time and historical cryptocurrency data collection. It provides both REST \nand WebSocket endpoints for flexible interaction with currency pair data. The service supports the following key \nfunctionalities:\n* **Multiple Data Provider Support**: You can choose between **cryptocompare**, **huobi** or **kraken** platform to \ncollect market data.\n* **Worker Management**: You can add, update, list, or remove background workers responsible for collecting data for \nspecific currency pairs. These workers handle data pulling at defined intervals.\n* **Symbol Management**: Add, update, list, or delete currency symbols that are tracked by the system.\n* **Price Retrieval**: Fetch the most recent price data for a selected currency pair, including cached data when the \ndata provider is unavailable.\n* **WebSocket Integration**: Provides a WebSocket connection for real-time updates. Clients can receive actual market \ninfo, subscribe or unsubscribe to specific currency pairs that are currently being collected.\n\n## Build app\n```bash\n$ go build -o ccd .\n````\n## Run app\nTo configure app, export some environment variables:\n```bash\nexport CCDC_DATAPROVIDER=cryptocompare #huobi, kraken\nexport CCDC_DATABASEURL=postgres://username:password@127.0.0.1:5432/dbname?sslmode=disable\nexport CCDC_APIKEY=put you api key here\nexport CCDC_SESSIONSTORE=redis // or \"db\", default value is \"db\"\nexport REDIS_URL=redis://:redis_password@127.0.0.1:6379/0 // only when \"redis\" session store selected\n```\nAnd run application:\n```bash\n$ ./ccd -debug\n```\nThe default port is 8080, test the application in a browser or with curl:\n```bash\n$ curl 127.0.0.1:8080/healthz\n```\nChoose a different port and run more than one copy of **ccd** on the local host. For example:\n```bash\n$ ./ccd -port 8081\n``` \nSpecify some setting before run application: \n```bash\n$ ./ccd -h\nccd is a microservice that collect data from several crypto data providers cryprocompare using its API.\n\nUsage of ccd:\n  -dataprovider string\n        use selected data provider (\"cryptocompare\", \"huobi\", \"kraken\") (default \"cryptocompare\")\n  -debug\n        run the program in debug mode\n  -h    display help\n  -port int\n        set specify port (default 8080)\n  -session string\n        set session store \"db\" or \"redis\" (default \"db\")\n  -timeout int\n        how long to wait for a response from the api server before sending data from the cache (default 5000)\n  -v    display version\n\n```\nSince the release of v2.3.0, the ccd service has moved to API v2, all v1 endpoints have been deprecated and \nare not recommended for use. List of the implemented endpoints v2 API:\n\n| Method | Endpoint               | Description                                                                                         |\n|:------:|:-----------------------|:----------------------------------------------------------------------------------------------------|\n|  GET   | **/healthz**           | check node status                                                                                   |\n|  GET   | **/v2/collect**        | list of all running workers                                                                         |\n|  POST  | **/v2/collect**        | add new worker to collect data for the selected pair                                                |\n|  PUT   | **/v2/collect**        | update pulling interval for the selected pair                                                       |\n| DELETE | **/v2/collect**        | stop and remove worker and collecting data for the selected pair                                    |\n|  GET   | **/v2/symbols**        | list of all symbols presented                                                                       |\n|  POST  | **/v2/symbols**        | add currency symbol                                                                                 |\n|  PUT   | **/v2/symbols**        | update currency symbol                                                                              |\n| DELETE | **/v2/symbols**        | delete currency symbol                                                                              |\n|  GET   | **/v2/price**          | get actual (or cached when dataprovider is unavailable) info for the selected pair                  |\n|  GET   | **/v2/ws**             | websocket connection url, subscribe/unsubscribe to updates or get market data for the selected pair |\n|  GET   | **/v2/ws/subscribe**   | subscribe to collect data for the selected pair                                                     |\n|  GET   | **/v2/ws/unsubscribe** | unsubscribe to stop collect data for the selected pair                                              |\n## Usage examples\nGet actual info about selected pair:\n```bash\n$ curl \"http://localhost:8080/v2/price?fsym=ETH\u0026tsym=USDT\"\n```\nAdd a new worker:\n```bash\n$ curl -X POST -H \"Content-Type: application/json\" -d '{ \"fsym\": \"BTC\", \"tsym\": \"USD\", \"interval\": 60}' \"http://localhost:8080/v2/collect\"\n```\nRemove worker:\n```bash\n$ curl -X DELETE \"http://localhost:8080/v2/collect?fsym=BTC\u0026tsym=USD\u0026interval=60\"\n```\nSubscribe data provider wss channel:\n```bash\n$ curl \"http://localhost:8080/v2/ws/subscribe?fsym=BTC\u0026tsym=USD\"\n```\n## Websocket Server\nConnect to the endpoint **/v2/ws** using any ws client, then you will see server welcome message:\n```bash\n[11:34:54] CONNECTION OPENED\n[11:34:54] HOST =\u003e {\"type\":\"message\",\"message\":\"Welcome to the CCD Server! To get the latest price send request like this: {\\\"type\\\": \\\"price\\\", \\n\\\"pair\\\": { \\\"fsym\\\":\\\"CRYPTO\\\",\\\"tsym\\\":\\\"COMMON\\\" }}\",\"timestamp\":1747643694594}\n```\nTo get the **latest price** send request like this:\n```bash\n[11:42:43] YOU =\u003e {\"type\": \"price\", \"pair\":{\"fsym\":\"BTC\",\"tsym\":\"USDT\"}}\n[11:42:43] HOST =\u003e {\"type\":\"data\",\"data\":{\"id\":0,\"from_sym\":\"BTC\",\"to_sym\":\"USDT\",\"change_24_hour\":0,\"change_pct_24_hour\":0,\"open_24_hour\":106428.5,\"volume_24_hour\":113.86533148,\"low_24_hour\":102161.5,\"high_24_hour\":107000,\"price\":104305.13154,\"supply\":7181,\"mkt_cap\":0,\"last_update\":1747644163933,\"display_data_raw\":\"{\\\"from_symbol\\\":\\\"BTC\\\",\\\"to_symbol\\\":\\\"USDT\\\",\\\"change_24_hour\\\":0,\\\"changepct_24_hour\\\":0,\\\"open_24_hour\\\":106428.5,\\\"volume_24_hour\\\":113.86533148,\\\"volume_24_hour_to\\\":0,\\\"low_24_hour\\\":102161.5,\\\"high_24_hour\\\":107000,\\\"price\\\":104305.13154,\\\"supply\\\":7181,\\\"mkt_cap\\\":0,\\\"last_update\\\":1747644163933}\"},\"timestamp\":1747644163933}\n```\nTo **subscribe** to updates for the selected currency pair that is currently being collected, send request like this:\n```bash\n[11:43:31] YOU =\u003e {\"type\": \"subscribe\", \"pair\":{\"fsym\":\"BTC\",\"tsym\":\"USDT\"}}\n[11:43:31] HOST =\u003e {\"type\":\"message\",\"message\":\"Successfully subscribed on BTC/USDT pair updates\",\"timestamp\":1747644211951}\n```\nTo **unsubscribe** from updates for the selected currency pair, send request like this:\n```bash\n[11:43:53] YOU =\u003e {\"type\": \"unsubscribe\", \"pair\":{\"fsym\":\"BTC\",\"tsym\":\"USDT\"}}\n[11:43:53] HOST =\u003e {\"type\":\"message\",\"message\":\"Successfully unsubscribed from BTC/USDT pair updates\",\"timestamp\":1747644233841}\n```\nTo **ping** server connection (this is not the same as ping on the protocol layer), you can send a request like the\nnext one and wait for the server to respond with your timestamp:\n```bash\n[11:45:10] YOU =\u003e {\"type\": \"ping\", \"timestamp\":1747644233841}\n[11:45:10] HOST =\u003e {\"type\":\"pong\",\"timestamp\":1747644233841}\n```\nTo **close** connection from the server side, send request like this:\n```bash\n[11:47:50] YOU =\u003e {\"type\": \"close\"}\n[11:47:50] HOST =\u003e {\"type\":\"message\",\"message\":\"Connection will be closed at the client's request.\",\"timestamp\":1747644470162}\n```\nThere is no option to directly request a **heartbeat** subscription, the heartbeats will be automatically generated on \nsubscription to any pair. Heartbeat messages are sent approximately once every second in the absence of any other\nchannel updates.\n```bash\n[15:56:53] HOST =\u003e {\"type\":\"heartbeat\",\"timestamp\":1747659413044}\n```\nBy default, ws server read timeout is one minute, but if there are active subscriptions, there is no read timeout.\nThis means that if you want to keep the connection alive without adding a subscription, you should **ping** the ws \nserver or request the **latest price** at intervals less than one minute.\n## Contributing\nContributions are welcome! If you encounter any issues, have suggestions for new features, or want to improve **CCD**, please feel free to open an issue or submit a pull request on the project's GitHub repository.\n## License\n**CCD** is released under the _MIT_ License. See the [LICENSE](https://github.com/streamdp/ccd/blob/main/LICENSE) file for complete license details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamdp%2Fccd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamdp%2Fccd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamdp%2Fccd/lists"}