{"id":15061750,"url":"https://github.com/application-research/delta-edge","last_synced_at":"2026-01-29T12:01:15.294Z","repository":{"id":65349711,"uuid":"586746054","full_name":"application-research/delta-edge","owner":"application-research","description":"Edge Gateway + Storage deal making using Delta","archived":false,"fork":false,"pushed_at":"2023-07-13T04:51:29.000Z","size":13528,"stargazers_count":11,"open_issues_count":28,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-13T20:40:31.215Z","etag":null,"topics":["delta","ipfs","uploader"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/application-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-09T05:58:34.000Z","updated_at":"2025-09-13T08:55:32.000Z","dependencies_parsed_at":"2024-06-19T19:00:04.154Z","dependency_job_id":"f8d8c4e5-92d5-4a69-abd2-2ada7600b0e0","html_url":"https://github.com/application-research/delta-edge","commit_stats":null,"previous_names":["application-research/delta-edge","application-research/edge-ur"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/application-research/delta-edge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Fdelta-edge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Fdelta-edge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Fdelta-edge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Fdelta-edge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/application-research","download_url":"https://codeload.github.com/application-research/delta-edge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Fdelta-edge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28877037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T10:31:27.438Z","status":"ssl_error","status_checked_at":"2026-01-29T10:31:01.017Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["delta","ipfs","uploader"],"created_at":"2024-09-24T23:24:36.823Z","updated_at":"2026-01-29T12:01:15.281Z","avatar_url":"https://github.com/application-research.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EdgeUR (Upload/Retrieve)\n\n[![CodeQL](https://github.com/application-research/edge-ur/actions/workflows/codeql.yml/badge.svg)](https://github.com/application-research/edge-ur/actions/workflows/codeql.yml)\n\n*Edge is currently under heavy development.*\n\n## Goal/Purpose\nDedicated light node to upload and retrieve their CIDs. To do this, we decoupled the upload and retrieval aspect from the Estuary API node so we can create a node that can live on the \"edge\" closer to the customer.\n\nBy decoupling this to a light node, we achieve the following:\n- dedicated node assignment for each customer. The customer or user can now launch an edge node and use it for both uploading to Estuary and retrieval using the same API keys issued from Estuary.\n- switches the upload protocol. The user still needs to upload via HTTP but the edge node will transfer the file over to a delta node to make deals.\n\n![image](https://user-images.githubusercontent.com/4479171/227985970-58bfead8-0906-4f2e-b7ae-b314508ee3e5.png)\n\n## Features\n- Only supports online/e2e verified deals for now.\n- Accepts concurrent uploads (small to large)\n- Stores the CID and content on the local blockstore using whypfs\n- Save the data on local sqlite DB\n- retries the storage deals if it fails. Uses delta `auto_retry` feature.\n- periodically checks the status of the deals and update the database.\n- For 32GB and above, the node will split the file into 32GB chunks and make deals for each chunk and car them. **[WIP]**\n\n## Quick how to\n\n### Upload a file with a given miner\nWe have a working live server that you can use. To run basic upload, please get API key first. You can get one here https://auth.estuary.tech/register-new-token\n\n```\ncurl --location 'http://localhost:3000/api/v1/content/add' \\\n--header 'Authorization: Bearer [APIKEY]' \\\n--form 'data=@\"/file.zip\"' \\\n--form 'miners=\"f0137168,f0717969\"' // list of miners (optional)\n{\n    \"status\": \"success\",\n    \"message\": \"File uploaded and pinned successfully to miners. Please take note of the ids.\",\n    \"contents\": [\n        {\n            \"ID\": 53,\n            \"name\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y.zip\",\n            \"size\": 1157548,\n            \"cid\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y\",\n            \"delta_content_id\": 0,\n            \"delta_node_url\": \"https://node.delta.store\",\n            \"status\": \"pinned\",\n            \"last_message\": \"\",\n            \"miner\": \"f0137168\",\n            \"replication\": 0,\n            \"created_at\": \"2023-04-22T14:40:43.10918+02:00\",\n            \"updated_at\": \"2023-04-22T14:40:43.10918+02:00\"\n        },\n        {\n            \"ID\": 54,\n            \"name\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y.zip\",\n            \"size\": 1157548,\n            \"cid\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y\",\n            \"delta_content_id\": 0,\n            \"delta_node_url\": \"https://node.delta.store\",\n            \"status\": \"pinned\",\n            \"last_message\": \"\",\n            \"miner\": \"f0717969\",\n            \"replication\": 0,\n            \"created_at\": \"2023-04-22T14:40:43.112161+02:00\",\n            \"updated_at\": \"2023-04-22T14:40:43.112161+02:00\"\n        }\n    ]\n}\n```\n\n### Check status\nGet the content id and use the following call\n```\ncurl --location 'http://localhost:3000/api/v1/status/53' \\\n--header 'Authorization: Bearer [APIKEY]'\n{\n    \"content\": {\n        \"ID\": 53,\n        \"name\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y.zip\",\n        \"size\": 1157548,\n        \"cid\": \"bafybeicgdjdvwes3e5aaicqljrlv6hpdfsducknrjvsq66d4gsvepolk6y\",\n        \"delta_content_id\": 67547,\n        \"delta_node_url\": \"https://node.delta.store\",\n        \"status\": \"uploaded-to-delta\",\n        \"last_message\": \"\",\n        \"miner\": \"f0137168\",\n        \"replication\": 0,\n        \"created_at\": \"2023-04-22T14:40:43.10918+02:00\",\n        \"updated_at\": \"2023-04-22T14:40:46.564971+02:00\"\n    }\n}\n```\n### View the file using the gateway url\n```\nhttp://localhost:1313/gw/\u003ccid\u003e\nhttp://localhost:1313/gw/content/\u003ccontent_id\u003e\n```\n\n## Getting Started\nTo get started, follow the guide [here](docs/README.md).\n\n# Author\nProtocol Labs Outercore Engineering\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplication-research%2Fdelta-edge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapplication-research%2Fdelta-edge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplication-research%2Fdelta-edge/lists"}