{"id":31756768,"url":"https://github.com/threefoldtech/mycelium-cdn-registry","last_synced_at":"2025-10-09T19:23:45.528Z","repository":{"id":300758813,"uuid":"1004175392","full_name":"threefoldtech/mycelium-cdn-registry","owner":"threefoldtech","description":"Myceliumd CDN metadata registry and tools ","archived":false,"fork":false,"pushed_at":"2025-07-11T12:52:27.000Z","size":201,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-11T15:00:16.274Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/threefoldtech.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-18T08:24:20.000Z","updated_at":"2025-07-11T12:52:12.000Z","dependencies_parsed_at":"2025-07-11T14:10:24.603Z","dependency_job_id":"355b1386-9eff-48b8-bb97-ddf4f34b60c3","html_url":"https://github.com/threefoldtech/mycelium-cdn-registry","commit_stats":null,"previous_names":["threefoldtech/mycelium-cdn-registry"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/threefoldtech/mycelium-cdn-registry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Fmycelium-cdn-registry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Fmycelium-cdn-registry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Fmycelium-cdn-registry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Fmycelium-cdn-registry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/threefoldtech","download_url":"https://codeload.github.com/threefoldtech/mycelium-cdn-registry/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Fmycelium-cdn-registry/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001979,"owners_count":26083243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-09T19:23:40.610Z","updated_at":"2025-10-09T19:23:45.517Z","avatar_url":"https://github.com/threefoldtech.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mycelium CDN Registry\n\nThe Mycelium CDN Registry is a system for storing and retrieving object metadata in a distributed content delivery network. This repo consists of three main components:\n\n1. **cdn-meta**: A library that defines the metadata format for objects\n2. **mycdnctl**: A command-line tool for uploading objects to the CDN\n3. **registry**: A service for storing and retrieving metadata\n\n## System Architecture\n\n```mermaid\ngraph TD\n    Client[Client] --\u003e|Upload file| mycdnctl[mycdnctl CLI]\n    mycdnctl --\u003e|Split \u0026 encrypt| Chunks[Encrypted Chunks]\n    mycdnctl --\u003e|Create metadata| Meta[Metadata]\n    Chunks --\u003e|Erasure coding| Shards[Shards]\n    Shards --\u003e|Distribute| ZDB[0-DB Instances]\n    Meta --\u003e|Encrypt| EncMeta[Encrypted Metadata]\n    EncMeta --\u003e|Store| Registry[Registry Service]\n    Registry --\u003e|Store in| PostgreSQL[(PostgreSQL)]\n    \n    User[End User] --\u003e|Request content| CDN[Mycelium CDN]\n    CDN --\u003e|Fetch metadata| Registry\n    CDN --\u003e|Fetch shards| ZDB\n    CDN --\u003e|Geo-aware loading| User\n```\n\n## Using mycdnctl\n\nThe `mycdnctl` tool is the primary tool needed by users to interract with the mycelium\nCDN. This tool allows you to upload files and directories to the Mycelium CDN. The\nuploaded content is split into chunks, encrypted, and distributed across multiple\n0-DB instances using Reed-Solomon erasure coding for redundancy.\n\n### Installation\n\nEither download an artifact for your platform from the latest release, or build from source.\n\nThe `mycdnctl` tool is written in Rust and can be built from source:\n\n```bash\ncd crates/mycdnctl\ncargo build --release\n```\n\n### Configuration\n\nBefore using `mycdnctl`, you need to create a configuration file (default: `config.toml`) that specifies the 0-DB instances to use for storage. Here's an example configuration:\n\n```toml\n# Number of shards required to recover the data (minimum)\nrequired_shards = 3\n\n# List of 0-DB instances to store shards\n[[zdbs]]\nhost = \"192.168.1.1:9900\"\nnamespace = \"mycelium\"\nsecret = \"optional-password\"\n\n[[zdbs]]\nhost = \"192.168.1.2:9900\"\nnamespace = \"mycelium\"\nsecret = \"optional-password\"\n\n[[zdbs]]\nhost = \"192.168.1.3:9900\"\nnamespace = \"mycelium\"\nsecret = \"optional-password\"\n\n[[zdbs]]\nhost = \"192.168.1.4:9900\"\nnamespace = \"mycelium\"\n```\n\nThe configuration specifies:\n- `required_shards`: The minimum number of shards needed to recover the data\n- `zdbs`: A list of 0-DB instances to store the shards, each with:\n  - `host`: The host address and port of the 0-DB instance\n  - `namespace`: The namespace to use in the 0-DB\n  - `secret`: An optional password for the namespace\n\n### Uploading Files\n\nTo upload a file to the CDN:\n\n```bash\nmycdnctl upload --config config.toml path/to/file.txt\n```\n\nOptional parameters:\n- `--mime`: Specify the MIME type of the file (otherwise inferred)\n- `--name`: Specify a custom name for the file (otherwise uses the filename)\n- `--chunk-size`: Specify the size of chunks (default: 5 MiB, range: 1-5 MiB)\n- `--include-password`: Include 0-DB namespace passwords in the metadata (not recommended for public content)\n- `--registry`: Specify the registry URL (default: https://cdn.mycelium.grid.tf)\n\n### Uploading Directories\n\nTo upload a directory to the CDN:\n\n```bash\nmycdnctl upload --config config.toml path/to/directory\n```\n\nThis will upload all files in the directory (non-recursive) and create a directory metadata object that references all the files.\n\n### Understanding the Output\n\nAfter uploading an object, `mycdnctl` will output a URL for accessing the object, which can be used when running mycelium\nlocally:\n\n```\nObject path/to/file.txt saved. Url: http://[encrypted-hash].[registry-url]/?key=[plaintext-hash]\n```\n\nThe URL contains:\n- The encrypted hash as a subdomain\n- The plaintext hash as a query parameter, which serves as the decryption key\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2Fmycelium-cdn-registry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreefoldtech%2Fmycelium-cdn-registry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2Fmycelium-cdn-registry/lists"}