{"id":52091311,"url":"https://github.com/aether-lang-dev/fast_blob_storage_port","last_synced_at":"2026-08-04T06:01:29.063Z","repository":{"id":374076843,"uuid":"1261533863","full_name":"aether-lang-dev/fast_blob_storage_port","owner":"aether-lang-dev","description":"Fast Blob Storage (FBS) is a s3 compatible blob storage and CDN solution primarly for self hosting. Forked to push Aether's own development","archived":false,"fork":false,"pushed_at":"2026-07-06T19:41:35.000Z","size":18589,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-31T20:21:57.623Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aether-lang-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null,"disclosure":null}},"created_at":"2026-06-06T20:23:40.000Z","updated_at":"2026-07-31T17:35:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/aether-lang-dev/fast_blob_storage_port","commit_stats":null,"previous_names":["aether-lang-dev/fast_blob_storage_port"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/aether-lang-dev/fast_blob_storage_port","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aether-lang-dev%2Ffast_blob_storage_port","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aether-lang-dev%2Ffast_blob_storage_port/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aether-lang-dev%2Ffast_blob_storage_port/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aether-lang-dev%2Ffast_blob_storage_port/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aether-lang-dev","download_url":"https://codeload.github.com/aether-lang-dev/fast_blob_storage_port/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aether-lang-dev%2Ffast_blob_storage_port/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36263568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-08-04T02:00:06.901Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-08-04T06:01:28.166Z","updated_at":"2026-08-04T06:01:29.056Z","avatar_url":"https://github.com/aether-lang-dev.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fbs-core\n\n**fbs-core** is a self-hosted, lightweight object storage ecosystem designed for single-node deployments like homelabs or small to medium projects. It provides a high-performance **AWS S3-compatible API**, acting as the core backend storage engine.\n\n## Overview\n\nThe `fbs-core` backend handles all S3-compatible data operations, disk I/O, and exposes a Management API for external dashboards (such as a Web Dashboard or Terminal Dashboard). \n\n- **Language:** [Aether](https://github.com/aether-lang-org/aether) (compiles to C). Originally Go; see `BUILD-AETHER.md` for the port.\n- **Storage Metadata:** SQLite (WAL mode) via `contrib.sqlite`\n- **Object Storage:** Local File System (atomic write via `std.fs.write_atomic`)\n- **HTTP:** `std.http` server; tests use the [aeocha](https://github.com/aether-lang-org/aeocha) BDD framework\n- **Build:** `aeb` (`cmd/.build.ae`) or `ae build` with an `aether.toml` linking `-lsqlite3`\n\n## Features\n\n### Core S3 API Capabilities\n- **Object Operations:** `PUT`, `GET`, `DELETE`, `HEAD`\n- **Bucket Operations:** `CREATE` (CreateBucket), `LIST` (ListObjectsV2)\n- **Multipart Uploads:** `CreateMultipartUpload`, `UploadPart`, `CompleteMultipartUpload`\n\n### Performance \u0026 Caching\n- **Zero-Copy Transfers:** Utilizes `sendfile()` to stream objects from disk directly to the network.\n- **In-Memory Caching:** Go Memory LRU Cache for hot SQLite metadata to prevent disk I/O bottlenecks.\n- **SQLite Optimization:** Tuned with `synchronous=NORMAL` and `busy_timeout=5000` for high concurrency.\n\n### Security \u0026 Data Integrity\n- **Unified Authentication:** Protected S3 and Management API routes accept both AWS SigV4 and Bearer tokens. Authorization still depends on the stored user role.\n- **First-Start Bootstrap:** On an empty database, create the initial admin once from localhost with `POST /api/setup/bootstrap`.\n- **Path Sanitization:** Secure handling of object keys to prevent path traversal attacks.\n- **Data Consistency:** Writes data to disk first (`.tmp/UUID` \\u2192 rename), then inserts metadata into SQLite. Built-in startup reconciliation to purge orphaned data.\n- **Upload Checksums:** Real-time validation of `Content-MD5` and `x-amz-checksum-*` headers on upload.\n- **Signed Public Reads:** `/public/{bucket}/{key}` is reserved for signed read URLs, so `public` cannot be used as a bucket name.\n\n## First Run\n\nWhen the database has no users, call the loopback-only bootstrap endpoint from the server host:\n\n```bash\ncurl -X POST http://127.0.0.1:9000/api/setup/bootstrap \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"display_name\":\"Admin User\"}'\n```\n\nThe response includes the initial admin Bearer token and SigV4 access key/secret once. Store them immediately; secrets are not returned by later management reads. `GET /api/setup/status` reports whether bootstrap is still required.\n\nManagement and S3 protected routes accept either `Authorization: Bearer ...` or AWS SigV4. SigV4 uses service scope `s3` and region `us-east-1`.\n\n## Project Structure\n\n- `cmd/`: Main application entry point (`main.ae`, `.build.ae`).\n- `internal/`:\n  - `auth/`: Authentication (AWS SigV4 \u0026 Bearer tokens) + token issuance.\n  - `metadata/`: SQLite schema + repositories (buckets, objects, users, multipart, activity, management).\n  - `s3/`: S3 API handlers, route wiring, XML, setup/management API, startup reconcile, `appstate.c`.\n  - `s3compat/`, `responses/`, `publicread/`: region constant, JSON response helper, signed public-read URLs.\n  - `storage/`: Disk storage engine (write/read/delete, path sanitizing, multipart).\n- `lib/`: Reusable Aether libs — `crypto/` (HMAC-SHA256, SigV4), `urlescape/`, `pathutil/`, `uuid/`.\n- `aethertests/`: aeocha test suites mirroring the source tree.\n- `docs/`: Completed project documentation for architecture, configuration, APIs, storage, operations, and development.\n\n## Documentation\n\nThe detailed project documentation lives in [`docs/README.md`](./docs/README.md). For Docker setup and first S3 operations, start with [`docs/quickstart.md`](./docs/quickstart.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faether-lang-dev%2Ffast_blob_storage_port","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faether-lang-dev%2Ffast_blob_storage_port","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faether-lang-dev%2Ffast_blob_storage_port/lists"}