{"id":51168013,"url":"https://github.com/bartvanbenthem/cn-objectstore","last_synced_at":"2026-06-26T21:30:26.779Z","repository":{"id":358464589,"uuid":"1241315605","full_name":"bartvanbenthem/cn-objectstore","owner":"bartvanbenthem","description":"Provider agnostic object store client for Cloud Native applications. Supports AWS S3, Azure Blob Storage and other S3 compatible backends.","archived":false,"fork":false,"pushed_at":"2026-06-01T12:35:20.000Z","size":31,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-01T14:20:15.912Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bartvanbenthem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-17T08:19:44.000Z","updated_at":"2026-06-01T12:35:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/bartvanbenthem/cn-objectstore","commit_stats":null,"previous_names":["bartvanbenthem/kube-objstore","bartvanbenthem/cn-objectstore"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/bartvanbenthem/cn-objectstore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcn-objectstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcn-objectstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcn-objectstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcn-objectstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bartvanbenthem","download_url":"https://codeload.github.com/bartvanbenthem/cn-objectstore/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcn-objectstore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34834415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-26T02:00:06.560Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-26T21:30:26.374Z","updated_at":"2026-06-26T21:30:26.773Z","avatar_url":"https://github.com/bartvanbenthem.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud Native ObjectStore\n\nA reusable, provider agnostic object store client library for Cloud Native applications and Kubernetes operators.\n\nWraps [`object_store`](https://docs.rs/object_store) with a simpler, higher-level API covering\nCRUD, prefix listing, latest-file retrieval, and background change detection — all driven by\nenvironment variables so the same operator binary works against Azure Blob Storage, AWS S3, MinIO,\nCloudian, or any S3-compatible vendor with no code changes.\n\n---\n\n## Features\n\n- **Azure Blob Storage** and **S3-compatible** backends (AWS S3, MinIO, Cloudian, …)\n- Environment-variable-driven config — no credentials in code\n- Typed error enum — pattern-match on `NotFound`, `StoreBuildError`, etc.\n- Convenience methods: `exists`, `copy`, `rename`, `delete_prefix`, `write_if_absent`, `read_latest`\n- Background prefix watcher with configurable poll interval\n- `Clone`-able client backed by `Arc\u003cdyn ObjectStore\u003e` — share freely across tasks\n- Fully testable via `ObjectStoreClient::from_store` with `object_store::memory::InMemory`\n\n---\n\n## Installation\n\nAdd to your `Cargo.toml`:\n\n```toml\n[dependencies]\nkube-objstore = { path = \"../kube-objstore\" }   # local\n# or once published:\n# kube-objstore = \"0.1.0\"\n```\n\n---\n\n## Environment variables\n\n| Variable                | Required | Description                                              |\n|-------------------------|----------|----------------------------------------------------------|\n| `CLOUD_PROVIDER`        | Yes      | `azure` or `s3`                                         |\n| `OBJECT_STORAGE_BUCKET` | Yes      | Bucket (S3) or container name (Azure)                   |\n| `OBJECT_STORAGE_ACCOUNT`| Yes      | Storage account name (Azure) or access key ID (S3)      |\n| `OBJECT_STORAGE_SECRET` | Yes      | Storage access key (Azure) or secret access key (S3)    |\n| `S3_ENDPOINT_URL`       | No       | Custom S3 endpoint for MinIO / Cloudian / other vendors |\n\nFor AWS S3 the region is sourced from the standard `AWS_DEFAULT_REGION` environment variable or\nthe AWS SDK default chain.\n\n---\n\n## Quick start\n\n```rust\nuse kube_objstore::{ObjectStoreClient, ObjectStoreConfig};\n\n#[tokio::main]\nasync fn main() -\u003e anyhow::Result\u003c()\u003e {\n    let config = ObjectStoreConfig::from_env()?;\n    let client = ObjectStoreClient::new(config).await?;\n\n    // Write\n    client.write(\"snapshots/v1.bin\", b\"my data\").await?;\n\n    // Read\n    if let Some(bytes) = client.read(\"snapshots/v1.bin\").await? {\n        println!(\"Read {} bytes\", bytes.len());\n    }\n\n    // Read the most recently modified object under a prefix\n    if let Some(bytes) = client.read_latest(\"snapshots/\").await? {\n        println!(\"Latest snapshot: {} bytes\", bytes.len());\n    }\n\n    Ok(())\n}\n```\n\n---\n\n## API reference\n\n### `ObjectStoreClient`\n\n| Method | Description |\n|---|---|\n| `new(config)` | Build a client from `ObjectStoreConfig` |\n| `from_store(store, bucket)` | Wrap any `ObjectStore` (useful in tests) |\n| `inner()` | Access the underlying `Arc\u003cdyn ObjectStore\u003e` |\n| **CRUD** | |\n| `write(path, data)` | Write bytes, overwriting any existing object. Returns `ObjectMeta`. |\n| `read(path)` | Read bytes. Returns `None` if not found. |\n| `read_required(path)` | Read bytes. Returns `Err(NotFound)` if not found. |\n| `delete(path)` | Delete an object. Returns `false` if it did not exist. |\n| `metadata(path)` | HEAD request — returns `ObjectMeta` or `None`. |\n| `exists(path)` | `true` if the object exists, no content downloaded. |\n| **Listing** | |\n| `list(prefix)` | All objects under a prefix. |\n| `latest(prefix)` | Metadata of the most recently modified object under a prefix. |\n| `read_latest(prefix)` | Content of the most recently modified object under a prefix. |\n| **Batch / convenience** | |\n| `copy(src, dst)` | Copy an object within the same store. |\n| `rename(src, dst)` | Copy then delete source. |\n| `delete_prefix(prefix)` | Delete all objects under a prefix. Returns count deleted. |\n| `write_if_absent(path, data)` | Write only if the object does not already exist. |\n\n### `ObjectStoreWatcher`\n\nPolls a prefix on a configurable interval and sends a `()` signal on a Tokio `mpsc` channel\nwhenever objects are added or removed. The first poll establishes a baseline without firing.\nThe background task shuts down automatically when all receivers are dropped.\n\n```rust\nuse kube_objstore::{ObjectStoreClient, ObjectStoreConfig, ObjectStoreWatcher};\nuse tokio::sync::mpsc;\nuse std::time::Duration;\n\n#[tokio::main]\nasync fn main() -\u003e anyhow::Result\u003c()\u003e {\n    let config = ObjectStoreConfig::from_env()?;\n    let client = ObjectStoreClient::new(config).await?;\n\n    let (tx, mut rx) = mpsc::channel(8);\n\n    let _handle = ObjectStoreWatcher::new(client, \"configs/\")\n        .with_interval(Duration::from_secs(15))\n        .spawn(tx);\n\n    while let Some(()) = rx.recv().await {\n        println!(\"Prefix changed — trigger reconcile\");\n    }\n\n    Ok(())\n}\n```\n\n### `ObjectStoreConfig`\n\nBuild from environment variables:\n\n```rust\nlet config = ObjectStoreConfig::from_env()?;\n```\n\nOr programmatically with the builder:\n\n```rust\nuse kube_objstore::config::{ObjectStoreConfigBuilder, Provider};\nuse std::time::Duration;\n\nlet config = ObjectStoreConfigBuilder::default()\n    .provider(Provider::S3 { endpoint_url: Some(\"http://minio:9000\".into()) })\n    .bucket(\"my-bucket\")\n    .account(\"minioadmin\")\n    .secret(\"minioadmin\")\n    .poll_interval(Duration::from_secs(60))\n    .build()?;\n```\n\n---\n\n## Error handling\n\nAll methods return `Result\u003cT, ObjectStoreClientError\u003e`. The error enum covers:\n\n```rust\npub enum ObjectStoreClientError {\n    MissingEnvVar(String),\n    UnsupportedProvider(String),\n    StoreBuildError(String),\n    NotFound(String),\n    StoreError(object_store::Error),\n    Io(std::io::Error),\n    Other(anyhow::Error),\n}\n```\n\n---\n\n## Testing\n\nTests use `object_store::memory::InMemory` — no cloud credentials or network needed:\n\n```bash\ncargo test\n```\n\nEnable log output:\n\n```bash\nRUST_LOG=debug cargo test -- --nocapture\n```\n\nTo write your own tests against the library:\n\n```rust\nuse std::sync::Arc;\nuse object_store::memory::InMemory;\nuse kube_objstore::ObjectStoreClient;\n\nfn test_client() -\u003e ObjectStoreClient {\n    ObjectStoreClient::from_store(Arc::new(InMemory::new()), \"test-bucket\")\n}\n```\n\n---\n\n## Provider examples\n\n### Azure Blob Storage\n\n```bash\nexport CLOUD_PROVIDER=azure\nexport OBJECT_STORAGE_BUCKET=my-container\nexport OBJECT_STORAGE_ACCOUNT=mystorageaccount\nexport OBJECT_STORAGE_SECRET=base64accesskey==\n```\n\n### AWS S3\n\n```bash\nexport CLOUD_PROVIDER=s3\nexport OBJECT_STORAGE_BUCKET=my-bucket\nexport OBJECT_STORAGE_ACCOUNT=AKIAIOSFODNN7EXAMPLE\nexport OBJECT_STORAGE_SECRET=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\nexport AWS_DEFAULT_REGION=eu-west-1\n```\n\n### S3-compatible\n\n```bash\nexport CLOUD_PROVIDER=s3\nexport OBJECT_STORAGE_BUCKET=my-bucket\nexport OBJECT_STORAGE_ACCOUNT=minioadmin\nexport OBJECT_STORAGE_SECRET=minioadmin\nexport S3_ENDPOINT_URL=http://minio.minio-ns.svc.cluster.local:9000\n```\n\n---\n\n## Logging\n\nThe library uses [`tracing`](https://docs.rs/tracing). Add a subscriber in your operator's\n`main.rs` to see structured logs from the library:\n\n```rust\ntracing_subscriber::fmt()\n    .with_env_filter(tracing_subscriber::EnvFilter::from_default_env())\n    .init();\n```\n\nThen set `RUST_LOG=kube_objstore=debug` for verbose output.\n\n---\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbartvanbenthem%2Fcn-objectstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbartvanbenthem%2Fcn-objectstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbartvanbenthem%2Fcn-objectstore/lists"}