{"id":18016548,"url":"https://github.com/webassembly/wasi-blobstore","last_synced_at":"2025-08-01T00:38:22.766Z","repository":{"id":105883515,"uuid":"536176070","full_name":"WebAssembly/wasi-blobstore","owner":"WebAssembly","description":null,"archived":false,"fork":false,"pushed_at":"2025-01-08T02:14:50.000Z","size":87,"stargazers_count":19,"open_issues_count":13,"forks_count":12,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-03-22T06:41:48.455Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WebAssembly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-13T14:51:56.000Z","updated_at":"2025-03-12T15:00:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"1857c935-04a5-4979-9c37-75f394ee346d","html_url":"https://github.com/WebAssembly/wasi-blobstore","commit_stats":null,"previous_names":["webassembly/wasi-blobstore"],"tags_count":1,"template":false,"template_full_name":"WebAssembly/wasi-proposal-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAssembly%2Fwasi-blobstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAssembly%2Fwasi-blobstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAssembly%2Fwasi-blobstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAssembly%2Fwasi-blobstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WebAssembly","download_url":"https://codeload.github.com/WebAssembly/wasi-blobstore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245713002,"owners_count":20660331,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T04:18:26.567Z","updated_at":"2025-08-01T00:38:22.748Z","avatar_url":"https://github.com/WebAssembly.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# WASI Blob Store\n\nA proposed [WebAssembly System Interface](https://github.com/WebAssembly/WASI) API.\n\n### Current Phase\n\nPhase 1\n\n### Champions\n\n- Jiaxiao Zhou\n- Kevin Hoffman\n- David Justice\n- Dan Chiarlone\n\n### Phase 4 Advancement Criteria\n\n* [ ] At least two independent production implementations.\n* [ ] At least two cloud provider implementations.\n* [ ] Implementations available for at least Windows, Linux \u0026 MacOS.\n* [ ] A test suite that passes on the platforms and implementations mentioned above.\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Goals](#goals)\n- [Non-goals](#non-goals)\n- [API walk-through](#api-walk-through)\n  - [Process Blob Contents](#process-blob-contents)\n  - [Write to a Blob Stream](#write-to-a-blob-stream)\n  - [List Objects within a Container](#list-objects-within-a-container)\n- [Detailed Design Discussion](#detailed-design-discussion)\n  - [Handling Large Files](#handling-large-files)  \n- [Stakeholder Interest \u0026 Feedback](#stakeholder-interest--feedback)\n\n### Introduction\n\n**Blob storage** is a type of data storage used for unstructured data such as images, videos, documents, backups, etc. Blob storage is also commonly referred to as _object storage_. The term **blob** is actually an acronym for **B**inary **L**arge **OB**ject but can be used to refer to all types of unstructured data.\n\nWithin the context of this proposal, blob storage refers to granting WebAssembly components access to a common abstraction of a blob store. Examples of blob storage services include [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/), [AWS S3](https://aws.amazon.com/s3/), or [Google Cloud Storage](https://cloud.google.com/storage), but can be anything that can be represented as unstructured binary data that conforms to the interface, including file systems.\n\n### Goals\n\nThe primary goal of this API is to provide a common abstraction for blob storage services, so that WebAssembly components can be written to work with any implementation, without needing to know the details of the underlying service.\n\nAdditionally, components using this API will be unable to tell the difference between a blob storage service and a file system, allowing them to be written to work with either and will not need to configure the store within the component code.\n\n### Non-goals\nThe following is a list of goals explicitly out of scope for this API specification:\n\n* Cover all edge cases and niche scenarios\n* Configuration of service access\n* Secrets management\n* Definition or direct use of networking protocols\n* Monitoring and Observability\n\n### API walk-through\n\nThe following sections provide an overview of how this API might be used. Note that while the samples are in Rust, any language targetable by wasm components via code generation should work.\n\n#### Process Blob Contents\nThis example shows obtaining a reference to the container and the desired object within that container, and then using `read_into` in a loop to access the blob contents.\n\n```rust\n// Count the number of lines in an object\n// For simplicity, assume the object contains ascii text and lines end in '\\n'\nfn count_lines(store: \u0026impl BlobStore, id: \u0026ObjectId) -\u003e Result\u003cusize, Error\u003e {\n  let mut stream = store.get_container(\u0026id.container_name)?.read_object(\u0026id.object_name)?;\n  let mut buf = [0u8; 4096];\n  let mut num_lines = 0;\n  while let Some(bytes) = stream.read_into(\u0026mut buf)? {\n    num_lines += buf[0..bytes as usize].iter().filter(|\u0026c| *c == b'\\n').count();\n  }\n  Ok(num_lines)\n}\n```\n\n#### Write to a Blob Stream\nThe following code sample shows how to obtain a reference to a container and a writable reference to a stream that will be stored in a blob.\n\n```rust\n// Download a file from an http url and save it to the blob store.\n// When completed, returns metadata for the new object\nfn download(url: \u0026str, store: \u0026impl BlobStore, id: \u0026ObjectId) -\u003e Result\u003cObjectMetadata, Error\u003e {\n    let container = store.get_container(\u0026id.container_name)?;\n    // retrieve a url via wasi-http fetch() method\n    // the http service hasn't been defined yet, but assume its fetch() method returns a readable stream.\n    let mut download_stream = http::fetch(url)?;\n    let mut buf = [0u8; 4096];\n    let mut save_stream = container.write_object(\u0026id.object_name)?;\n    while let Some(bytes) = download_stream.read_into(\u0026mut buf)? {\n        save_stream.write(\u0026buf[0..bytes as usize])?;\n    }\n    // ensure stream is flushed and object is created, before we query the metadata\n    save_stream.close()?;\n    let obj = container.object_info(\u0026id.object_name)?;\n    Ok(obj)\n}\n```\n\n#### List Objects within a Container\nThe following code shows how to enumerate the objects within a container.\n\n```rust\n// suppose the \"logs\" container has objects with names that start with a timestamp, like \"2022-01-01-12-00-00.log\"\n// for every day that activity occurred. To count the number of logs from january 2022, call:\n//    `count_objects_with_prefix(store, \"logs\", \"2022-01\")`\nfn count_objects_with_prefix(store: \u0026impl BlobStore, container_name: \u0026str, prefix: \u0026str) -\u003e Result\u003cusize,Error\u003e {\n  let container = store.get_container(container_name)?;\n  let names = container.list_objects()?;\n  let count = names.filter(|n| n.starts_with(prefix)).count();\n  Ok(count)\n}\n```\n\n### Detailed Design Discussion\n\nSee the [wit files](./wit)\n\n#### Handling Large Files\n\nHandling large files may require changes to the API that are not accounted for in this current proposal. If a component attempts to allocate more memory than the host is willing to give it, then the component could be terminated by the host runtime and the processing will fail.\n\nAdditionally, if a component spends too long processing a file, either processing one large blob or by processing many small blobs in a tight loop, then the component could again be shut off because it consumed too many resources or too much time.\n\nHow to handle this and whether the `callback` approach belongs in the blob store API or in a lower level [wasm-io API](https://github.com/WebAssembly/wasi-io/issues/31) is still under discussion.\n\n### Stakeholder Interest \u0026 Feedback\n\nTODO before entering Phase 3.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebassembly%2Fwasi-blobstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwebassembly%2Fwasi-blobstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebassembly%2Fwasi-blobstore/lists"}