{"id":15350587,"url":"https://github.com/gaul/are-we-consistent-yet","last_synced_at":"2025-10-25T08:02:51.647Z","repository":{"id":24698954,"uuid":"28110333","full_name":"gaul/are-we-consistent-yet","owner":"gaul","description":"Compare eventual consistency of object stores","archived":false,"fork":false,"pushed_at":"2024-02-05T08:36:07.000Z","size":43,"stargazers_count":172,"open_issues_count":10,"forks_count":11,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-05-08T21:15:14.643Z","etag":null,"topics":["aws-s3","azure","consistency","google-cloud-storage","openstack-swift","s3"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"nirgn975/Angular-Django-Seed-Project","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gaul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-12-16T22:46:28.000Z","updated_at":"2025-04-07T00:28:28.000Z","dependencies_parsed_at":"2024-02-05T10:15:09.329Z","dependency_job_id":null,"html_url":"https://github.com/gaul/are-we-consistent-yet","commit_stats":{"total_commits":34,"total_committers":7,"mean_commits":4.857142857142857,"dds":"0.23529411764705888","last_synced_commit":"c570d8e4366474e881fa87610f90fb228b4a981e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gaul%2Fare-we-consistent-yet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gaul%2Fare-we-consistent-yet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gaul%2Fare-we-consistent-yet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gaul%2Fare-we-consistent-yet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gaul","download_url":"https://codeload.github.com/gaul/are-we-consistent-yet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253149619,"owners_count":21861740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-s3","azure","consistency","google-cloud-storage","openstack-swift","s3"],"created_at":"2024-10-01T11:58:46.223Z","updated_at":"2025-10-25T08:02:51.640Z","avatar_url":"https://github.com/gaul.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Are We Consistent Yet?\n\nObserved and documented eventual consistency of object stores, e.g., Amazon S3,\nOpenStack Swift.\n\n## What is eventual consistency?\n\nTraditional systems provide\n[strong consistency](https://en.wikipedia.org/wiki/Strong_consistency), where\nclients can immediately view updates.\nSome distributed systems relax their consistency model to allow greater\navailability or better performance.\n[Eventual consistency](https://en.wikipedia.org/wiki/Eventual_consistency)\nmanifests itself to clients as stale views of data.\n\n## Observed consistency\n\nWe ran a test in which we wrote (i.e., create, update, delete) an object and\nthen attempted to read the object. Across many trials, we count the number of\ntimes the object was not immediately available. When the object is not\nimmediately found, it's an occurrence of observed eventual consistency. \n\nObserved instances of eventual consistency with a number of operations during\ntests around 17 December 2014 with a 1-byte object size:\n\n| Provider | read after create | read after delete | read after overwrite | list after create | list after delete | number of operations |\n| --- | :---: | :---: | :---: | :---: | :---: | :---: |\n| Amazon S3 (us-standard)         |  * |  * |  * | 16 |  * | 100,000 |\n| Amazon S3 (us-standard\u0026dagger;) | 16 |  8 | 12 |  8 |  3 |   1,000 |\n| Amazon S3 (us-west)             |  - |  1 |  1 |  * |  * | 100,000 |\n| Ceph (DreamObjects)             |  - |  - |  - |  - |  - |   1,000 |\n| Google Cloud Storage            |  - |  - |  - |  2 |  2 |   1,000 |\n| Microsoft Azure Storage         |  - |  - |  - |  - |  - |   1,000 |\n| OpenStack Swift (Rackspace)     |  - | 17 |  3 | 30 | 20 |   1,000 |\n\nLegend:\n\n* \\- zero observed instances\n* \\* zero observed instances but expect non-zero\n* \u0026dagger; writing to N. Virginia and reading from Pacific Northwest\n\n## Documented consistency\n\n* [Amazon S3](http://aws.amazon.com/s3/faqs/#What_data_consistency_model_does_Amazon_S3_employ) - buckets in the US Standard region provide eventual consistency.  Buckets in all other regions provide read-after-write consistency for PUTs of new objects and eventual consistency for overwrite PUTs and DELETEs.\n* [Ceph](https://ceph.com/wp-content/uploads/2016/08/weil-rados-pdsw07.pdf) - provides well-defined safety semantics and strong consistency guarantees.\n* [Google Cloud Storage](https://cloud.google.com/storage/docs/concepts-techniques#consistency) - provides strong global consistency for all read-after-write, read-after-update, and read-after-delete operations, including both data and metadata. Bucket and object listing are also strongly consistent.\n* [Microsoft Azure Storage](http://azure.microsoft.com/blog/2014/09/08/managing-concurrency-in-microsoft-azure-storage-2/) - was designed to embrace a strong consistency model which guarantees that when the Storage service commits a data insert or update operation all further accesses to that data will see the latest update.\n* [OpenStack Swift](http://docs.openstack.org/developer/swift/overview_architecture.html#updaters) - For example, suppose a container server is under load and a new object is put in to the system. The object will be immediately available for reads as soon as the proxy server responds to the client with success. However, the container server did not update the object listing, and so the update would be queued for a later update. Container listings, therefore, may not immediately contain the object.  [Additional reference](http://lists.openstack.org/pipermail/openstack-dev/2014-June/038881.html).\n\n## References\n\n* [Apache jclouds](https://jclouds.apache.org/) - provides object storage support\n* [A Middleware Guaranteeing Client-Centric Consistency on Top of Eventually\nConsistent Datastores](http://www.aifb.kit.edu/images/4/44/Ic2e2013consistency.pdf)\n* [Benchmarking Eventual Consistency: Lessons Learned from Long-Term Experimental Studies](http://www.aifb.kit.edu/images/8/8d/Ic2e2014.pdf)\n* [Eventual Consistency: How soon is eventual?](https://www.aifb.kit.edu/images/1/17/How_soon_is_eventual.pdf)\n* [Eventual Consistency Today: Limitations, Extensions, and Beyond](https://queue.acm.org/detail.cfm?id=2462076)\n* [Probabilistically Bounded Staleness](http://pbs.cs.berkeley.edu/) - includes discussion of Riak and Cassandra\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaul%2Fare-we-consistent-yet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgaul%2Fare-we-consistent-yet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaul%2Fare-we-consistent-yet/lists"}