{"id":13763360,"url":"https://github.com/henryr/cap-faq","last_synced_at":"2025-05-10T16:32:28.510Z","repository":{"id":8394481,"uuid":"9972588","full_name":"henryr/cap-faq","owner":"henryr","description":"The CAP FAQ","archived":false,"fork":false,"pushed_at":"2023-07-14T23:44:41.000Z","size":43,"stargazers_count":795,"open_issues_count":6,"forks_count":69,"subscribers_count":39,"default_branch":"master","last_synced_at":"2024-11-16T22:32:59.465Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/henryr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-05-10T02:04:15.000Z","updated_at":"2024-11-09T14:23:06.000Z","dependencies_parsed_at":"2024-11-16T22:31:38.089Z","dependency_job_id":"01653cd8-bed6-41b2-ad63-16b8bca09d4d","html_url":"https://github.com/henryr/cap-faq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henryr%2Fcap-faq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henryr%2Fcap-faq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henryr%2Fcap-faq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henryr%2Fcap-faq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/henryr","download_url":"https://codeload.github.com/henryr/cap-faq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253445837,"owners_count":21909870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T15:00:43.270Z","updated_at":"2025-05-10T16:32:28.204Z","avatar_url":"https://github.com/henryr.png","language":null,"funding_links":[],"categories":["System-Design related topics-Some very useful articles","Others","Scalability"],"sub_categories":["Other important resources","Theory:"],"readme":"---\nredirect_to: \"http://www.the-paper-trail.org/page/cap-faq\"\n---\n\n# The CAP FAQ\n\n    Version 1.0, May 9th 2013\n    By: Henry Robinson / henry.robinson@gmail.com / @henryr\n\u003cpre\u003e\u003ca href=\"http://the-paper-trail.org/\"\u003ehttp://the-paper-trail.org/\u003c/a\u003e\u003c/pre\u003e\n\n## 0. What is this document?\n\nNo subject appears to be more controversial to distributed systems\nengineers than the oft-quoted, oft-misunderstood CAP theorem. The\npurpose of this FAQ is to explain what is known about CAP, so as to\nhelp those new to the theorem get up to speed quickly, and to settle\nsome common misconceptions or points of disagreement.\n\nOf course, there's every possibility I've made superficial or\ncompletely thorough mistakes here. Corrections and comments are\nwelcome: \u003ca href=\"mailto:henry.robinson+cap@gmail.com\"\u003elet me have\nthem\u003c/a\u003e.\n\nThere are some questions I still intend to answer. For example\n\n* *What's the relationship between CAP and performance?*\n* *What does CAP mean to me as an engineer?*\n* *What's the relationship between CAP and ACID?*\n\nPlease suggest more.\n\n## 1. Where did the CAP Theorem come from?\n\nDr. Eric Brewer gave a keynote speech at the Principles of Distributed\nComputing conference in 2000 called 'Towards Robust Distributed\nSystems' [1]. In it he posed his 'CAP Theorem' - at the time unproven - which\nillustrated the tensions between being correct and being always\navailable in distributed systems.\n\nTwo years later, Seth Gilbert and Professor Nancy Lynch - researchers\nin distributed systems at MIT - formalised and proved the conjecture\nin their paper “Brewer's conjecture and the feasibility of consistent,\navailable, partition-tolerant web services” [2].\n\n[1] http://www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-keynote.pdf\n[2] https://users.ece.cmu.edu/~adrian/731-sp04/readings/GL-cap.pdf\n\n## 2. What does the CAP Theorem actually say?\n\nThe CAP Theorem (henceforth 'CAP') says that it is impossible to build\nan implementation of read-write storage in an asynchronous network that\nsatisfies all of the following three properties:\n\n* Availability - will a request made to the data store always eventually complete?\n* Consistency - will all executions of reads and writes seen by all nodes be _atomic_ or _linearizably_ consistent?\n* Partition tolerance - the network is allowed to drop any messages.\n\nThe next few items define any unfamiliar terms.\n\nMore informally, the CAP theorem tells us that we can't build a\ndatabase that both responds to every request and returns the results\nthat you would expect every time. It's an _impossibility_ result - it\ntells us that something we might want to do is actually provably out\nof reach. It's important now because it is directly applicable to the\nmany, many distributed systems which have been and are being built in\nthe last few years, but it is not a death knell: it does _not_ mean\nthat we cannot build useful systems while working within these\nconstraints.\n\nThe devil is in the details however. Before you start crying 'yes, but\nwhat about...', make sure you understand the following about exactly\nwhat the CAP theorem does and does not allow.\n\n## 3. What is 'read-write storage'?\n\nCAP specifically concerns itself with a theoretical\nconstruct called a _register_. A register is a data structure with two\noperations:\n\n* set(X) sets the value of the register to X\n* get() returns the last value set in the register\n\nA key-value store can be modelled as a collection of registers. Even\nthough registers appear very simple, they capture the essence of what\nmany distributed systems want to do - write data and read it back.\n\n## 4. What does _atomic_ (or _linearizable_) mean?\n\nAtomic, or linearizable, consistency is a guarantee about what values\nit's ok to return when a client performs get() operations. The idea is\nthat the register appears to all clients as though it ran on just one\nmachine, and responded to operations in the order they arrive.\n\nConsider an execution consisting the total set of operations performed\nby all clients, potentially concurrently. The results of those\noperations must, under atomic consistency, be equivalent to a single\nserial (in order, one after the other) execution of all operations.\n\nThis guarantee is very strong. It rules out, amongst other guarantees,\n_eventual consistency_, which allows a delay before a write becomes\nvisible. So under EC, you might have:\n\n    set(10), set(5), get() = 10\n\nBut this execution is invalid under atomic consistency.\n\nAtomic consistency also ensures that external communication about the\nvalue of a register is respected. That is, if I read X and tell you\nabout it, you can go to the register and read X for yourself. It's\npossible under slightly weaker guarantees (_serializability_ for\nexample) for that not to be true. In the following we write A:\u003cset or\nget\u003e to mean that client A executes the following operation.\n\n    B:set(5), A:set(10), A:get() = 10, B:get() = 10\n\nThis is an atomic history. But the following is not:\n\n    B:set(5), A:set(10), A:get() = 10, B:get() = 5\n\neven though it is equivalent to the following serial history:\n\n    B:set(5), B:get() = 5, A:set(10), A:get() = 10\n\nIn the second example, if A tells B about the value of the register\n(10) after it does its get(), B will falsely believe that some\nthird-party has written 5 between A:get() and B:get(). If external\ncommunication isn't allowed, B cannot know about A:set, and so sees a\nconsistent view of the register state; it's as if B:get really did\nhappen before A:set.\n\nWikipedia [1] has more information. Maurice Herlihy's original paper\nfrom 1990 is available at [2].\n\n[1] http://en.wikipedia.org/wiki/Linearizability\n[2] http://cs.brown.edu/~mph/HerlihyW90/p463-herlihy.pdf\n\n## 5. What does _asynchronous_ mean?\n\nAn _asynchronous_ network is one in which there is no bound on how\nlong messages may take to be delivered by the network or processed by\na machine. The important consequence of this property is that there's\nno way to distinguish between a machine that has failed, and one whose\nmessages are getting delayed.\n\n## 6. What does _available_ mean?\n\nA data store is available if and only if all get and set requests\neventually return a response that's part of their specification. This\ndoes _not_ permit error responses, since a system could be trivially\navailable by always returning an error.\n\nThere is no requirement for a fixed time bound on the response, so the\nsystem can take as long as it likes to process a request. But the\nsystem must eventually respond.\n\nNotice how this is both a strong and a weak requirement. It's strong\nbecause 100% of the requests must return a response (there's no\n'degree of availability' here), but weak because the response can take\nan unbounded (but finite) amount of time.\n\n## 7. What is a _partition_?\n\nA partition is when the network fails to deliver some messages to one\nor more nodes by losing them (not by delaying them - eventual delivery\nis not a partition).\n\nThe term is sometimes used to refer to a period during which _no_\nmessages are delivered between two sets of nodes. This is a more\nrestrictive failure model. We'll call these kinds of partitions _total\npartitions_.\n\nThe proof of CAP relied on a total partition. In practice,\nthese are arguably the most likely since all messages may flow through\none component; if that fails then message loss is usually total\nbetween two nodes.\n\n## 8. Why is CAP true?\n\nThe basic idea is that if a client writes to one side of a partition,\nany reads that go to the other side of that partition can't possibly\nknow about the most recent write. Now you're faced with a choice: do\nyou respond to the reads with potentially stale information, or do you\nwait (potentially forever) to hear from the other side of the\npartition and compromise availability?\n\nThis is a proof by _construction_ - we demonstrate a single situation\nwhere a system cannot be consistent and available. One reason that CAP\ngets some press is that this constructed scenario is not completely\nunrealistic. It is not uncommon for a total partition to occur if\nnetworking equipment should fail.\n\n## 9. When does a system have to give up C or A?\n\nCAP only guarantees that there is _some_ circumstance in which a\nsystem must give up either C or A. Let's call that circumstance a\n_critical condition_.  The theorem doesn't say anything about how\nlikely that critical condition is. Both C and A are strong guarantees:\nthey hold only if 100% of operations meet their requirements. A single\ninconsistent read, or unavailable write, invalidates either C or\nA. But until that critical condition is met, a system can be happily\nconsistent _and_ available and not contravene any known laws.\n\nSince most distributed systems are long running, and may see millions\nof requests in their lifetime, CAP tells us to be\ncautious: there's a good chance that you'll realistically hit one of\nthese critical conditions, and it's prudent to understand how your\nsystem will fail to meet either C or A.\n\n## 10. Why do some people get annoyed when I characterise my system as CA?\n\nBrewer's keynote, the Gilbert paper, and many other treatments, places\nC, A and P on an equal footing as desirable properties of an\nimplementation and effectively say 'choose two!'. However, this is\noften considered to be a misleading presentation, since you cannot\nbuild - or choose! - 'partition tolerance': your system either might experience\npartitions or it won't.\n\nCAP is better understood as describing the tradeoffs you\nhave to make when you are building a system that may suffer\npartitions. In practice, this is every distributed system: there is no\n100% reliable network. So (at least in the distributed context) there\nis no realistic CA system. You will potentially suffer partitions,\ntherefore you must at some point compromise C or A.\n\nTherefore it's arguably more instructive to rewrite the theorem as the\nfollowing:\n\n\u003cpre\u003ePossibility of Partitions =\u003e Not (C and A)\u003c/pre\u003e\n\ni.e. if your system may experience partitions, you can not always be C\nand A.\n\nThere are some systems that won't experience partitions - single-site\ndatabases, for example. These systems aren't generally relevant to the\ncontexts in which CAP is most useful. If you describe your distributed\ndatabase as 'CA', you are misunderstanding something.\n\n## 11. What about when messages don't get lost?\n\nA perhaps surprising result from the Gilbert paper is that no\nimplementation of an atomic register in an asynchronous network can be\navailable at all times, and consistent only when no messages are lost.\n\nThis result depends upon the asynchronous network property, the idea\nbeing that it is impossible to tell if a message has been dropped and\ntherefore a node cannot wait indefinitely for a response while still\nmaintaining availability, however if it responds too early it might be\ninconsistent.\n\n## 12. Is my network really asynchronous?\n\nArguably, yes. Different networks have vastly differing characteristics.\n\nIf\n\n* Your nodes do not have clocks (unlikely) or they have clocks that may drift apart (more likely)\n* System processes may arbitrarily delay delivery of a message (due to retries, or GC pauses)\n\nthen your network may be considered _asynchronous_.\n\nGilbert and Lynch also proved that in a _partially-synchronous_\nsystem, where nodes have shared but not synchronised clocks and there\nis a bound on the processing time of every message, that it is still\nimpossible to implement available atomic storage.\n\nHowever, the result from #8 does _not_ hold in the\npartially-synchronous model; it is possible to implement atomic\nstorage that is available all the time, and consistent when all\nmessages are delivered.\n\n## 13. What, if any, is the relationship between FLP and CAP?\n\nThe Fischer, Lynch and Patterson theorem ('FLP') (see [1] for a link\nto the paper and a proof explanation) is an extraordinary\nimpossibility result from nearly thirty years ago, which determined\nthat the problem of consensus - having all nodes agree on a common\nvalue - is unsolvable in general in asynchronous networks where one\nnode might fail.\n\nThe FLP result is not directly related to CAP, although they are\nsimilar in some respects. Both are impossibility results about\nproblems that may not be solved in distributed systems. The devil is\nin the details. Here are some of the ways in which FLP is different\nfrom CAP:\n\n* FLP permits the possibility of one 'failed' node which is totally\n  partitioned from the network and does not have to respond to\n  requests.\n* Otherwise, FLP does not allow message loss; the network\n  is only asynchronous but not lossy.\n* FLP deals with _consensus_, which is a similar but different problem\n  to _atomic storage_.\n\nFor a bit more on this topic, consult the blog post at [2].\n\n[1] http://the-paper-trail.org/blog/a-brief-tour-of-flp-impossibility/\n[2] https://www.the-paper-trail.org/post/2012-03-25-flp-and-cap-arent-the-same-thing/\n\n## 14. Are C and A 'spectrums'?\n\nIt is possible to relax both consistency and availability guarantees\nfrom the strong requirements that CAP imposes and get useful\nsystems. In fact, the whole point of CAP is that you\n_must_ do this, and any system you have designed and built relaxes one\nor both of these guarantees. The onus is on you to figure out when,\nand how, this occurs.\n\nReal systems choose to relax availability - in the case of systems for\nwhom consistency is of the utmost importance, like ZooKeeper. Other\nsystems, like Amazon's Dynamo, relax consistency in order to maintain\nhigh degrees of availability.\n\nOnce you weaken any of the assumptions made in the statement or proof\nof CAP, you have to start again when it comes to proving an\nimpossibility result.\n\n## 15. Is a failed machine the same as a partitioned one?\n\nNo. A 'failed' machine is usually excused the burden of having to\nrespond to client requests. CAP does not allow any machines to fail\n(in that sense it is a strong result, since it shows impossibility\nwithout having any machines fail).\n\nIt is possible to prove a similar result about the impossibility of\natomic storage in an asynchronous network when there are up to N-1\nfailures. This result has ramifications about the tradeoff between how\nmany nodes you write to (which is a performance concern) and how fault\ntolerant you are (which is a reliability concern).\n\n## 16. Is a slow machine the same as a partitioned one?\n\nNo: messages eventually get delivered to a slow machine, but they\nnever get delivered to a totally partitioned one. However, slow\nmachines play a significant role in making it very hard to distinguish\nbetween lost messages (or failed machines) and a slow machine. This\ndifficulty is right at the heart of why CAP, FLP and other results are\ntrue.\n\n## 17. Have I 'got around' or 'beaten' the CAP theorem?\n\nNo. You might have designed a system that is not heavily affected by\nit. That's good.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenryr%2Fcap-faq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhenryr%2Fcap-faq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenryr%2Fcap-faq/lists"}