Let’s imagine a Berliner (a person, not the donut) living in Mitte. The general practitioner (GP) this Berliner goes to as a patient has two offices: One in Kreuzberg, and one in Pankow. This Berliner has choices when he (let’s assume it’s me) wants to visit the GP. If he has other errands to do in Kreuzberg, he can visit the one in Kreuzberg. Otherwise, it is almost always faster to go to the Pankow office.

We’ll get back to this example later on. But now…

Let’s begin with good old CAP Theorem

CAP is the theorem where you get to choose 2 out of 3 in a distributed system, right?

  • Consistency: Linearizability of the effects of incoming requests across all nodes
  • Availability: Frequency of being available to serve incoming requests
  • Partitioning of Network: Case of two or more nodes not talking to each other

It is mostly about choosing between Consistency or Availability in case of a network partition, though. Because network partitioning is not something we can choose; it’s something that happens. So it eventually comes down to what you’d prioritize in case two parts of your distributed system cannot talk to each other (i.e., network partitioning): Do you stop serving requests to prioritize consistency or do you keep serving requests even though some part of your system may be serving stale data?

At the doctor’s office

Today, our Berliner decided to go to the office in Kreuzberg. He checks in, spends about an hour in the waiting room, and finally sees the doctor. At the end of the visit, the doctor decides on a prescription but first calls the other office (the one in Pankow) to get the latest medications prescribed for the Berliner there. The doctor wants to know if the new medication she’s prescribing does not have any interactions with ones that were prescribed by the Pankow office.

But there is a problem: The call does not go through. (Network partitioning)

What would the doctor do?

  • She can prescribe now and check for medication interactions later. Availability over Consistency; so this is AP.
  • Or she can tell he patient to come back later when the call can be made. Consistency over Availability; and this is CP.

There is one problem with this approach…

PACELC

…and that is, network partitioning is not that common and CAP theorem is optimizing for rare cases. Let’s say there is a 99.9% chance of that the network is healthy and two distributed nodes can talk to each other. In that case the CAP theorem optimizes for 0.1%. What about the other 99.9%? Being an extension to CAP, PACELC answers to that need. Basically it goes like this:

In case of a Network (P)artition, choose (A)vailability or (C)onsistency. (E)lse, choose (L)atency or (C)onsistency.

Both parts of the conditional may seem similar. They both have consistency after all and the only thing that changes is the axis where Availability sits. We just swap it with Latency, because if our system will be available anyway if there is no network partitioning.

I think the value of PACELC comes from this question: Under normal circumstances, should your system or business favor consistency over latency, or vice versa?

Back at the doctor’s office

Drug interactions could lead to pretty serious health effects. Can you, as a doctor, bear spending 2 minutes on the phone (latency) for each patient to avoid a possible drug interaction (consistency)? The answer is probably yes. The business case here favors consistency over latency. You want to know possible drug interactions to avoid serious side effects for all patients. So the doctor’s office is more likely a PC/EC system: In case of a network partition, favor consistency. Else, favor consistency.

Different businesses require different priorities

If we were talking about a completely different system, say, a restaurant booking system, our priorities would probably be different.

Let’s say there are multiple people (John and Jane) who answer phone calls for reservation requests. Each of them keep their own book for noting down reservations. A customer calls the restaurant for a 4-people booking tonight, John answers the call and checks his book. There is availability. However, Jane is not around, so he cannot check Jane’s book to confirm availability. What to do?

  • Should John book the customer in and confirm with Jane later? AP
  • Or should John tell the customer to call again in 5 minutes so Jane will be around? CP

I’d say the former is the likelier case here. We can always confirm later, but it may not make sense to defer the decision because the restaurant may lose a potential customer.

What if under normal circumstances? Let’s imagine Jane is within John’s reach and he can go and ask Jane while keeping the customer on the line for a minute or two. If we know the merge conflicts are rare, we can favor latency over consistency. We don’t need to keep all the customers on the line for minutes to book a reservation. If there comes up a conflict, we can always sort it out later. So this seems to be a PA/EL system: In case of a network partition, favor availability. Else, favor (low) latency.