Beau
Okay, so just to make sure I'm following from last time... we've established that thanks to the Two Generals' Problem, we can never be one hundred percent certain a message got through an unreliable network.
Transcript
Beau
Okay, so just to make sure I'm following from last time... we've established that thanks to the Two Generals' Problem, we can never be one hundred percent certain a message got through an unreliable network.
Jo
Exactly. There's always that final acknowledgement that could have gotten lost, leaving one side in a state of 'I think they know, but I'm not totally sure.'
Beau
And then the CAP theorem piled on, saying when the network breaks—which it will—we have to choose. We can either shut down parts of our system to stay perfectly correct... or we can stay online, but the data might be a little... out of sync for a bit.
Jo
And for something like a big ERP system that needs to be available for users globally, shutting down is rarely an option. So we choose availability.
Beau
Right. So... we're left with this world of uncertainty and temporary incorrectness. This feels very different from the cozy, predictable world of a single PostgreSQL database with its ACID guarantees.
Jo
It is a huge mental shift. But it's not chaos. We just trade one kind of consistency for another. We move from 'strong consistency'—where everyone sees the exact same data at the exact same time—to something called 'eventual consistency'.
Beau
Eventual consistency. The word 'eventual' doesn't exactly inspire confidence. It sounds like my laundry will 'eventually' get folded.
Jo
It's a fair point. But in computing, 'eventually' is a guarantee, not a hope. It means the system will become consistent, we just can't say precisely when. It might be milliseconds, or it could be a few seconds.
Beau
Okay, so give me a 'mental movie' for this. In our ERP, what does this actually look like?
Jo
Perfect. Imagine a user clicks 'Place Order'. The first thing that needs to happen is we need to reserve the item in inventory, right? So we send a message to the Inventory service: 'Decrement stock for item XYZ by one'.
Beau
Got it. So if we have ten widgets, we now have nine.
Jo
Exactly. That happens almost instantly. But placing an order also means we need to record the sale in the accounting books. That's a different service, maybe a different database. So the Order service also sends a message to the Accounting service: 'Create an invoice for item XYZ'.
Beau
Okay... and because of network latency or just because the accounting service is busy, that message might take a second or two to be processed.
Jo
Precisely. So for those two seconds, a tiny window of time exists where the system is technically inconsistent. The inventory shows nine widgets, but the accounting ledger doesn't yet show a sale. The money isn't booked.
Beau
Ah, so that's the 'eventual' part. The system as a whole isn't correct 'right now,' but it will be... eventually. Once the accounting service catches up.
Jo
You got it. That brief period of inconsistency is what we call an 'in-flight' state. The transaction isn't fully complete, it's... traveling between systems. We've accepted this temporary inconsistency as the price for keeping the system available and responsive.
Beau
That makes sense. But... what if someone in accounting runs a report during that two-second window? Their numbers will be wrong. That sounds like a big problem.
Jo
It could be, which is why we have to distinguish between technical consistency and business consistency.
Beau
Okay, what's the difference?
Jo
Technical consistency is what we just described: every database record is perfectly aligned at the exact same nanosecond. Business consistency is about whether the system is correct enough for the business process to work. Does the finance department really need up-to-the-millisecond sales data? Or is a report that's accurate as of five seconds ago perfectly fine?
Beau
Right, I guess for a quarterly report, a few seconds of lag doesn't matter at all. The business is 'consistent enough'.
Jo
Exactly. But for the user experience, we have to be more careful. When the customer clicks 'Place Order', they need immediate feedback. We can't make them wait five seconds for the accounting service to finish before we tell them their order was successful.
Beau
So we tell them 'Order Confirmed!' right after the inventory is updated, even though the accounting part is still 'in-flight'.
Jo
Yes. And that's a user experience design choice we make. Maybe in their order history, the status says 'Processing' for a few seconds before it flips to 'Completed'. Users are actually very used to this. Think about ordering food for delivery. It goes from 'Order placed' to 'Preparing' to 'Out for delivery'. Those are all 'in-flight' states that reflect an eventually consistent process.
Beau
That's a great example. I never thought of my pizza order as a distributed system, but it absolutely is. The restaurant got the order, but the driver hasn't been assigned yet. The whole system isn't 'consistent' until the pizza is in my hands.
Jo
And you, the user, are perfectly happy with that latency because the app gives you updates along the way. You accept the eventual consistency of the pizza arriving.
Beau
So this is really a psychological shift. As a developer used to PostgreSQL transactions, where everything happens at once or not at all, I have to get comfortable with the idea of a process being 'in progress' across multiple systems, and design the user interface to reflect that reality.
Jo
That's the core of it. We're moving from a world of technical guarantees of 'now' to a world of business guarantees of 'eventually'. And that change in perspective is what allows us to build systems that are resilient and can scale.