HavocLabAll scenarios →
SEV-1Resilience

Multiverse of Madness

Tim pruned a "spare" VLAN from a trunk port as he figured it was unused. Little did he know it was carrying the node heartbeats, and he accidentally split the link between the east and west zones clean in half. At T = 200ms the zones stop seeing each other, and both are completely fine, which is the problem. The standby has to decide whether the primary is dead or just delayed. Guess wrong and every order starts existing in two states at once. It gets worse, at T = 600ms the primary actually dies. At T = 900ms it comes back, considering itself the primary. The Job: An order that got a yes has to still be a yes after the zones heal, without taking the store down to guarantee it. Settle which node is really in charge, and clean up after Tim (as usual).

Free to play, no account needed. A deterministic engine grades your design against this scenario's SLA - error rate, latency, and cost.