InfraForge
Assembling a production system
InfraForge
Recorded load · real failures

See exactly why your system breaks.

Design infrastructure, run it under recorded load, and watch every part respond, second by second.

02 · Live traffic

The balancer splits every request between two servers.

  1. 59s130 requests a second, 65 to each server
  2. 59sboth servers near 43% CPU, the database at 62%
03 · The failure

app-2 crashes. Like a host dying mid-shift.

  1. 60sa crash is injected into app-2: its process stops
  2. 61sapp-2 answers nothing; its cables drain
  3. 61sthe balancer is left with one server
04 · The re-route

Everything shifts to app-1.

  1. 61sapp-1 takes all 130 requests a second
  2. 61sits CPU jumps from 43% to 86%
05 · Saturation

app-1 pins at 100% CPU. The database saturates.

  1. 62sthe database reaches 95% CPU
  2. 63sapp-1 reaches 100% CPU: no headroom left
06 · Why it broke

Each server was sized for half the load. Losing one left no room for the other half.

A fix to try: a third server or autoscaling with headroom, then replay this exact recording to compare.

Evaluated at second 90

Now break your own.

Scroll to run the recording