The latency table engineers should memorize: better than an architecture diagram at stopping knee-jerk decisions

Dhanian posted a latency table on X. It spans from 0.5 ns for L1 cache to 150 ms for a packet round trip from California to the Netherlands. Tables like this have circulated in engineering circles for a long time; Jeff Dean's version is often used as a reference, and this time Dhanian has compressed the commonly used items onto one page.

First, remember the conversions. 1 µs is 1,000 ns, and 1 ms is one million ns. The table spans from 0.5 ns to 150 ms, across more than eight orders of magnitude. Human intuition about fast and slow is unreliable here; orders of magnitude are what count.

At the nanosecond scale: L1 cache 0.5 ns, branch misprediction 5 ns, L2 cache 7 ns, mutex lock and unlock 25 ns, main memory access 100 ns. Main memory is 200 times slower than L1, and L2 is 14 times slower than L1. When writing code, a single memory access feels cheap, but put it in a loop and on a hot path, and the bill can look ugly.

At the microsecond scale, network and storage start to appear. Compressing 1 KB of data 3 µs, sending 1 KB over a 1 Gbps network 10 µs, random 4K read from SSD 150 µs, sequential 1 MB read from memory 250 µs. Here you can already see the relationship between a local SSD and a data-center round trip: a random 4K SSD read is about 150 µs, a same-data-center round trip is 500 µs—the former is about three times faster. Before putting a remote call on a high-frequency path, do the math first.

At the millisecond scale, things get closer to everyday system experience. Same-data-center round trip 0.5 ms, sequential 1 MB read from SSD 1 ms, disk seek 10 ms, sequential 1 MB read from disk 20 ms. Disk seek is 20 times more expensive than a same-data-center round trip, and sequential 1 MB read from disk is 20 times slower than sequential 1 MB read from SSD. Choices like cache layers, storage layers, and sync vs. async are all being driven underneath by this set of numbers.

The global network segment is the most easily underestimated. A packet round trip from California to the Netherlands is 150 ms. One transoceanic request is 150 ms; ten serial dependencies make it 1.5 seconds. A line drawn between two boxes in an architecture diagram may be 150 milliseconds. The diagram looks clean; the user's wait does not.

This table cannot finish the design for you, but it can stop some assumptions ahead of time. When choosing memory, SSD, disk, or network, look at the order of magnitude first, then the business requirements. Dhanian's original post is here: https://x.com/e_opore/status/2116607999037034845 . The complete system design handbook is here: https://codewithdhanian.gumroad.com/l/urcjee .