CAComputer Architecture · Lesson 6 of 7

Thinking in Nanoseconds

Great engineers carry a mental price list for operations. Once you know a RAM access costs ~100ns and a disk read ~100µs, whole categories of design decisions become obvious.

Text

Latency numbers every programmer should know (~2020s):

  L1 cache reference ................ 1 ns
  Branch mispredict ................. 3 ns
  L2 cache reference ................ 4 ns
  Mutex lock/unlock ................ 17 ns
  Main memory reference ........... 100 ns
  Compress 1KB (Snappy) .......... 2,000 ns
  Read 1MB sequentially from RAM . 10,000 ns
  SSD random read ............... 16,000 ns
  Read 1MB sequentially from SSD  ~200,000 ns
  Round trip in same datacenter . 500,000 ns
  Read 1MB from spinning disk .. 1,000,000 ns
  Packet US -> Europe -> US  150,000,000 ns

Scaled up: if L1 = 1 second, RAM = 100 seconds,
SSD read = 4.4 hours, transatlantic packet = 4.8 YEARS.

Read the table vertically and rules of thumb fall out. Sequential beats random at every level. Memory beats disk by ~100x, disk beats network round trips. One database query per item in a loop (the N+1 problem) is catastrophic not because queries are slow, but because each one is a network round trip — half a million nanoseconds of waiting per item.

Python

# The same lesson at application level:

# BAD: 1000 network round trips
for user_id in user_ids:            # 1000 ids
    user = db.query("SELECT ... WHERE id = ?", user_id)

# GOOD: 1 round trip carrying 1000 rows
users = db.query("SELECT ... WHERE id IN (...)", user_ids)

# Same asymptotic complexity. ~1000x faster in practice.
# The machine rewards batching, always.

✦ Tip
Measure before optimizing — profilers (perf on Linux, Instruments on macOS) show where time actually goes, and it's rarely where you'd guess. But knowing the price list tells you what's plausible: if your code reads 1 GB from SSD, anything under ~2 seconds means caching is helping; anything over 30 means you're doing random reads.