Lesson 6 of 7

Thinking in Nanoseconds

It helps to know roughly how long common operations take. Once you know a RAM access costs ~100ns and a disk read ~100µs, many design decisions become easier.

Text
Latency numbers every programmer should know (~2020s):

  L1 cache reference ................ 1 ns
  Branch mispredict ................. 3 ns
  L2 cache reference ................ 4 ns
  Mutex lock/unlock ................ 17 ns
  Main memory reference ........... 100 ns
  Compress 1KB (Snappy) .......... 2,000 ns
  Read 1MB sequentially from RAM . 10,000 ns
  SSD random read ............... 16,000 ns
  Read 1MB sequentially from SSD  ~200,000 ns
  Round trip in same datacenter . 500,000 ns
  Read 1MB from spinning disk .. 1,000,000 ns
  Packet US -> Europe -> US  150,000,000 ns

Scaled up: if L1 = 1 second, RAM = 100 seconds,
SSD read = 4.4 hours, transatlantic packet = 4.8 YEARS.

Read the table vertically and rules of thumb fall out. Sequential beats random at every level. beats disk by ~100x, disk beats network round trips. One query per item in a (the N+1 problem) is very slow not because queries are slow, but because each one is a network round trip — half a million nanoseconds of waiting per item.

# The same lesson at application level:

# BAD: 1000 network round trips
for user_id in user_ids:            # 1000 ids
    user = db.query("SELECT ... WHERE id = ?", user_id)

# GOOD: 1 round trip carrying 1000 rows
users = db.query("SELECT ... WHERE id IN (...)", user_ids)

# Same asymptotic complexity. ~1000x faster in practice.
# The machine rewards batching, always.
Python (loads on first run)Ctrl+Enter runs
Finished reading?Mark it done to track your progress.

Tip: use the ← and → keys to move between lessons.