CAComputer Architecture · Lesson 5 of 7

Cores, Threads & Why Free Lunch Ended

CPUs stopped getting dramatically faster per-core around 2005 — physics said no. Instead we got more cores. Using them is now the programmer's job, and it's the hardest part of the machine.

Clock speeds hit a power wall (~4 GHz produces too much heat to cool practically), so vendors put multiple full CPUs — cores — on one chip. Each core runs one instruction stream (thread) at a time; 8 cores genuinely execute 8 things simultaneously. Simultaneous multithreading (Intel's 'hyper-threading') lets one core juggle two threads to fill idle execution slots — helpful, but not double speed.

Text

One core, 3 GHz, in one second:  ~10 billion instructions
Eight cores: ~80 billion — IF the work splits evenly.

Amdahl's Law — speedup is capped by the serial fraction:
  90% parallel work, infinite cores -> at most 10x faster.
  50% parallel work, infinite cores -> at most 2x.

The catch: cores share memory. Two cores writing the
same data need coordination — and that's where bugs live.

// The classic data race:
// Two threads both run: counter++
// which is really THREE instructions:
//   load  counter into register
//   add   1
//   store register back to counter
//
// Interleaving:
//   Thread A loads 100      Thread B loads 100
//   A adds -> 101           B adds -> 101
//   A stores 101            B stores 101
// Two increments, counter went up by ONE. Lost update.
//
// Fixes: locks (mutex), atomic instructions
// (lock xadd on x86), or sharing nothing at all.

✦ Tip
Modern hardware parallelism also includes SIMD (one instruction operating on 8-16 values at once — how video codecs and NumPy fly) and GPUs (thousands of simple cores for uniform work — graphics and neural networks). Different shapes of the same idea: do more per clock, since clocks stopped climbing.

◆ Note
Cache coherency ties the whole chip together: when core A writes data cached by core B, hardware invalidates B's copy automatically. Correctness is preserved, but ping-ponging a hot cache line between cores ('false sharing') silently wrecks performance — a classic advanced gotcha.