Lesson 14 of 34 · 0 of 3 exercises solved

Iterators & Generators

A holds every element in at once; a produces elements one at a time, on demand. That lets you work through very large files or endless sequences using very little memory.

Every for in Python is powered by the : iter() gets an iterator, next() pulls the next value, and StopIteration signals the end. Generators let you write an iterator without writing a .

# The protocol, by hand
nums = [1, 2, 3]
it = iter(nums)
print(next(it))   # 1
print(next(it))   # 2
print(next(it))   # 3
# next(it) now raises StopIteration

# A generator function — "yield" instead of "return"
def countdown(n):
    while n > 0:
        yield n
        n -= 1
    print("liftoff!")

for x in countdown(3):
    print(x)      # 3, 2, 1  then "liftoff!"

# Execution PAUSES at each yield and resumes on the next next() call.
gen = countdown(2)
print(next(gen))  # 2   — body runs up to the first yield, then freezes
print(next(gen))  # 1
Python (loads on first run)Ctrl+Enter runs
# Generator expressions — like a list comprehension with () instead of []
squares_list = [x * x for x in range(1_000_000)]   # ~40 MB in memory
squares_gen  = (x * x for x in range(1_000_000))   # a few hundred bytes

print(sum(squares_gen))   # streams through, never materializes the list

# Reading a huge file line by line without loading it all
def read_lines(path):
    with open(path) as f:
        for line in f:          # file objects are already iterators
            yield line.rstrip()

def only_errors(lines):
    for line in lines:
        if "ERROR" in line:
            yield line

# Pipelines compose: each stage pulls one item at a time
# for err in only_errors(read_lines("app.log")):
#     print(err)

# Infinite generators are fine — just do not call list() on them
def naturals():
    n = 0
    while True:
        yield n
        n += 1

from itertools import islice
print(list(islice(naturals(), 5)))   # [0, 1, 2, 3, 4]
Python (loads on first run)Ctrl+Enter runs
from itertools import chain, groupby, count, cycle, takewhile

# itertools is the standard library's generator toolbox
print(list(chain([1, 2], [3, 4])))            # [1, 2, 3, 4]
print(list(takewhile(lambda n: n < 5, count()))) # [0, 1, 2, 3, 4]

# yield from — delegate to another generator
def flatten(nested):
    for item in nested:
        if isinstance(item, list):
            yield from flatten(item)
        else:
            yield item

print(list(flatten([1, [2, [3, [4]]], 5])))   # [1, 2, 3, 4, 5]

# A custom iterator class (when you need extra state or methods)
class Fibonacci:
    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):
        a, b = 0, 1
        for _ in range(self.limit):
            yield a
            a, b = b, a + b

print(list(Fibonacci(8)))   # [0, 1, 1, 2, 3, 5, 8, 13]
Python (loads on first run)Ctrl+Enter runs

Common question

Is a generator a kind of list?

An iterator is an object that hands out values one at a time through next() and signals the end with StopIteration. A generator is the easiest way to write one: a with yield in it, whose state (local and where it is in the body) is paused between calls. Because values are made on demand, generators handle streams of any size, even endless ones, in constant memory.

Use generators to build : one yields lines, the next filters them, the next parses them, and nothing is computed until the final step asks for a value. Use itertools for the common patterns (islice, chain, groupby), and remember that a generator is used up after one pass. Call the function again for a fresh one.

Source: PEP 255: "A generator function is a function that, when called, returns an iterator object. The body is not executed until the iterator's next is invoked."

Common mix-ups

Each of these sounds reasonable. Think about why it is wrong, then open the explanation.

  • A generator produces a sequence of values, so it's a list built one piece at a time.

  • Calling a generator function runs its body.

  • A for loop is special syntax that has nothing to do with iterators.

Practice

Write countdown(n), a generator function that yields n, n-1, … down to 1. Because it uses yield, the values are produced one at a time — nothing is stored in a list.

Exercise

Countdown generator

3 checks
def countdown(n):
    ...


for value in countdown(5):
    print(value)
Python (loads on first run)Ctrl+Enter runs
Output
Press Run to try your code, then Check answer to grade it.

Write fibonacci(), an infinite generator of the Fibonacci sequence starting 0, 1, 1, 2, 3, …. Then use itertools.islice to collect the first ten values into first_ten.

Exercise

Infinite Fibonacci

3 checks
from itertools import islice


def fibonacci():
    a, b = 0, 1
    ...


first_ten = []
print(first_ten)
Python (loads on first run)Ctrl+Enter runs
Output
Press Run to try your code, then Check answer to grade it.

Write chunked(items, size) that yields consecutive lists of size elements from any iterable; the last chunk may be shorter. chunked(range(7), 3) gives [0,1,2], [3,4,5], [6].

Exercise

Chunk an iterable

3 checks
def chunked(items, size):
    ...


for chunk in chunked(range(7), 3):
    print(chunk)
Python (loads on first run)Ctrl+Enter runs
Output
Press Run to try your code, then Check answer to grade it.

Check yourself

Quick check

What happens when you call next() on a generator that has already finished?

Finished reading?3 exercises still open. You can come back to them later.

Tip: use the ← and → keys to move between lessons.