Generator Functions with yield — Producing Values One at a Time to Save Memory

In the previous article you saw closures — how to seal state inside a function and return a different value on each call. Python has a dedicated mechanism for this kind of "return the next value on each call" function: the generator function. Use yield instead of return, and the function pauses partway through to return a value, then resumes from where it left off the next time it's called.

It's a great fit when you don't want to materialize a big collection all at once and would rather stream one value at a time — log processing, bulk data crunching, and similar workloads.

yield Basics — Returning One Value at a Time

A function with yield value in its body is a generator function. Calling it like a regular function doesn't run the body — you just get back a special generator object.

To actually pull a value out, call next(obj).

The first next() runs the body up to the first yield and returns that value.

The next next() resumes from there and runs to the next yield.

Once there are no more yields left, another next() raises StopIteration.

def simple():
    yield 1
    yield 2

gen = simple()
print(type(gen))    # <class 'generator'>

print(next(gen))    # 1
print(next(gen))    # 2
print(next(gen))    # 3
# print(next(gen))  ← StopIteration (no more yields)

Difference from return

A regular function returns and the entire scope vanishes — the next call starts from the top. A generator function, by contrast, pauses on yield and keeps locals and the position around. The next next() resumes right where you left off, and that's the big difference.

Build a generator function that yields order IDs 1, 2, 3 in order, and pull them out one by one with next().

① Define def order_ids(): with three lines: yield 1 / yield 2 / yield 3.

② Build a generator object: gen = order_ids().

③ Call print(next(gen)) three times in a row and confirm 1 / 2 / 3 come out.

(When the answer is correct, the explanation will appear.)

Pulling Values with for — No Need to Worry About StopIteration

Writing next() every time gets tedious, and you shouldn't have to handle StopIteration yourself either. Use for value in generator: instead, and Python calls next() under the hood and breaks out of the loop automatically when the generator is done. This is by far the more common form.

def count_up_to(max_value):
    print("generator starting")
    for i in range(max_value):
        print(f"  before yield: {i}")
        yield i
        print(f"  after yield: {i}")

for v in count_up_to(3):
    print(f"received: {v}")

# Output flow:
# generator starting
#   before yield: 0
# received: 0
#   after yield: 0
#   before yield: 1
# ... (continues)

Use for to receive customer data from a generator that streams one at a time.

① Define def each_customer(): and loop with for name in ["Alice", "Bob", "Charlie"]:, yielding name.

② Pull values with for name in each_customer(): and print f"Next customer: {name}".

If all three appear in order, you're done.

Difference from list — Cutting Memory Use

When you're working with values from 0 to 999_999, a list comprehension materializes all 1,000,000 integers at once in memory. A generator, on the other hand, keeps only the current value and computes the next one on demand. The list ends up in the multi-MB range; the generator object itself is just a few hundred bytes.

import sys

MAX = 10 ** 6

# List: loads everything into memory at once
data_list = [i for i in range(MAX)]
print(sys.getsizeof(data_list))
# e.g. 8000056 (~8 MB)

# Generator expression: only the current item
data_gen = (i for i in range(MAX))
print(sys.getsizeof(data_gen))
# e.g. ~200 bytes

# Caller-side code is identical
for v in data_gen:
    if v > 2:
        break
    print(v)
# 0
# 1
# 2

The Generator Expression Shortcut

Swap the square brackets [ ... ] of a list comprehension for parentheses ( ... ) and you have a generator expression. (i for i in range(1_000_000)) gives you the same effect as a def-based generator function in a single line. Pass it straight to sum() / max() / any() and friends — it just works.

Build a generator expression (a list comprehension with parentheses instead of square brackets) for some price data, confirm the type with type(), then pull values one at a time with for.

① Build prices = (base * 100 for base in range(1, 6)). The trick is using ( ) instead of [ ].

② Print the type with print(type(prices)) and confirm <class 'generator'> shows up.

③ Pull values with for p in prices: and print f"Price: {p}".

Chaining Generators with yield from

When you want a generator to forward values from another generator as-is, you can write yield from sub_generator in one line instead of for v in sub: yield v. It's handy when you want to merge multiple data sources into a single generator.

For instance, with a function that streams sales for the Tokyo branch and another for Osaka, lining up yield from tokyo_sales() and yield from osaka_sales() gives the caller a generator that looks like one continuous stream.

def tokyo_sales():
    yield 1200
    yield 980

def osaka_sales():
    yield 850
    yield 1340

def all_sales():
    yield from tokyo_sales()
    yield from osaka_sales()

for amount in all_sales():
    print(amount)
# 1200
# 980
# 850
# 1340

Build a generator that combines per-store inventory lists into a single stream.

① Define def store_a(): and for item in ["apple", "orange"]: yield item.

② Define def store_b(): and for item in ["banana", "grape", "strawberry"]: yield item.

③ Define def all_items(): and write yield from store_a() followed by yield from store_b().

④ Iterate with for item in all_items(): print(item) to print all 5 in order.