Python - Generator Functions

Python provides a generator to create your own iterator function. A generator is a special type of function which does not return a single value, instead, it returns an iterator object with a sequence of values. In a generator function, a yield statement is used rather than a return statement. The following is a simple generator function.

def mygenerator():
    print('First item')
    yield 10

    print('Second item')
    yield 20

    print('Last item')
    yield 30

In the above example, the mygenerator() function is a generator function. It uses yield instead of return keyword. So, this will return the value against the yield keyword each time it is called. However, you need to create an iterator for this function, as shown below.

gen = mygenerator() 
val = next(gen) #First item
print(val) #10

val = next(gen) #Second item
print(val) #20

val = next(gen) #Last item
print(val) #30

val = next(gen) #error            

The generator function cannot include the return keyword. If you include it, then it will terminate the function. The difference between yield and return is that yield returns a value and pauses the execution while maintaining the internal states, whereas the return statement returns a value and terminates the execution of the function.

The following generator function includes the return keyword.

def mygenerator():
    print('First item')
    yield 10

    return

    print('Second item')
    yield 20

    print('Last item')
    yield 30

Now, execute the above function as shown below.

gen = mygenerator() 
gen = mygenerator()
val = next(gen) #First item
print(val) #10

val = next(gen) #error

As you can see, the above generator stops executing after getting the first item because the return keyword is used after yielding the first item.

Using for Loop with Generator Function

The generator function can also use the for loop.

def get_sequence_upto(x):
    for i in range(x):
        yield i

As you can see above, the get_sequence_upto function uses the yield keyword. The generator is called just like a normal function. However, its execution is paused on encountering the yield keyword. This sends the first value of the iterator stream to the calling environment. However, local variables and their states are saved internally.

The above generator function get_sequence_upto() can be called as below.

seq = get_sequence_upto(5) 
print(next(seq)) #0
print(next(seq)) #1
print(next(seq)) #2
print(next(seq)) #3
print(next(seq)) #4
print(next(seq)) #error

The function resumes when next() is issued to the iterator object. The function finally terminates when next() encounters the StopIteration error.

In the following example, function square_of_sequence() acts as a generator. It yields the square of a number successively on every call of next().

def square_of_sequence(x):
    for i in range(x):
        yield i*i

The following script shows how to call the above generator function.

gen=square_of_sequence(5)
while True:
    try:
        print ("Received on next(): ", next(gen))
    except StopIteration:
        break

The above script uses the try..except block to handle the StopIteration error. It will break the while loop once it catches the StopIteration error.

Received on next(): 0
Received on next(): 1
Received on next(): 4
Received on next(): 9
Received on next(): 16

We can use the for loop to traverse the elements over the generator. In this case, the next() function is called implicitly and the StopIteration is also automatically taken care of.

squres = square_of_sequence(5)
for sqr in squres:
    print(sqr)

0
1
4
9
16

Generator Expression

Python also provides a generator expression, which is a shorter way of defining simple generator functions. The generator expression is an anonymous generator function. The following is a generator expression for the square_of_sequence() function.

squre = (x*x for x in range(5))
print(next(squre)) #0
print(next(squre)) #1                                            
print(next(squre)) #4                                            
print(next(squre)) #9                                            
print(next(squre)) #16

In the above example, (x*x for x in range(5)) is a generator expression. The first part of an expression is the yield value and the second part is the for loop with the collection.

The generator expression can also be passed in a function. It should be passed without parentheses, as shown below.

import math

val = sum(x*x for x in range(5)) 
print(val)

In the above example, a generator expression is passed without parentheses into the built-in function sum.