More on iteration

Published

2023-07-31

A deeper dive into iteration in Python

What follows is a bit more detail about iterables and iteration in Python. You can skip this section entirely if you wish. This is presented here for the sole purpose of demonstrating what goes on “behind the scenes” when we iterate over some object in Python. With that said, let’s start with the case of a list (it works the same with a tuple or any other iterable).

>>> m = ['Greninja', 'Lucario', 'Mimikyu', 'Charizard']

When we ask Python to iterate over some iterable, it calls the function iter() which returns an iterator for the iterable (in this case a list).

>>> iterator = iter(m)
>>> type(iterator)
<class 'list_iterator'>

The way iterators work is that Python keeps asking “give me the next member”, until the iterator is exhausted. This is done (behind the scenes) by calls to the iterator’s __next__() method.

>>> iterator.__next__()
'Greninja'
>>> iterator.__next__()
'Lucario'
>>> iterator.__next__()
'Mimikyu'
>>> iterator.__next__()
'Charizard'

Now what happens if we call __next__() one more time?

We get a StopIteration error

>>> iterator.__next__()
Traceback (most recent call last):
  File "/Library/.../code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
StopIteration

Again, behind the scenes, when iterating through an iterable, Python will get an iterator for that object, and then call __next__() until a StopIteration error occurs, and then it stops iterating.

What about range()?

As we’ve seen earlier, range() returns an object of the range type.

>>> r = range(5)
>>> type(r)
<class 'range'>

But ranges are iterable, so they work with the same function iter(r), and the resulting iterator will have __next__() as a method.

>>> iterator = iter(r)
>>> iterator.__next__()
0
>>> iterator.__next__()
1
>>> iterator.__next__()
2
>>> iterator.__next__()
3
>>> iterator.__next__()
4
>>> iterator.__next__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

When Python iterates over a sequence (list, tuple, string) or a range object, it first creates an iterator, then iterates the iterator.

It’s important to note that enumerate objects are themselves iterators. This is why an enumerate object can be iterated only once. Python does not create a new iterator for these objects automatically, as it does when iterating sequences or range objects. (We’ll see something similar later on when we get to csv.reader objects, but that’s for another time.)

Original author: Clayton Cafiero < [given name] DOT [surname] AT uvm DOT edu >

No generative AI was used in producing this material. This was written the old-fashioned way.

This material is for free use under either the GNU Free Documentation License or the Creative Commons Attribution-ShareAlike 3.0 United States License (take your pick).