The essence of Python language: Itertools library
Foreword
Did you know that Python's Itertools library is considered a gem of Python? Some users even consider it one of the coolest and most amazing Python libraries. We can use Itertools modules to enrich our applications and create a reliable working solution in a shorter time.
This article will help readers understand how to use Itertools modules in their projects.
The article is divided into three parts, each part will explain the specific functions of the Itertools library. Specifically:
- Infinite iterator
- Terminate iterator
- Composite iterator
Concept introduction
What is an iterator?
Iterators are objects composed of next methods. It has a status. The state is used to remember the execution during the iteration. Therefore, the iterator knows its current state, which makes it memory efficient. This is why iterators are used in memory efficient and fast applications.
We can open unlimited data streams (such as reading a file) and get the next item (such as the next line in the file). Then we can perform an operation on the project and proceed to the next project. This may mean that we can have an iterator that returns an infinite number of elements, because we only need to know the current item.
When there is no next item to return, the iterator will raise a StopIteration exception.
What is iterable?
Iterable is an object that can return an iterator. It has an iter method that returns an iterator. Iterable is also an object that we can loop and call iter(). It has a getitem method, which can take consecutive indexes from 0 (and raise an IndexError when the index is no longer valid).
What is Itertools?
Itertools is a Python module and part of the Python 3 standard library. It allows us to perform efficient memory and computational tasks on the iterator. It is inspired by the construction of APL, Haskell and SML.
In essence, this module contains many fast and memory efficient methods that can help us build applications in pure Python concisely and efficiently.
Infinite iterator
What if we want to construct an iterator that returns infinitely evenly spaced values? Or, what if we have to generate an element loop from the iterator? Or, maybe we want to repeat the elements of the iterator?
The itertools library provides a set of functions that we can use to perform all the functions we need.
The three functions listed in this section construct and return iterators that can be an infinite stream of items.
Count
As an example, we can generate an infinite sequence of equidistant values:
start =10
stop =1
my_counter = it.count(start, stop)for i in my_counter:
# this loop will run for ever
print(i)
result:
101112131415
Cycle
We can use the cycle method to generate an infinite loop of elements from the input.
The input of this method needs to be an iterable object, such as a list, string or dictionary, etc.
my_cycle = it.cycle('Python')for i in my_cycle:print(i)
result:
P
y
t
h
o
n
P
y
t
h
o
n
P
Repeat
To repeat an item (such as a string or a collection), you can use the repeat() function:
to_repeat ='FM'
how_many_times =4
my_repeater = it.repeat(to_repeat, how_many_times)for i in my_repeater:print(i)
# Prints
FM
FM
FM
FM
This will repeat the string "FM" 4 times. If we don't provide the second parameter, it will repeat this string indefinitely.
Terminate iterator
In this section, I will explain the powerful features of termination iterations. These functions can be used in many scenarios, such as:
- We may have many iterations, and we want to perform an operation on all iteration elements in a sequence one by one.
- Or when we have a lot of functions that we want to execute on each element of iterable
- Or sometimes we delete elements from the iterator and then perform operations on other elements.
Chain
This method allows us to create an iterator that returns all the elements in the input iteration in the sequence until there are no more elements left. Therefore, it can treat a continuous sequence as a single sequence.
chain = it.chain([1,2,3],['a','b','c'],['End'])for i in chain:print(i)
result
123
a
b
c
End
Drop While
We can pass an iterable and a condition, and this method will start calculating the condition for each element until the condition returns False for an element. Once the condition value of an element is False, the function will return the rest of the iterable elements.
For example, suppose we have a job list and we want to iterate through the elements and return the elements only if the conditions are not met. Once the value of the condition is False, we expect to return the remaining elements of the iterator.
jobs =['job1','job2','job3','job10','job4','job5']
dropwhile = it.dropwhile(lambda x :len(x)==4, jobs)for i in dropwhile:print(i)
result:
job10
job4
job5
This method returns the above three items, because the length of the element job10 is not equal to 4 characters, so job10 and other elements are returned.
Take While
This method is the opposite of the dropwhile() method. Essentially, it returns all elements of an iterable until the first condition returns False, then it does not return any other elements.
For example, suppose we have a job list and want to stop returning to jobs immediately when the conditions are not met.
jobs =['job1','job2','job3','job10','job4','job5']
takewhile = it.takewhile(lambda x :len(x)==4, jobs)for i in takewhile:print(i)
result:
job1
job2
job3
GroupBy
This function constructs an iterator after grouping successive elements of the iterable. This function returns an iterator of key and value pairs, where the key is the group key and the value is a collection of consecutive elements grouped by key.
Consider the following code snippet:
iterable ='FFFAARRHHHAADDMMAAALLIIKKK'
my_groupby = it.groupby(iterable)for key, group in my_groupby:print('Key:', key)print('Group:',list(group))
Note that the group attribute is iterable, so I will materialize it as a list.
So this will print:
Key: F
Group:[‘F’, ‘F’, ‘F’]
Key: A
Group:[‘A’, ‘A’]
Key: R
Group:[‘R’, ‘R’]
Key: H
Group:[‘H’, ‘H’, ‘H’]
Key: A
Group:[‘A’, ‘A’]
Key: D
Group:[‘D’, ‘D’]
Key: M
Group:[‘M’, ‘M’]
Key: A
Group:[‘A’, ‘A’, ‘A’]
Key: L
Group:[‘L’, ‘L’]
Key: I
Group:[‘I’, ‘I’]
Key: K
Group:[‘K’, ‘K’, ‘K’]
Tee
This method can split an iteration and generate a new iteration from the input. The output is also an iterator, which returns the iterable value of a given number of items. To understand it better, please see the following code snippet:
iterable ='FM'
tee = it.tee(iterable,5)for i in tee:print(list(i))
This method returns the entire iterable FM, 5 times:
[ ‘F’, ‘M’][‘F’, ‘M’][‘F’, ‘M’][‘F’, ‘M’][‘F’, ‘M’]
Combination iterator
Permutations
By using the permutation method, we can create an iterator to return the continuous arrangement of elements in the input iterable.
We can pass in a parameter to specify the length of the arrangement. It defaults to the length of the iterable.
This means that when length is missing, the method will generate all possible full-length permutations.
iterable ='FM1'
length =2
permutations = it.permutations(iterable, length)for i in permutations:print(i)
result
( ‘F’, ‘M’, ‘1’)(‘F’, ‘1’, ‘M’)(‘M’, ‘F’, ‘1’)(‘M’, ‘1’, ‘F’)(‘1’, ‘F’, ‘M’)(‘1’, ‘M’, ‘F’)
If the length is 2, it generates:
( ‘F’, ‘M’)(‘F’, ‘1’)(‘M’, ‘F’)(‘M’, ‘1’)(‘1’, ‘F’)(‘1’, ‘M’)(‘F’, ‘M’)(‘F’, ‘1’)(‘M’, ‘1’)
Combinations
Finally, I want to explain how to generate combinations of iterables.
Given an iterator, we can construct an iterator to return a subsequence of elements of a given length.
According to their position, the elements are considered unique, and only different elements are returned.
iterable ='FM1'
combinations = it.combinations(iterable,2)for i in combinations:print(i)
result
( ‘F’, ‘M’)(‘F’, ‘1’)(‘M’, ‘1’)
·END·
Recommended Posts