It takes about 3.6 minutes to read this article.
Yield can implement generators and coroutines. What is a generator, what is a coroutine, if you still don’t understand, you can continue to look down, the concept does not need to understand, as long as you understand its function and effect.
First look at the meaning of the English word yield:
to produce or provide sth, for example a profit, result or crop
Chinese meaning: produce (crop); produce (revenue, benefit, etc.); provide.
When I first learned Python, when I encountered the yield keyword, I understood it as a special kind of return. I could understand it, and I rarely used it myself, so I didn't study it in depth. Until now, I need to process a large amount of data, and the size of the data even exceeds the available memory of the computer. At this time, I think of yield. For example, I operate a db2 database to query data. When the result of the data is very large, I don't want to read it into the memory at once. I use the yield keyword to return a row of data. After the program is processed, I will fetch the next row:
def read(self, sql, params=()):
stmt = ibm_db.prepare(self.connection, sql)for index, param inenumerate(params):
ibm_db.bind_param(stmt, index +1, param)
ibm_db.execute(stmt)
row = ibm_db.fetch_tuple(stmt)while row:yield row
row = ibm_db.fetch_tuple(stmt)
The usage of the keyword yield can be understood in this way: it returns a value, but the program does not exit. Next time, the code behind the yield continues to run until there is no code behind, and the operation ends. Here we give a simple example to see the effect:
>>> def iter_fun():...print("a")...yield1...print("b")...yield2...print("c")...yield3...>>>iter_fun()<generator object iter_fun at 0x107e372a0>>>>for i initer_fun():...print(i)...
a
1
b
2
c
3>>> x =iter_fun()>>> x.__next__()
a
1>>> x.__next__()
b
2>>> x.__next__()
c
3>>> x.__next__()Traceback(most recent call last):
File "<stdin>", line 1,in<module>
StopIteration
Through the above example, we can find that the yield key automatically generates the private method next for us, so that all the data will not be taken out and stored in the memory. Use as much as you want to save memory space.
In addition, the execution speed of yield will increase when the amount of data is large:
In [14]: def normal_fun(num):...: result =[]...:for i inrange(0,num):...:if i%2==0:...: result.append(i)...:return result
...:
In [14]: def iter_fun(num):...:for i inrange(0,num):...:if i %2==0:...:yield i
...:
In [15]:%time for i initer_fun(1000000): a = i
CPU times: user 97 ms, sys:2.55 ms, total:99.6 ms
Wall time:97.2 ms
In [16]:%time for i innormal_fun(1000000): a = i
CPU times: user 115 ms, sys:13.6 ms, total:129 ms
Wall time:128 ms
In [17]:%time for i innormal_fun(100000000): a = i
CPU times: user 10.8 s, sys:668 ms, total:11.4 s
Wall time:11.4 s
In [18]:%time for i initer_fun(100000000): a = i
CPU times: user 9.1 s, sys:6.49 ms, total:9.11 s
Wall time:9.12 s
The above code is executed in the Ipython terminal. It can be seen that the function using yield is faster to execute.
Yield is the most convenient way to implement a generator yourself. The generator of the Python language is one of the most useful features, and it is also a feature that is not widely used. I have asked friends around who use java if there are similar features. I said no. I searched on the Internet. It is indeed a mainstream programming language. None, so the generator feature of Python has not attracted the attention of engineers who convert other languages to Python.
Why are generators so useful?
When the size of the data you need to process exceeds the available memory of your computer, the generator's lazy loading (read into the memory when it is used) is very effective.
For example, when reading a file, if you use the following method, if the file is particularly large, the memory may be full at once:
withopen("file.txt","r")as reader:for line in reader.readlines():
# do something
pass
The recommended way is to use the generator of the file itself, so that no matter how big the file is, the memory will not be burst:
withopen("file.txt","r")as reader:for line in reader:
# do something
pass
In fact, we can take a look at the reader methods:
>>> withopen("/etc/passwd","r")as reader:...dir(reader)...['_CHUNK_SIZE','__class__','__del__','__delattr__','__dict__','__dir__','__doc__','__enter__','__eq__','__exit__','__format__','__ge__','__getattribute__','__getstate__','__gt__','__hash__','__init__','__init_subclass__','__iter__','__le__','__lt__','__ne__','__new__','__next__','__reduce__','__reduce_ex__','__repr__','__setattr__','__sizeof__','__str__','__subclasshook__','_checkClosed','_checkReadable','_checkSeekable','_checkWritable','_finalizing','buffer','close','closed','detach','encoding','errors','fileno','flush','isatty','line_buffering','mode','name','newlines','read','readable','readline','readlines','reconfigure','seek','seekable','tell','truncate','writable','write','write_through','writelines']>>>
We can see that there is a iter method and a next method, these two methods are the signs of the generator.
If you want to learn more about generators, iterators, and iterable objects, you can read my previous tweets:
[ Python basic series-iterable objects, iterators and generators](http://mp.weixin.qq.com/s?__biz=MzU0OTg3NzU2NA==&mid=2247483827&idx=1&sn=6a661c78b3f0db2e4c94cee1ddc3b036&chksm=fba863e3fecene0ccdmvdbcdcfdf103e0ccd21350e3fee0ccdfeccdm
[ In-depth understanding of iterators and generators](http://mp.weixin.qq.com/s?__biz=MzU0OTg3NzU2NA==&mid=2247484255&idx=1&sn=cb340bdf3038a8d1715a89a8752c65c5&chksm=fba8610cccd3ccrebdirect4f5315ec#fba8610c65c5&chksm=f5315ec
The yield keyword can also implement the coroutine, although there are now asyncio and await keywords to facilitate the implementation of the coroutine, before that, the realization of the coroutine relied on yield.
Yield has a send method, which can change the return value of yield. It is used like this. First look at the code:
In [20]: def fun():...:print("a")...: a =yield1...:print("b")...:print("a = ",a)...: b =yield a
...: print("c")...:print("b = ",b)...:
In [21]: x =fun()
In [22]: x.__next__()
a
Out[22]:1
In [23]: x.send(4)
b
a =4
Out[23]:4
In [24]: x.send(5)
c
b =5---------------------------------------------------------------------------
StopIteration Traceback(most recent call last)<ipython-input-24-9183f5e81876>in<module>---->1 x.send(5)
When the next method of x is executed for the first time, the function executes to the first yield, prints a and returns the value 1. At this time, the variable a does not get the return value of yield, and a is None. When x.send is executed (4), when a gets the value 4, the program runs to the second yield, and the subsequent process is the same.
Using this feature, we can communicate with the called function, and then we can implement a producer consumer model, the code is as follows:
import time
def consume():
r =''while True:
n =yield r
if not n:returnprint('[consumer] consuming %s...'% n)
time.sleep(1)
r ='well received'
def produce(c):next(c)
n =0while n <5:
n = n +1print('[producer] producing %s...'% n)
r = c.send(n)print('[producer] consumer return: %s'% r)
c.close()if __name__=='__main__':
c =consume()produce(c)
The results are as follows:
[ producer] producing 1...[consumer] consuming 1...[producer] consumer return: well received
[ producer] producing 2...[consumer] consuming 2...[producer] consumer return: well received
[ producer] producing 3...[consumer] consuming 3...[producer] consumer return: well received
[ producer] producing 4...[consumer] consuming 4...[producer] consumer return: well received
[ producer] producing 5...[consumer] consuming 5...[producer] consumer return: well received
The produce and consume functions are executed in one thread, and the functions of the coroutine are realized by switching between the send method and yield by calling the send method.
For learning about coroutine, you can see my previous tweets:
[ Coroutine study notes)(http://mp.weixin.qq.com/s?__biz=MzU0OTg3NzU2NA==&mid=2247483684&idx=1&sn=0a106dcbe89438b4a1d3dd77eadd2900&chksm=fba86377ccdfea6110de27f78e828cfdaacrechat32f507e6434ebrenerechat26e64e6e6e36e26e32f48e828cfd2900
[ Use multithreading or coroutine for concurrency? ](http://mp.weixin.qq.com/s?__biz=MzU0OTg3NzU2NA==&mid=2247484280&idx=1&sn=ad46f3ef58ee7739da7d4f98ea97134c&chksm=fba8612bccdfe83d32e78e8ee7feadd4fc4benebdg21480revccd21e35e8ee7fe60
If you are interested in the article, please give "Follow" or "Watching" support. Search "Python No. 7" on WeChat and follow to get first-hand original dry goods.
Recommended Posts