Secrets of the new features of Python 3.8

From the powerful new assignment syntax to the big changes at the bottom, Python 3.8 moves towards a more modern Python.

Python 3.8 is the latest version of the Python language, which is suitable for scripting, automation, and various tasks such as machine learning and web development. Now Python 3.8 has entered the official beta stage. This version brings many syntax changes, memory sharing, more efficient serialization and deserialization, improved dictionaries and more new features.

Python 3.8 also introduces many performance improvements. In general, we are about to have a faster, more precise, more consistent and more modern Python. Here are the new features and most important changes in Python 3.8.

1、 Assignment expression

The most obvious change in Python 3.8 is the assignment expression, the := operator. Assignment expressions can assign a value to a variable, even if the variable does not exist. It can be used in expressions and does not need to appear as a separate statement.

while(line := file.readline())!="end":print(chunk)

In the above example, if the variable line does not exist, it will be created, and then the return value of file.readline() will be assigned to it. Then check if the line is "end". If not, read the next line, save it in line, and continue the test.

Assignment expressions follow Python’s tradition of simplicity, just like list comprehensions. The purpose is to avoid some boring boilerplate code in specific Python programming patterns. For example, the above code needs to write two more lines of code in general writing.

2、 Parameters specified by position only

The parameter specified by position only is a new syntax in the function definition, which allows the programmer to force a parameter to be specified only by position. This can solve the ambiguity of which parameter is a positional parameter and which parameter is a keyword parameter in the Python function definition.

Parameters specified only by position can be used in situations where a function accepts arbitrary keyword parameters, but can also accept one or more unknown parameters. This is usually the case with Python's built-in functions, so allowing programmers to do so can enhance the consistency of the Python language.

The examples given in the Python documentation are as follows:

def pow(x, y, z=None,/):
 r = x**y
 if z is not None:
  r %= z
 return r

The symbol / separates positional parameters and keyword parameters. In this example, all parameters are unknown parameters. In previous versions of Python, z would be considered a keyword argument. But using the above function definition, pow(2, 10) and pow(2, 10, 5) are both correct calling methods, and pow(2, 10, z=5) is incorrect.

3、 Support f string debugging

The f string format makes it easier to calculate output text and values or variables in the same expression, and it is more efficient.

x =3print(f'{x+1}')

Output 4.

No = added at the end of the f string expression can output the value of the f expression itself, followed by the calculated value

x =3print(f'{x+1=}')

The output is x+1=4.

4、 Multi-process shared memory

In Python 3.8, the multiprocessing module provides the SharedMemory class, which can create a shared memory area between different Python processes.

In the old version of Python, sharing data between processes can only be done by writing files, sending them through network sockets, or serializing them using Python's pickle module. Shared memory provides a faster way to transfer data between processes, making Python's multi-processor and multi-core programming more efficient.

The shared memory segment can be allocated as a pure byte area, or as an unmodifiable list-like object, which can store a small number of Python objects such as numeric types, strings, byte objects, and None objects.

5、 Improvement of Typing module

Python is a dynamically typed language, but you can add type hints through the typing module so that third-party tools can verify Python code. Python 3.8 adds some new elements to typing, so it can support more robust checks:

• Final decorator and Final type annotation indicate that the modified or annotated object should not be rewritten, inherited, or reassigned at any time.
• The Literal type restricts an expression to a specific value or a list of values (not necessarily the same type of value).
• TypedDict can be used to create a dictionary whose specific key values are restricted to one or more types. Note that these restrictions are only used to determine the legality of values at compile time, and cannot be restricted at runtime.

6、 New version of the pickle protocol

Python's pickle module provides a way to serialize and deserialize Python data structures or instances, and you can save the dictionary as it is for later reading. Different versions of Python support different pickle protocols, and the latest version supports a wider, more powerful, and more efficient serialization.

The fifth version of the pickle protocol introduced by Python 3.8 can pickle objects in a new way, which can support Python's buffer protocol, such as bytes, memoryviews, or Numpy array. The new pickle avoids many memory copy operations when pickling these objects.

External libraries such as NumPy and Apache Arrow support the new pickle protocol in their Python bindings. The new pickle can also be used as a plugin for Python 3.6 and 3.7 and can be installed from PyPI.

7、 Reversible dictionary

The dictionary was rewritten in Python 3.6, using a new implementation contributed by the PyPy project. In addition to being faster and more compact, the current dictionary will inherit the order of the elements-the elements will be arranged in the order they were added, just like a list. Python 3.8 also allows reversed() on dictionaries.

8、 Performance improvement

• The speed of many built-in methods and functions has been increased by 20%~50%, because many functions previously required unnecessary parameter conversion.
• A new opcode cache can increase the speed of specific instructions in the interpreter. However, currently only the LOAD_GLOBAL opcode has achieved speed improvement, which has increased its speed by 40%. Similar optimizations will be made in future versions.
• File copy operations such as shutil.copyfile() and shutil.copytree() now use platform-specific calls and other optimization measures to increase the speed of operations.
• The newly created list is now an average of 12% smaller than before, thanks to the optimization of the list constructor if the list length can be known in advance.
• In Python 3.8, writing to class variables of new classes (such as class A(object)) has become faster. operator.itemgetter() and collections.namedtuple() have also been optimized for speed.

9、 Python C API and CPython implementation

The most recent version of Python has put a lot of effort into refactoring the C API used in CPython (a reference implementation of Python written in C). So far these works are still being added, and the existing results include:

• Python Initialization Configuration has a new C API, which enables tighter control and more detailed feedback on Python initialization routines. In this way, it is easier to embed the Python runtime into other applications, and it is also possible to pass startup parameters to the Python program programmatically. The new API also ensures that all Python configuration controls have a single, consistent location, so future changes (such as Python's new UTF-8 mode) are also easier.
• Another new C API of CPython-the "vectorcall" calling protocol-enables faster calls to Python internal methods without creating temporary objects. The API is still unstable, but it has improved significantly. The API is planned to mature in Python 3.9.
• The audit hook of the Python runtime provides two APIs for the Python runtime, which can be used to hook events to ensure that external tools such as test frameworks, logs, and audit systems can monitor them.