Magic method#

View the magic method of the class

classA:
 pass
dir(A)  #Can get all public members of the class

The output is as follows

['__ class__','__delattr__','__dict__','__dir__','__doc__','__eq__','__format__','__ge__','__getattribute__','__gt__','__hash__','__init__','__le__','__lt__','__module__','__ne__','__new__','__reduce__','__reduce_ex__','__repr__','__setattr__','__sizeof__','__str__','__subclasshook__','__weakref__']

In Python, all methods wrapped in __ double underscores are collectively called magic methods. For example, the most common __init__.

Create/Destroy#

__ new__: object.__new__(cls) Method of creating a class: constructor
__ del__: delete class: destructor
__ init__: initialization function

classA:
 def __new__(cls,*args,**kwargs):print('new')return object.__new__(cls)

 def __init__(self):print('init')
  self.x =3

 def __del__(self):print('del')A() #Return a class<__main__.A at 0x7f4a84767978>
# Output
newinit

a =A()
del a  #Output del

Whenever the instance space is reclaimed (during garbage collection), __del__ will be executed automatically.

Operator overloading#

classPoint:
 def __init__(self, x, y):
  self.x = x
  self.y = y

 def __add__(self, other):returnPoint(self.x + other.x, self.y + other.y)

 def __sub__(self, other):returnPoint(self.x - other.x, self.y - other.y)

a =Point(0,0)
b =Point(3,5)
c = a + b
c +=Point(4,6)print(c.x, c.y)  # 7,11  
p =Point(3,5)-Point(2,1)print(p.x, p.y)  # 1,4

Addition and subtraction operations can be performed between objects of a class, as long as the class implements the magic methods corresponding to the addition and subtraction operations. The concrete realization of addition is __add__, and the concrete realization of subtraction is __sub__.

The overloaded functions corresponding to specific operators can refer to the implementation of operator overloading in the int class: help(int)

Don't overuse operator overloading

Point.__add__ = lambda self, value: self - value
p =Point(3,5)+Point(4,6)print(p.x, p.y)  #Output-1,-1

__ If the specific implementation of add__ is written as a subtraction, this type of error is very difficult to find. Therefore, if you are not writing a library for use by a third party, you basically cannot use operator overloading.

hash

When using the built-in function hash to calculate the hash value of an object, the object's __hash__ method will be called. The sample code is as follows

In [1]:classPoint:...:     def __hash__(self):...:return1...:     

In [2]:hash(Point())
Out[2]:1

__ hash__ method must return int, otherwise TypeError will be thrown

In [1]:classPoint:...:     def __hash__(self):...:return'aaa'...:     

In [2]:hash(Point())---------------------------------------------------------------------------
TypeError                                 Traceback(most recent call last)<ipython-input-5-a919dcea3eae>in<module>()---->1hash(Point())

TypeError: __hash__ method should return an integer

A hash object is an object with a __hash__ method

In [6]:classPoint:...:     def __hash__(self):...:return1...:         

In [7]:set([Point(),12]) #Can hash
Out[7]:{<__main__.Point at 0x7f19d4073320>,12}

In [8]: Point.__hash__ = None

In [9]:set([Point(),12])  #Cannot be placed in the collection because it cannot be hashed
---------------------------------------------------------------------------
TypeError                                 Traceback(most recent call last)<ipython-input-10-25999920b521>in<module>()---->1set([Point(),12])

TypeError: unhashable type:'Point'

If a class does not override the __hash__ method, each object of this class usually has a different hash

In [1]:classPoint:...:     pass
   ...: 

In [2]: p1 =Point()

In [3]: p2 =Point()

In [4]:hash(p1)
Out[4]:8757059543567

In [5]:hash(p2)
Out[5]:8757059543756

Usually __hash__ will be used together with __eq__, because the interpreter usually judges whether the hash is equal and whether the instance is equal

classPoint:
 def __init__(self, x, y):
  self.x = x
  self.y = y

 def __hash__(self):returnhash('{}:{}'.format(self.x, self.y))

 def __eq__(self, other):return self.x == other.x and self.y == other.y

p1 =Point(3,5)
p2 =Point(3,5)set([p1, p2])  #return{<__main__.Point at 0x7f286092d588>}hash(p1)==hash(p2)  #Return True
p1 == p2  #Return True

size#

When the object implements the __len__ method, you can use the built-in method len to find the length of the object, and the __len__ method must return a non-negative integer

lst =[1,2,3]len(lst)  #Returns 3
lst.__len__()  #Returns 3

Therefore, the built-in function and the __len__ method have the same effect.

classSized:
 def __len__(self):return10len(Sized())  #Returns 10

bool

When the object o implements the __bool__ method, the return value of bool(o) is o.__bool__()

classF:
 def __bool__(self):return False

bool(F())  #Return False

classT:
 def __bool__(self):return True

bool(T())  #Return True

When the object o does not implement the __bool__ method, if o implements the __len__ method, the return value of bool(o) is len(o) != 0

classL:
 def __len__(self):return3bool(L())  #Return True

classQ:
 def __len__(self):return0bool(Q())  #Return False

When the object o neither implements the __bool__ method nor the __len__ method, the return value of bool(o) is True

classBoolean:
 pass

bool(Boolean())  #Return True

__ bool__ has higher priority than __len__

classSized:
 def __init__(self, size):
  self.size = size

 def __len__(self):return self.size

 def __bool__(self):return self.size ==0bool(Sized(0))  #Return True
bool(Sized(10))  #Return False

__ bool__ method must return bool type

classB:
 def __bool__(self):return None  #An error will occur when returning a value of non-bool type, even if an int type is returned, an error will be reported

bool(B())---------------------------------------------------------------------------
TypeError                                 Traceback(most recent call last)<ipython-input-80-4efbb03885fe>in<module>()---->1bool(B())

TypeError: __bool__ should return bool, returned NoneType

Visualization#

__ The str__ method, the print function essentially calls the object’s __str__ method for reading
__ The repr__ method, the repr function essentially calls the object’s __repr__ method for reading to the machine

classPoint:
 def __init__(self, x, y):
  self.x = x
  self.y = y

 def __str__(self):  #To read
  return'Point<{}, {}>'.format(self.x, self.y)

 def __repr__(self): #Machine-readable
  return'Point({}, {})'.format(self.x, self.y)print(Point(3,5))  # Point<3,5>print(repr(Point(3,5)))  # Point(3,5)

repr: returns the normalized string representation of the object

Callable object#

classFn:
 def __call__(self):print('{} called'.format(self))

f =Fn()f()

# Output
<__ main__.Fn object at 0x7fd254367470> called

An object, as long as it implements the __call__ method, can be called through parentheses. This type of object is called a callable object

Adding a function to an object means adding parameters to the __call__ method:

classAdd:
 def __call__(self, x, y):return x + y

Add()(3,5)  #Returns 8, which is equivalent to add=Add()add(3,5)

Application examples of callable objects: implement expirable and swappable cache decorators

import inspect
import datetime
from functools import wraps

classCache:
 def __init__(self, size=128, expire=0):
  self.size = size
  self.expire =0
  self.data ={}

 @ staticmethod
 def make_key(fn, args, kwargs):
  ret =[]
  names =set()
  params = inspect.signature(fn).parameters
  keys =list(params.keys())for i, arg inenumerate(args):
   ret.append((keys[i], arg))
   names.add(keys[i])
  ret.extend(kwargs.items())
  names.update(kwargs.keys())for k, v in params.items():if k not in names:
    ret.append((k, v.default))
  ret.sort(key=lambda x: x[0])return'&'.join(['{}={}'.format(name, arg)for name, arg in ret])

 def __call__(self, fn):
  @ wraps(fn)
  def wrap(*args,**kwargs):
   key = self.make_key(fn, args, kwargs)
   now = datetime.datetime.now().timestamp()if key in self.data.keys():
    value, timestamp, _ = self.data[key]if expire ==0 or now - timestamp < expire:
     self.data[key]=(value, timestamp, now)return value
    else:
     self.data.pop(key)
   value =fn(*args,**kwargs)iflen(self.data)>= self.size: 
    # Expired cleanup
    if self.expire !=0:
     expires =set()for k,(_, timestamp, _)in self.data.items():if now - timestamp >= self.expire:
       expires.add(k)for k in expires:
      self.data.pop(k)iflen(self.data)>= self.size:
    # Swap out
    k =sorted(self.data.items(), key=lambda x: x[1][2])[0][0]
    self.data.pop(k)
   self.data[key]=(value, now, now)return value
  return wrap

@ Cache()
def add(x, y):return x + y

add(1,2)  #Returns 3

Use __call__ to implement callable objects, and closures have the same goal, usually to encapsulate some internal state

Context management#

Objects that support context management##

classContext:
 def __enter__(self):print('enter context')

 def __exit__(self,*args,**kwargs):print('exit context')

When an object implements both __enter__ and __exit__ methods, then this object is an object that supports context management.

Objects that support context management can be processed using the following statement blocks:

with obj:
 pass

such as

withContext():print('do somethings')print('out of context')

# Output
enter context
do somethings
exit context
out of context

Therefore, with opens a statement block. Before executing this statement block, the __enter__ method will be executed. After executing this statement block, the __exit__ method will be executed, which means that some operations will be performed before and after this statement block. So it is also called context.

**Even if the with block throws an exception, __enter__ and __exit__ will be executed, so context management is safe. **

withContext():
 raise Exception()

enter context
exit context
---------------------------------------------------------------------------
Exception                                 Traceback(most recent call last)<ipython-input-126-c1afee4bfdab>in<module>()1withContext():---->2     raise Exception()

Exception:

Even if the interpreter is actively exited in the with block, __enter__ and __exit__ can still be executed

import sys

withContext():
 sys.exit()

enter context
exit context
An exception has occurred, use %tb to see the full traceback.

SystemExit

/home/clg/.pyenv/versions/3.5.2/envs/normal/lib/python3.5/site-packages/IPython/core/interactiveshell.py:2889: UserWarning: To exit: use 'exit','quit', or Ctrl-D.warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

with block of as word##

The as clause can get the return value of theenter` method

classContext:
 def __enter__(self):print('enter context')return self  # __enter__The return value of the function

 def __exit__(self,*args,**kwargs):print('exit context')

ctx =Context()with ctx as c:print(id(ctx))print(id(c))print(c)

# Output result
enter context
140541332713712140541332713712<__ main__.Context object at 0x7fd2543670f0>
exit context

` enter`method##

__ The return value of the enter__ method can be captured by the as clause
__ enter__ does not take any parameters except self

classContext:
 def __enter__(self,*args,**kwargs):print('enter context')print(args)print(kwargs)

 def __exit__(self,*args,**kwargs):print('exit context')

# Output
enter context(){}
exit context

Both args and kwargs are empty, so the __enter__ function does not take any parameters except self during context management.

` exit`method##

__ There is no way to get the return value of exit__. If an exception is thrown in the with block when __exit__ returns False, an exception will be thrown upwards, and True will be returned, and the exception will be shielded

classContext:
 def __enter__(self):print('enter context')

 def __exit__(self,*args,**kwargs):print('exit context')return'haha'withContext()as c:print(c)

# Output
enter context
None
exit context

__ The three parameter exception types of exit__, exception, traceback

classContext:
 def __enter__(self):print('enter context')

 def __exit__(self,*args,**kwargs):print('exit context')print(args)print(kwargs)withContext():
 pass

# Output
enter context
exit context(None, None, None){}

args outputs three None, which means three positional parameters, and kwargs is empty, which means there are no keyword arguments.

withContext():
 raise Exception()

enter context
exit context(<class'Exception'>,Exception(),<traceback object at 0x7f28608fdc88>){}---------------------------------------------------------------------------
Exception                                 Traceback(most recent call last)<ipython-input-145-c1afee4bfdab>in<module>()1withContext():---->2     raise Exception()

Exception:

Use variables to accept the three parameters of __exit__: exc_type, exc_value, traceback

classContext:
 def __enter__(self):print('enter context')

 def __exit__(self, exc_type, exc_value, traceback):print('exit context')print('exception type: {}'.format(exc_type))print('exception value: {}'.format(exc_value))print('exception traceback: {}'.format(traceback))return True

withContext():
 raise TypeError('hahaha')

# Output
enter context
exit context
exception type:<class'TypeError'>
exception value: hahaha
exception traceback:<traceback object at 0x7fd257c18608>

Application scenarios of context management##

The with statement is suitable for accessing resources to ensure that necessary "cleanup" operations are performed regardless of whether an exception occurs during use, and resources are released, such as automatic closing of files after use, automatic acquisition and release of locks in threads, etc. That is, all scenarios where the code is inserted before and after the code block are applicable**

Resource management
ASD

Take the timer as an example below

from functools import wraps
classTimeit:
 def __init__(self, fn=None):wraps(fn)(self)

 def __call__(self,*args,**kwargs):
  start = datetime.datetime.now()
  ret = self.__wrapped__(*args,**kwargs)
  cost = datetime.datetime.now()- start
  print(cost)return ret

 def __enter__(self):
  self.start = datetime.datetime.now()

 def __exit__(self,*args):
  cost = datetime.datetime.now()- self.start
  print(cost)withTimeit():
 z =3+8  #Output 0:00:00.000037

@ Timeit
def add(x, y):return x + y

add(3,8)  #Output 0:00:00.000044 returns 11

A total of two timing methods have been implemented, which can be used to time statement blocks or functions.

Use of contextmanager##

Contextlib is a more beautiful thing than with, and it is also a module that provides a context management mechanism. It is implemented through the Generator decorator instead of using __enter__ and __exit__. The contextmanager in contextlib serves as a decorator to provide a context management mechanism for the function level.

import contextlib

@ contextlib.contextmanager
def context():print('enter context') #The initialization part is equivalent to__enter__method
 try:yield'haha' #Equivalent to__enter__The return value
 finally:print('exit context') #Clean up part, equivalent to__exit__method

withcontext()as c:print(c)
 raise Exception()

# Output
enter context
haha
exit context
---------------------------------------------------------------------------
Exception                                 Traceback(most recent call last)<ipython-input-189-4c1dae6b647a>in<module>()1withcontext()as c:2print(c)---->3     raise Exception()

Exception:

After yield, it must be used with finally, otherwise, if an exception is thrown, the program will not execute the department behind yield, that is, the part of __exit__ will not be executed.

reflection#

**Python's reflection, the core essence is actually to use the form of strings to manipulate (find/get/delete/add) members in objects (modules), which is a string-based event-driven! **

For the python reflection and reflection mechanism analysis of the module, please refer to: python reflection mechanism in-depth analysis

The following mainly analyzes the reflection mechanism of class objects

getattr setattr hasattr

The prototypes of the three functions:

getattr: getattr(object, name[, default]) -> value. getattr(x,'y') is equivalent to xy
setattr: setattr(obj, name, value, /). setattr(x,'y', v) is equivalent to xy = v
hasattr：hasattr(obj, name, /)

The main function is to get the members of the object by the member name of the object

classPoint:
 def __init__(self, x, y):
  self.x = x
  self.y = y

 def print(self, x, y):print(x, y)

p =Point(3,5)
p.__dict__['x'] #Return 3. For attributes, you can pass__dict__Obtain
getattr(p,'print')(3,5) #Member method failed__dict__Obtained, but can be obtained through the getattr function# p.print(3,5)getattr(p,'x') #getattrr can also get attributes
setattr(p,'haha','abcd') # p.haha ='abcd', Add attribute haha to object p
p.haha  #Return abcd
hasattr(p,'print')  #Return True

The object of setattr is an instance. If you want to add methods to the instance dynamically, you need to convert the function into a method first. The conversion method is as follows:

import types

def mm(self):print(self.x)setattr(p,'mm', types.MethodType(mm, p))  #After converting the mm function into the method of object p, add p
p.mm()  #Output 3

Use getattr setattr hasattr to implement a command router:

classCommand:
 def cmd1(self):print('cmd1')
 def cmd2(self):print('cmd2')
 def run(self):while True:
   cmd =input('>>>').strip()if cmd =='quit':returngetattr(self, cmd, lambda :print('not found cmd {}'.format(cmd)))()

command =Command()
command.run()

# Output
>>> cmd1
cmd1
>>> cmd2
cmd2
>>> cmd3
not found cmd cmd3
>>> quit

getattr``setattr``delattr

When a class defines the __getattr__ method, if a member that does not exist is accessed, the __getattr__ method will be called

classA:
 def __init__(self):
  self.x =3

a =A()
a.x  #Returns 3
a.y  #If not realized__getattr__Method, an error will be reported when accessing non-existent members
---------------------------------------------------------------------------
AttributeError                            Traceback(most recent call last)<ipython-input-228-cc7049c6eeec>in<module>()---->1 a.y

AttributeError:'A' object has no attribute 'y'

Add __getattr__ method

classA:
 def __init__(self):
  self.x =3

 def __getattr__(self, name):return'missing property {}'.format(name)

a =A()
a.x  #Returns 3
a.y  #return'missing property y'. That is, to access non-existent members, it will call__getattr__method

When a class implements __setattr__, anywhere to add attributes to objects of this class, or assign values to existing attributes, __setattr__ will be called

classA:
 def __init__(self):
  self.x =3

 def __setattr__(self, name, value):print('set {} to {}'.format(name, value))setattr(self, name, value)

a =A()
a.x  #Returns 3
a.y =5  #Output set y to 5

When a class implements the __delattr__ method, delete the attributes of its instance, this method will be called

classA:
 def __init__(self):
  self.x =3

 def __delattr__(self, name):print('you cannot delete property: {}'.format(name))

a =A()
a.x  #Returns 3
del a.x  #Output you cannot delete property: x

Python object-oriented magic method