Python decorator advanced usage

Full stack programmer webmaster 2021-04-07 12:27:45
python decorator advanced usage


stay Python in , Decorator Generally used to modify functions , Realize public functions , Achieve the purpose of code reuse . Add before function definition @xxxx, And then the function injects some behavior , Amazing ! However , This is just Grammatical sugar nothing more .

scene

hypothesis , There are some working functions , It's used to process data differently :

def work_bar(data):
pass
def work_foo(data):
pass

We want to do this before we call the function / Post output log , What do I do ?

Fool's solution

logging.info('begin call work_bar')
work_bar(1)
logging.info('call work_bar done')

If there are multiple code calls ? Think about it. !

Function packing

A fool's solution is nothing more than too much code redundancy , Write it every time you call a function logging. This part of redundant logic can be encapsulated in a new function :

def smart_work_bar(data):
logging.info('begin call: work_bar')
work_bar(data)
logging.info('call doen: work_bar')

such , Every time you call smart_work_bar that will do :

smart_work_bar(1)
# ...
smart_work_bar(some_data)

General closure

It looks perfect …… However , When work_foo There's the same need , We need to do it again smart_work_foo Do you ? It's obviously not scientific !

Don't worry. , We can use closures :

def log_call(func):
def proxy(*args, **kwargs):
logging.info('begin call: {name}'.format(name=func.func_name))
result = func(*args, **kwargs)
logging.info('call done: {name}'.format(name=func.func_name))
return result
return proxy

This function receives a function object ( Proxied function ) As a parameter , Returns a proxy function . When calling a proxy function , Output log first , Then the proxy function is called , Output the log after calling , Finally return the call result . such , The goal of generalization is achieved ?—— For any surrogate function func,log_call Easy to handle .

smart_work_bar = log_call(work_bar)
smart_work_foo = log_call(work_foo)
smart_work_bar(1)
smart_work_foo(1)
# ...
smart_work_bar(some_data)
smart_work_foo(some_data)

The first 1 In line ,log_call Receiving parameters work_bar, Returns a proxy function proxy, And give it to smart_work_bar. The first 4 In line , call smart_work_bar, That is, the proxy function proxy, Output log first , And then call func That is to say work_bar, Finally, output the log . be aware , In proxy function ,func And passed in work_bar Objects are closely related , This is it. Closure .

Let me mention it again. , You can override the proxy function name , With smart_ It's a bit cumbersome to take a new name for the prefix :

work_bar = log_call(work_bar)
work_foo = log_call(work_foo)
work_bar(1)
work_foo(1)

Grammatical sugar

Let's look at the following code first :

def work_bar(data):
pass
work_bar = log_call(work_bar)
def work_foo(data):
pass
work_foo = log_call(work_foo)

Although there is no redundancy in the code , But it's not intuitive to see . Now , Here comes the grammar candy ~~~

@log_call
def work_bar(data):
pass

therefore , Be careful. ( Key points. ), here @log_call It's just : tell Python Compiler insert code work_bar = log_call(work_bar).

Evaluate decorator

Guess the decorator first eval_now What's the role ?

def eval_now(func):
return func()

It looks curious , No proxy function defined , Is it a decorator ?

@eval_now
def foo():
return 1
print foo

This code output 1, That is to call and evaluate the function . So what's the use ? direct writing foo = 1 No way? ? In this simple example , Of course that's OK . Let's take a more complicated example —— Initialize a log object :

# some other code before...
# log format
formatter = logging.Formatter(
'[%(asctime)s] %(process)5d %(levelname) 8s - %(message)s',
'%Y-%m-%d %H:%M:%S',
)
# stdout handler
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setFormatter(formatter)
stdout_handler.setLevel(logging.DEBUG)
# stderr handler
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setFormatter(formatter)
stderr_handler.setLevel(logging.ERROR)
# logger object
logger = logging.Logger(__name__)
logger.setLevel(logging.DEBUG)
logger.addHandler(stdout_handler)
logger.addHandler(stderr_handler)
# again some other code after...

use eval_now The way :

# some other code before...
@eval_now
def logger():
# log format
formatter = logging.Formatter(
'[%(asctime)s] %(process)5d %(levelname) 8s - %(message)s',
'%Y-%m-%d %H:%M:%S',
)
# stdout handler
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setFormatter(formatter)
stdout_handler.setLevel(logging.DEBUG)
# stderr handler
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setFormatter(formatter)
stderr_handler.setLevel(logging.ERROR)
# logger object
logger = logging.Logger(__name__)
logger.setLevel(logging.DEBUG)
logger.addHandler(stdout_handler)
logger.addHandler(stderr_handler)
return logger
# again some other code after...

The purpose of the two pieces of code is the same , But the latter is clearly clearer , Code block style . what's more , Function calls complete initialization in the local namespace , Avoid temporary variables ( Such as formatter etc. ) Pollute the outer name space ( For example, the overall situation ).

With parameter decorator

Define a decorator , Used to record slow function calls :

def log_slow_call(func):
def proxy(*args, **kwargs):
start_ts = time.time()
result = func(*args, **kwargs)
end_ts = time.time()
seconds = start_ts - end_ts
if seconds > 1:
logging.warn('slow call: {name} in {seconds}s'.format(
name=func.func_name,
seconds=seconds,
))
return result
return proxy

The first 35 Line samples the current time before and after the function call , The first 7 Row calculation call time consuming , It takes more than one second to output a warning log .

@log_slow_call
def sleep_seconds(seconds):
time.sleep(seconds)
sleep_seconds(0.1) # No log output
sleep_seconds(2) # Output warning log 

However , The threshold setting always depends on the situation , Different functions may set different values . If only the threshold could be parameterized :

def log_slow_call(func, threshold=1):
def proxy(*args, **kwargs):
start_ts = time.time()
result = func(*args, **kwargs)
end_ts = time.time()
seconds = start_ts - end_ts
if seconds > threshold:
logging.warn('slow call: {name} in {seconds}s'.format(
name=func.func_name,
seconds=seconds,
))
return result
return proxy

However ,@xxxx Syntax sugar always calls decorators with decorated functions as parameters , That is to say, there is no chance to deliver threshold Parameters . What shall I do? ?—— Encapsulate with a closure threshold Parameters :

def log_slow_call(threshold=1):
def decorator(func):
def proxy(*args, **kwargs):
start_ts = time.time()
result = func(*args, **kwargs)
end_ts = time.time()
seconds = start_ts - end_ts
if seconds > threshold:
logging.warn('slow call: {name} in {seconds}s'.format(
name=func.func_name,
seconds=seconds,
))
return result
return proxy
return decorator
@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
time.sleep(seconds)

such ,log_slow_call(threshold=0.5) Call return function decorator, Function has closure variable threshold, The value is 0.5.decorator Re decoration sleep_seconds.

Apply default threshold , Function calls still cannot be omitted :

@log_slow_call()
def sleep_seconds(seconds):
time.sleep(seconds)

Virgo may feel uncomfortable with the first pair of brackets , So we can improve it like this :

def log_slow_call(func=None, threshold=1):
def decorator(func):
def proxy(*args, **kwargs):
start_ts = time.time()
result = func(*args, **kwargs)
end_ts = time.time()
seconds = start_ts - end_ts
if seconds > threshold:
logging.warn('slow call: {name} in {seconds}s'.format(
name=func.func_name,
seconds=seconds,
))
return result
return proxy
if func is None:
return decorator
else:
return decorator(func)

This writing is compatible with two different uses , usage A Default threshold ( No call ); usage B Custom threshold ( Have call ).

# Case A
@log_slow_call
def sleep_seconds(seconds):
time.sleep(seconds)
# Case B
@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
time.sleep(seconds)

usage A in , What happened was log_slow_call(sleep_seconds), That is to say func Parameter is not empty , It's a direct tune decorator Pack and return ( Threshold is the default ).

usage B in , What happened first log_slow_call(threshold=0.5),func The parameter is empty. , Go straight back to the new decorator decorator, Associate closure variable threshold, The value is 0.5; then ,decorator Redecoration function sleep_seconds, namely decorator(sleep_seconds). be aware , here threshold The associated value is 0.5, Complete customization .

You may have noticed , It's better to use the keyword parameter in this way —— Using positional parameters can be ugly :

# Case B-
@log_slow_call(None, 0.5)
def sleep_seconds(seconds):
time.sleep(seconds)

Yes, of course , Use keyword arguments as much as possible for function calls Is an excellent practice , Clear meaning , This is especially true when there are many parameters .

Intelligent decorator

The writing method introduced in the previous section , More nesting levels , If every similar decorator is implemented in this way , It's still a lot of work ( Not enough brain ), It's also easy to make mistakes .

Suppose there's an intelligent decorator smart_decorator, Decorator log_slow_call, You can get the same ability . such ,log_slow_call Definition will become clearer , It's easier to realize :

@smart_decorator
def log_slow_call(func, threshold=1):
def proxy(*args, **kwargs):
start_ts = time.time()
result = func(*args, **kwargs)
end_ts = time.time()
seconds = start_ts - end_ts
if seconds > threshold:
logging.warn('slow call: {name} in {seconds}s'.format(
name=func.func_name,
seconds=seconds,
))
return result
return proxy

Brain opening ,smart_decorator How to achieve it ? Actually, it's simple :

def smart_decorator(decorator):
def decorator_proxy(func=None, **kwargs):
if func is not None:
return decorator(func=func, **kwargs)
def decorator_proxy(func):
return decorator(func=func, **kwargs)
return decorator_proxy
return decorator_proxy

smart_decorator After that , That's the idea ! At this time ,log_slow_call, Namely decorator_proxy( Outer layer ), Associated closure variable decorator Is defined at the beginning of this section log_slow_call( To avoid ambiguity , be called real_log_slow_call).log_slow_call The following uses are supported :

# Case A
@log_slow_call
def sleep_seconds(seconds):
time.sleep(seconds)

usage A in , Execution is decorator_proxy(sleep_seconds)( Outer layer ),func Non empty ,kwargs It's empty ; Direct execution decorator(func=func, **kwargs), namely real_log_slow_call(sleep_seconds), The result is associated with the default parameter proxy.

# Case B
# Same to Case A
@log_slow_call()
def sleep_seconds(seconds):
time.sleep(seconds)

usage B in , Execute first decorator_proxy(),func And kwargs All empty. , return decorator_proxy object ( Inner layer ); Re execution decorator_proxy(sleep_seconds)( Inner layer ); Finally, execute decorator(func, **kwargs), Equivalent to real_log_slow_call(sleep_seconds), Effect and usage A Agreement .

# Case C
@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
time.sleep(seconds)

usage C in , Execute first decorator_proxy(threshold=0.5),func Empty but kwargs Non empty , return decorator_proxy object ( Inner layer ); Re execution decorator_proxy(sleep_seconds)( Inner layer ); Finally, execute decorator(sleep_seconds, **kwargs), Equivalent to real_log_slow_call(sleep_seconds, threshold=0.5), Threshold implementation customization !

Participation of this paper Tencent cloud media sharing plan , You are welcome to join us , share .

版权声明
本文为[Full stack programmer webmaster]所创,转载请带上原文链接,感谢
https://pythonmana.com/2021/04/20210407122137388s.html

  1. Python brush questions - letter graphics
  2. Python数据分析入门(七):Pandas层级索引
  3. Introduction to Python data analysis (7): Pandas hierarchical index
  4. Python 操作腾讯云短信(sms)详细教程
  5. Python operation Tencent cloud SMS (SMS) detailed tutorial
  6. Python数据可视化,完整版实操指南 !
  7. Python data visualization, full version of the practical guide!
  8. 上手Pandas,带你玩转数据(2)-- 使用pandas从多种文件中读取数据
  9. 上手Pandas,带你玩转数据(1)-- 实例详解pandas数据结构
  10. Using pandas to read data from various files
  11. Hands on pandas, take you to play with data (1) -- detailed explanation of pandas data structure with examples
  12. Pandas数据结构基础用法
  13. Basic usage of pandas data structure
  14. Python读取ini配置文件,保存到对象属性
  15. Python reads the INI configuration file and saves it to the object properties
  16. Foundation of Python: classes in Python
  17. python刷题-闰年判断
  18. python刷题-01字串
  19. How to judge leap year
  20. Python brush title-01 string
  21. 安装python
  22. 按尺寸切片pandas数据集DataFrame到多个文件
  23. Install Python
  24. Slice the pandas dataset dataframe to multiple files by size
  25. python 求最大值、最小值、平均值
  26. Finding maximum, minimum and average in Python
  27. 认识Python解释器和PyCharm编辑器
  28. Know Python interpreter and pycharm editor
  29. Python 小数据池和代码块缓存机制
  30. Python small data pool and code block caching mechanism
  31. python刷题-序列求和
  32. python刷题-圆的面积
  33. Sequence summation in Python
  34. The area of a circle
  35. Python functions, advanced syntax and usage
  36. Teach you to crawl novels in Python! Who can pay for novels these days!
  37. Python入门学习之:10分钟1500访问量
  38. Introduction to Python: 1500 visits in 10 minutes
  39. 数据分析之Pandas合并操作总结
  40. OpenCV-Python 雪花飘落特效
  41. Pandas merge operation summary of data analysis
  42. Opencv Python snowflake falling effect
  43. python logging模块“另一个程序正在使用此文件,进程无法访问。”问题解决办法
  44. Python logging module "this file is being used by another program and cannot be accessed by the process." Problem solving
  45. Mac 下python3 [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 解决方法
  46. Python 3 [SSL: Certificate] on MAC_ VERIFY_ Failed] certificate verify failed solution
  47. Python学习之解决python下载第三方依赖速度慢的问题
  48. Python learning to solve the problem of slow download speed of third party dependence on Python
  49. python操作Excel文件报lrd.biffh.XLRDError
  50. How to operate excel file with Python lrd.biffh.XLRDError
  51. 2021的挑战与机遇,今年Python数据分析岗位会很香!
  52. The challenge and opportunity of 2021, python data analysis post will be very popular this year!
  53. 【C++简明教程】Python和C++指定元素排序比较
  54. Comparison of Python and C + + specified element sorting
  55. Python Flask使用Nginx做代理时如何获取真实IP
  56. How to get real IP address when Python flash uses nginx as proxy
  57. Python培训出来好找工作吗?好找工作的关键是什么?
  58. Is Python training easy to find a job? What is the key to finding a good job?
  59. 从零开始学python | 什么是Python JSON?
  60. Learn Python from scratch | what is Python JSON?