Python code runs slowly? Take off at your speed with pypy module!

Watch the corner. 2020-11-14 10:56:38
python code runs slowly speed


 Insert picture description here

Python Is one of the most commonly used programming languages among developers , But it has some limitations . for example , For some applications , It may run slower than other languages 100 times . That's why Python After the speed of running has become a bottleneck for users , Many companies rewrite their applications in another language . But is there a way to keep Python It can improve the speed ? It is PyPy.

PyPy It's a very compatible Python Interpreter , It is CPython2.7、3.6 And the upcoming 3.7 A worthy alternative to . Use it when you install and run an application , Can significantly increase speed . How much faster depends on the application you're running .

In this tutorial , You will learn :

  • How to use PyPy Install and run code
  • PyPy And CPython The comparison in terms of speed
  • PyPy The function of and how to make Python Code runs faster
  • The examples in this tutorial use Python 3.6 , Because it is PyPy Compatible with the latest Python edition .

PyPy brief introduction

Python An interpreter can be implemented in many languages , Such as CPython( use C To write )、Jython( use Java To write )、Iron Python( use .NET To write ) and PyPy( use Python To write ).

CPython yes Python The initial implementation of the interpreter , It's also the most widely used and most maintained . When we are from Python Download and install the official website Python 3.x after , We've got an official version of the interpreter :CPython. This interpreter uses C Language development , So called CPython. Run on the command line python That is to start. CPython Interpreter .

however , because CPython It's a high-level interpretation language , So it has some limitations , And there's no advantage in terms of speed . This is it. PyPy Where it works . Because it conforms to Python language norm , therefore Py Py There is no need to make any changes to the code base , And you can significantly increase the speed through the following functions .

Now? , You may want to know , If CPython Use the same syntax , Why doesn't it come true Py Py The power of . as a result of , Implementing these features requires significant changes to the source code , It's going to be a lot of work .

Let's take a quick look at how to use PyPy.


Many people study python, I don't know where to start .
Many people study python, After mastering the basic grammar , I don't know where to look for cases to start .
A lot of people who have done cases , But I don't know how to learn more advanced knowledge .
So for these three kinds of people , I will provide you with a good learning platform , Get a free video tutorial , electronic text , And the source code of the course !
QQ Group :1156465813

install

Your operating system may already offer PyPy software package . for example , stay Mac OS On , You can go to Homebrew Install it with the help of :

 brew install pypy3

Or you can download binaries that match the operating system . After downloading , Just open tarball or ZIP File can . then , You can do the following :

$ tar xf pypy3.6-v7.3.1-osx64.tar.bz2
$ ./pypy3.6-v7.3.1-osx64/bin/pypy3
Python 3.6.9 (?, Jul 19 2020, 21:37:06)
[PyPy 7.3.1 with GCC 4.2.1]
Type "help", "copyright", "credits" or "license" for more information.

You need to execute the command at the above folder address . For a complete explanation , Please refer to the installation documentation . function PyPy You have now installed Py Py, And it's about to run ! So , Please create one called script.py Of Python file , And put the following code into it :

total = 0
for i in range(1, 10000):
for j in range(1, 10000):
total += i + j
print(f"The result is {total}")

In two nested for In circulation , take 1 To 9,999 Add the numbers between , And print the results . See how long it takes to run this script :

import time
start_time = time.time()
total = 0
for i in range(1, 10000):
for j in range(1, 10000):
total += i + j
print(f"The result is {total}")
end_time = time.time()
print(f"It took {end_time-start_time:.2f} seconds to compute")

The code now does the following :

  • The first 3 Save the current time to the variable start_time.
  • The first 5 to 8 Line run cycle .
  • The first 10 Line print results .
  • The first 12 Save the current time as end_time.
  • The first 13 The difference between the start time and the end time of line printing , To show the time required to run the script .

use Python To run it . Here I am Mac Book Pro The result on :

$ python3.6 script.py
The result is 999800010000
It took 20.66 seconds to compute

Now use Py Py Run it :

$ pypy3 script.py
The result is 999800010000
It took 0.22 seconds to compute

In this little experiment ,PyPy The speed is about Python Of 94 times !

You can browse PyPy Speed Center To see more rigorous testing .

please remember ,PyPy How you affect the performance of your code depends on what you do with it . In some cases ,Py Py Actually slower , I'll see later . however , In terms of geometric average , Its speed is Python Of 4.3 times .

PyPy And its characteristics

Py Py There are two definitions :

1、 A dynamic language framework for generating dynamic language interpreters 2、 Using the framework Python Realization
You should be aware of the second problem . You used Python Implementation is using what is called RPython Dynamic language framework written by , It's like CPython Yes, it is C Compiling , and Jython Yes, it is Java It's written the same way .
But I didn't mention PyPy Yes, it is Python Written by ? Um. , It's a little simple .PyPy Become useful Python Compiling Python Interpreter ( instead of RPython) The reason for that is RPython Used with Python The same grammar .

PyPy How did you get it ? The following points need to be explained :

1、 Its source code is to use RPython To write .
2、RPython Conversion tools are applied to the code , Fundamentally improve code efficiency , You can also compile code into machine code , This is it. Mac,Windows and Linux Why users have to download different versions .
3、 Binary executable file generated in the above way , That's what you run Python Interpreter .
You don't need to perform all of these steps to use PyPy. Because there are already executable files for you to install and use .
Besides , Because it's confusing to use the same word in both the framework and the implementation ,PyPy The team behind it decided to give up the double use . Now? ,PyPy Only Python Interpreter , And the framework is called RPython Transformation tool .
Next , You will find out under what circumstances PyPy Than Python Better 、 faster .

Just-In-Time (JIT) compiler
In understanding JIT Before the contents of the compiler , Let's review the compiled language first ( Such as C) And explanatory language ( Such as JavaScript) Characteristics of .

Before a program written in a compiled language is executed , A special compilation process is needed , Compile the source code into a machine language file , Such as exe File format , When you want to run again later , You can use the compilation results directly , Such as direct operation exe file . Because it only needs to be compiled once , You don't need to compile later , So compiler language is efficient . Platform specific , Generally, it can't be ported to other platforms . Such as C、C++、Objective All of them belong to compiler language .

Interpretive languages don't need to be compiled in advance , It directly interprets the source code into machine code and executes it immediately , So as long as a platform provides the corresponding interpreter can run the program . Every time an interpreted language runs, it needs to interpret the source code as machine code and execute it , Low efficiency ; As long as the platform provides the corresponding interpreter , You can run the source code , So it is easy to transplant the source program .

And then there are programming languages , for example Python, It's a mix of compilation and interpretation . say concretely ,Python First it's compiled into bytecode , Then from CPython explain . This makes code perform better than code written in a pure interpreted language , And remain portable Sexual advantage .

But its performance is still far lower than compiled languages . The reason is , The compiled code can perform many optimizations , And bytecode is impossible .

This is it. JIT The source of the compiler . It tries to gain both advantages by compiling and interpreting machine code . In short , Here are JIT Steps taken to compile to provide faster performance :

1、 Identify the most commonly used components in code , Like functions in loops .
2、 The runtime converts these parts into machine code .
3、 Optimize the generated machine code .
4、 Replace the previous implementation with an optimized version of machine code .

Remember the two nested loops at the beginning of the tutorial ?PyPy When duplicate operations are detected , Compile it into machine code , Optimize machine code , And then transform to implement . That's why you see this result .

Garbage collection mechanism
Whenever you create a variable 、 Function or any other object , Your computer will allocate memory to them . Final , Some of these objects will no longer need . If not cleaned up in time , Computers can run out of memory and crash programs .

stay C and C++ In programming languages , Usually you have to deal with this manually . Other programming languages ( Such as Python and Java) This will be done automatically for you . This is called the automatic garbage collection mechanism .

CPython Using a technique called reference counting . In essence , Whenever an object is referenced ,Python Object's reference count will increase , When the object is dereferenced, the count is decremented . When the reference count is zero ,CPython The memory release function is automatically called for the object . It's a simple and effective technique , But there is a trap .

When the reference count of a large object tree becomes zero , All related objects will be released . therefore , You may have a long pause , In the meantime, your program can't be executed at all .

Besides , Another example , Reference counting doesn't work at all . As shown below :

class A(object):
pass
a = A()
a.some_property = a
del a

In the code above , Defines a new class , then , Create an instance , And specify it as its own property . Last , Delete the instance .

here , Instances will no longer be accessible . however , Reference counting does not delete instances from memory , Because it has a reference to itself , So the reference count is not zero . This problem is called a reference loop , Cannot use reference counting to resolve .

This is a CPython Another tool used , It's called a recycling garbage collector . It comes from known roots ( Such as type objects ) Start traversing all objects in memory . then , It identifies all accessible objects , And release inaccessible objects , Because they no longer exist . This solves the reference loop problem . however , When there are a large number of objects in memory , It may create more obvious pauses .

On the other hand ,PyPy Don't use reference counting . contrary , It only uses the second Technology , The loop finder . in other words , It periodically traverses the active object from the root . This makes PyPy Than CPython It has some advantages , Because it doesn't need to consider reference counting , Thus, the total memory management time is less than CPython.

Besides ,PyPy Split the work into a variable number of parts , And run each part , Until there's no rest . This method only adds a few milliseconds after each secondary collection , Not like CPython Add hundreds of milliseconds at a time .

The garbage collection mechanism is very complicated , And there's a lot beyond the scope of this tutorial . You can find information about PyPy Details of the garbage collection mechanism .

PyPy The limitations of

PyPy It's not everything , It's not a tool for all your tasks . It may even make the application run faster than CPython Much slower . That's why you have to remember the following limitations .

It doesn't apply to C Expand
PyPy Perfect for pure Python Applications . Whenever used C Extension module , It runs faster than in CPython It's much slower . as a result of PyPy Can't optimize C Extension module , Because they are not fully supported . Besides ,PyPy Must simulate reference counting in code , Make it slower .

under these circumstances ,PyPy The team suggests removing CPython Expand and replace it with pure Python edition . If not , Must be used CPython.

For all that , The core team is working on C Expand . Some packages have been ported to PyPy, And it works just as fast .

It only works with long running programs
Imagine you want to go to a store close to your home . You can walk directly to , You can also drive .

Your car is obviously much faster than your feet . however , Please consider the steps you need to complete :
1. Go to your garage .
2、 Driving a car .
3、 Warm up the car .
4、 Drive to the store .
5、 Looking for parking space .
6、 Repeat the process on the way back .

Driving requires a series of troublesome steps , If the place you want to go is nearby , That's not necessarily worth it .

Now think about it. , If you want to go 50 Kilometers away from the neighboring city , What's going to happen ? It must be worth driving there , Instead of walking to .

Although the contrast in speed is not as obvious as the analogy above , but PyPy and CPython The same is true of this .

When using PyPy When running scripts , It does a lot of things to make the code run faster . If the script itself is simple , The actual script will run faster than CPython.
On the other hand , If you have a long running script , That could lead to significant performance improvements .

If you want to feel it for yourself , Please be there. CPython and PyPy Run the following script in :

import time
start_time = time.time()
for i in range(100):
print(i)
end_time = time.time()
print(f"It took {end_time-start_time:.10f} seconds to compute")

When you use PyPy When it runs , There will be a small delay at the beginning , and CPython It will run immediately . stay Mac Book Pro Run it on , use CPython need 0.0004873276 second , use PyPy need 0.0019447803 second .

It doesn't compile ahead of time
As you can see at the beginning of this tutorial ,PyPy It's not a fully compiled Python Realization . It compiles Python Code , But it's not Python The compiler of the code . because Python Some inherent characteristics , It's impossible to put Python Compile it into a separate binary and reuse it .

Py Py Faster than fully explanatory language , But more than a fully compiled language ( Such as C) slow .

summary

PyPy yes CPython A fast and powerful alternative to . Use it to run scripts , You can greatly increase speed without changing the code . But it's not everything , There are some limitations .

In this tutorial , You learned :

  • PyPy What is it? ?

  • How to install PyPy And use it to run scripts

  • PyPy And CPython The comparison in terms of speed

  • PyPy How to improve the program speed

  • In which cases PyPy There will be limitations

If your Python Scripts need to be a little bit faster , Welcome to try PyPy!

 

Be careful : If you're looking for python Well paid jobs . I suggest you write more about real enterprise projects and accumulate experience . Or you won't find a job , Of course, a lot of people have never been in a business , How can there be project experience ? So you have to find more enterprise projects and practice more . If you're lazy and don't want to find , You can also enter my Python Circle of communication :1156465813. There are some real enterprise project cases that I have written before in the group file . You can take it to study , If you don't understand, you can find me in your skirt , I'll answer you patiently when I have time .

 

The following is useless , For this blog to be crawled by search engines
(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)
python What is it Zero basis to learn python How long will it take? python Why is it called a reptile
python Novice reptile tutorial python Crawler universal code python How do reptiles make money
python Basic course Web crawler python python Classic examples of reptiles
python Reptiles
(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)(* ̄︶ ̄)
The above is useless , For this blog to be crawled by search engines

版权声明
本文为[Watch the corner.]所创,转载请带上原文链接,感谢

  1. 利用Python爬虫获取招聘网站职位信息
  2. Using Python crawler to obtain job information of recruitment website
  3. Several highly rated Python libraries arrow, jsonpath, psutil and tenacity are recommended
  4. Python装饰器
  5. Python实现LDAP认证
  6. Python decorator
  7. Implementing LDAP authentication with Python
  8. Vscode configures Python development environment!
  9. In Python, how dare you say you can't log module? ️
  10. 我收藏的有关Python的电子书和资料
  11. python 中 lambda的一些tips
  12. python中字典的一些tips
  13. python 用生成器生成斐波那契数列
  14. python脚本转pyc踩了个坑。。。
  15. My collection of e-books and materials about Python
  16. Some tips of lambda in Python
  17. Some tips of dictionary in Python
  18. Using Python generator to generate Fibonacci sequence
  19. The conversion of Python script to PyC stepped on a pit...
  20. Python游戏开发,pygame模块,Python实现扫雷小游戏
  21. Python game development, pyGame module, python implementation of minesweeping games
  22. Python实用工具,email模块,Python实现邮件远程控制自己电脑
  23. Python utility, email module, python realizes mail remote control of its own computer
  24. 毫无头绪的自学Python,你可能连门槛都摸不到!【最佳学习路线】
  25. Python读取二进制文件代码方法解析
  26. Python字典的实现原理
  27. Without a clue, you may not even touch the threshold【 Best learning route]
  28. Parsing method of Python reading binary file code
  29. Implementation principle of Python dictionary
  30. You must know the function of pandas to parse JSON data - JSON_ normalize()
  31. Python实用案例,私人定制,Python自动化生成爱豆专属2021日历
  32. Python practical case, private customization, python automatic generation of Adu exclusive 2021 calendar
  33. 《Python实例》震惊了,用Python这么简单实现了聊天系统的脏话,广告检测
  34. "Python instance" was shocked and realized the dirty words and advertisement detection of the chat system in Python
  35. Convolutional neural network processing sequence for Python deep learning
  36. Python data structure and algorithm (1) -- enum type enum
  37. 超全大厂算法岗百问百答(推荐系统/机器学习/深度学习/C++/Spark/python)
  38. 【Python进阶】你真的明白NumPy中的ndarray吗?
  39. All questions and answers for algorithm posts of super large factories (recommended system / machine learning / deep learning / C + + / spark / Python)
  40. [advanced Python] do you really understand ndarray in numpy?
  41. 【Python进阶】Python进阶专栏栏主自述:不忘初心,砥砺前行
  42. [advanced Python] Python advanced column main readme: never forget the original intention and forge ahead
  43. python垃圾回收和缓存管理
  44. java调用Python程序
  45. java调用Python程序
  46. Python常用函数有哪些?Python基础入门课程
  47. Python garbage collection and cache management
  48. Java calling Python program
  49. Java calling Python program
  50. What functions are commonly used in Python? Introduction to Python Basics
  51. Python basic knowledge
  52. Anaconda5.2 安装 Python 库(MySQLdb)的方法
  53. Python实现对脑电数据情绪分析
  54. Anaconda 5.2 method of installing Python Library (mysqldb)
  55. Python implements emotion analysis of EEG data
  56. Master some advanced usage of Python in 30 seconds, which makes others envy it
  57. python爬取百度图片并对图片做一系列处理
  58. Python crawls Baidu pictures and does a series of processing on them
  59. python链接mysql数据库
  60. Python link MySQL database