Python Is one of the most commonly used programming languages among developers , But it has some limitations . for example , For some applications , It may run slower than other languages 100 times . That's why Python After the speed of running has become a bottleneck for users , Many companies rewrite their applications in another language . But is there a way to keep Python It can improve the speed ？ It is PyPy.
PyPy It's a very compatible Python Interpreter , It is CPython2.7、3.6 And the upcoming 3.7 A worthy alternative to . Use it when you install and run an application , Can significantly increase speed . How much faster depends on the application you're running .
In this tutorial , You will learn ：
- How to use PyPy Install and run code
- PyPy And CPython The comparison in terms of speed
- PyPy The function of and how to make Python Code runs faster
- The examples in this tutorial use Python 3.6 , Because it is PyPy Compatible with the latest Python edition .
Python An interpreter can be implemented in many languages , Such as CPython（ use C To write ）、Jython（ use Java To write ）、Iron Python（ use .NET To write ） and PyPy（ use Python To write ）.
CPython yes Python The initial implementation of the interpreter , It's also the most widely used and most maintained . When we are from Python Download and install the official website Python 3.x after , We've got an official version of the interpreter ：CPython. This interpreter uses C Language development , So called CPython. Run on the command line python That is to start. CPython Interpreter .
however , because CPython It's a high-level interpretation language , So it has some limitations , And there's no advantage in terms of speed . This is it. PyPy Where it works . Because it conforms to Python language norm , therefore Py Py There is no need to make any changes to the code base , And you can significantly increase the speed through the following functions .
Now? , You may want to know , If CPython Use the same syntax , Why doesn't it come true Py Py The power of . as a result of , Implementing these features requires significant changes to the source code , It's going to be a lot of work .
Let's take a quick look at how to use PyPy.
Many people study python, I don't know where to start .
Many people study python, After mastering the basic grammar , I don't know where to look for cases to start .
A lot of people who have done cases , But I don't know how to learn more advanced knowledge .
So for these three kinds of people , I will provide you with a good learning platform , Get a free video tutorial , electronic text , And the source code of the course ！
QQ Group ：1156465813
Your operating system may already offer PyPy software package . for example , stay Mac OS On , You can go to Homebrew Install it with the help of ：
brew install pypy3
Or you can download binaries that match the operating system . After downloading , Just open tarball or ZIP File can . then , You can do the following ：
$ tar xf pypy3.6-v7.3.1-osx64.tar.bz2 $ ./pypy3.6-v7.3.1-osx64/bin/pypy3 Python 3.6.9 (?, Jul 19 2020, 21:37:06) [PyPy 7.3.1 with GCC 4.2.1] Type "help", "copyright", "credits" or "license" for more information.
You need to execute the command at the above folder address . For a complete explanation , Please refer to the installation documentation . function PyPy You have now installed Py Py, And it's about to run ！ So , Please create one called script.py Of Python file , And put the following code into it ：
total = 0
for i in range(1, 10000):
for j in range(1, 10000):
total += i + j
print(f"The result is {total}")
In two nested for In circulation , take 1 To 9,999 Add the numbers between , And print the results . See how long it takes to run this script ：
import time
start_time = time.time()
total = 0
for i in range(1, 10000):
for j in range(1, 10000):
total += i + j
print(f"The result is {total}")
end_time = time.time()
print(f"It took {end_time-start_time:.2f} seconds to compute")
The code now does the following ：
- The first 3 Save the current time to the variable
start_time. - The first 5 to 8 Line run cycle .
- The first 10 Line print results .
- The first 12 Save the current time as
end_time. - The first 13 The difference between the start time and the end time of line printing , To show the time required to run the script .
use Python To run it . Here I am Mac Book Pro The result on ：
$ python3.6 script.py The result is 999800010000 It took 20.66 seconds to compute
Now use Py Py Run it :
$ pypy3 script.py The result is 999800010000 It took 0.22 seconds to compute
In this little experiment ,PyPy The speed is about Python Of 94 times ！
You can browse PyPy Speed Center To see more rigorous testing .
please remember ,PyPy How you affect the performance of your code depends on what you do with it . In some cases ,Py Py Actually slower , I'll see later . however , In terms of geometric average , Its speed is Python Of 4.3 times .
Py Py There are two definitions ：
1、 A dynamic language framework for generating dynamic language interpreters 2、 Using the framework Python Realization
You should be aware of the second problem . You used Python Implementation is using what is called RPython Dynamic language framework written by , It's like CPython Yes, it is C Compiling , and Jython Yes, it is Java It's written the same way .
But I didn't mention PyPy Yes, it is Python Written by ？ Um. , It's a little simple .PyPy Become useful Python Compiling Python Interpreter （ instead of RPython） The reason for that is RPython Used with Python The same grammar .
PyPy How did you get it ？ The following points need to be explained ：
1、 Its source code is to use RPython To write .
2、RPython Conversion tools are applied to the code , Fundamentally improve code efficiency , You can also compile code into machine code , This is it. Mac,Windows and Linux Why users have to download different versions .
3、 Binary executable file generated in the above way , That's what you run Python Interpreter .
You don't need to perform all of these steps to use PyPy. Because there are already executable files for you to install and use .
Besides , Because it's confusing to use the same word in both the framework and the implementation ,PyPy The team behind it decided to give up the double use . Now? ,PyPy Only Python Interpreter , And the framework is called RPython Transformation tool .
Next , You will find out under what circumstances PyPy Than Python Better 、 faster .
Just-In-Time (JIT) compiler
In understanding JIT Before the contents of the compiler , Let's review the compiled language first （ Such as C） And explanatory language （ Such as JavaScript） Characteristics of .
Before a program written in a compiled language is executed , A special compilation process is needed , Compile the source code into a machine language file , Such as exe File format , When you want to run again later , You can use the compilation results directly , Such as direct operation exe file . Because it only needs to be compiled once , You don't need to compile later , So compiler language is efficient . Platform specific , Generally, it can't be ported to other platforms . Such as C、C++、Objective All of them belong to compiler language .
Interpretive languages don't need to be compiled in advance , It directly interprets the source code into machine code and executes it immediately , So as long as a platform provides the corresponding interpreter can run the program . Every time an interpreted language runs, it needs to interpret the source code as machine code and execute it , Low efficiency ; As long as the platform provides the corresponding interpreter , You can run the source code , So it is easy to transplant the source program .
And then there are programming languages , for example Python, It's a mix of compilation and interpretation . say concretely ,Python First it's compiled into bytecode , Then from CPython explain . This makes code perform better than code written in a pure interpreted language , And remain portable Sexual advantage .
But its performance is still far lower than compiled languages . The reason is , The compiled code can perform many optimizations , And bytecode is impossible .
This is it. JIT The source of the compiler . It tries to gain both advantages by compiling and interpreting machine code . In short , Here are JIT Steps taken to compile to provide faster performance ：
1、 Identify the most commonly used components in code , Like functions in loops .
2、 The runtime converts these parts into machine code .
3、 Optimize the generated machine code .
4、 Replace the previous implementation with an optimized version of machine code .
Remember the two nested loops at the beginning of the tutorial ？PyPy When duplicate operations are detected , Compile it into machine code , Optimize machine code , And then transform to implement . That's why you see this result .
Garbage collection mechanism
Whenever you create a variable 、 Function or any other object , Your computer will allocate memory to them . Final , Some of these objects will no longer need . If not cleaned up in time , Computers can run out of memory and crash programs .
stay C and C++ In programming languages , Usually you have to deal with this manually . Other programming languages （ Such as Python and Java） This will be done automatically for you . This is called the automatic garbage collection mechanism .
CPython Using a technique called reference counting . In essence , Whenever an object is referenced ,Python Object's reference count will increase , When the object is dereferenced, the count is decremented . When the reference count is zero ,CPython The memory release function is automatically called for the object . It's a simple and effective technique , But there is a trap .
When the reference count of a large object tree becomes zero , All related objects will be released . therefore , You may have a long pause , In the meantime, your program can't be executed at all .
Besides , Another example , Reference counting doesn't work at all . As shown below ：
class A(object): pass a = A() a.some_property = a del a
In the code above , Defines a new class , then , Create an instance , And specify it as its own property . Last , Delete the instance .
here , Instances will no longer be accessible . however , Reference counting does not delete instances from memory , Because it has a reference to itself , So the reference count is not zero . This problem is called a reference loop , Cannot use reference counting to resolve .
This is a CPython Another tool used , It's called a recycling garbage collector . It comes from known roots （ Such as type objects ） Start traversing all objects in memory . then , It identifies all accessible objects , And release inaccessible objects , Because they no longer exist . This solves the reference loop problem . however , When there are a large number of objects in memory , It may create more obvious pauses .
On the other hand ,PyPy Don't use reference counting . contrary , It only uses the second Technology , The loop finder . in other words , It periodically traverses the active object from the root . This makes PyPy Than CPython It has some advantages , Because it doesn't need to consider reference counting , Thus, the total memory management time is less than CPython.
Besides ,PyPy Split the work into a variable number of parts , And run each part , Until there's no rest . This method only adds a few milliseconds after each secondary collection , Not like CPython Add hundreds of milliseconds at a time .
The garbage collection mechanism is very complicated , And there's a lot beyond the scope of this tutorial . You can find information about PyPy Details of the garbage collection mechanism .
PyPy It's not everything , It's not a tool for all your tasks . It may even make the application run faster than CPython Much slower . That's why you have to remember the following limitations .
It doesn't apply to C Expand
PyPy Perfect for pure Python Applications . Whenever used C Extension module , It runs faster than in CPython It's much slower . as a result of PyPy Can't optimize C Extension module , Because they are not fully supported . Besides ,PyPy Must simulate reference counting in code , Make it slower .
under these circumstances ,PyPy The team suggests removing CPython Expand and replace it with pure Python edition . If not , Must be used CPython.
For all that , The core team is working on C Expand . Some packages have been ported to PyPy, And it works just as fast .
It only works with long running programs
Imagine you want to go to a store close to your home . You can walk directly to , You can also drive .
Your car is obviously much faster than your feet . however , Please consider the steps you need to complete ：
1. Go to your garage .
2、 Driving a car .
3、 Warm up the car .
4、 Drive to the store .
5、 Looking for parking space .
6、 Repeat the process on the way back .
Driving requires a series of troublesome steps , If the place you want to go is nearby , That's not necessarily worth it .
Now think about it. , If you want to go 50 Kilometers away from the neighboring city , What's going to happen ？ It must be worth driving there , Instead of walking to .
Although the contrast in speed is not as obvious as the analogy above , but PyPy and CPython The same is true of this .
When using PyPy When running scripts , It does a lot of things to make the code run faster . If the script itself is simple , The actual script will run faster than CPython.
On the other hand , If you have a long running script , That could lead to significant performance improvements .
If you want to feel it for yourself , Please be there. CPython and PyPy Run the following script in ：
import time
start_time = time.time()
for i in range(100):
print(i)
end_time = time.time()
print(f"It took {end_time-start_time:.10f} seconds to compute")
When you use PyPy When it runs , There will be a small delay at the beginning , and CPython It will run immediately . stay Mac Book Pro Run it on , use CPython need 0.0004873276 second , use PyPy need 0.0019447803 second .
It doesn't compile ahead of time
As you can see at the beginning of this tutorial ,PyPy It's not a fully compiled Python Realization . It compiles Python Code , But it's not Python The compiler of the code . because Python Some inherent characteristics , It's impossible to put Python Compile it into a separate binary and reuse it .
Py Py Faster than fully explanatory language , But more than a fully compiled language （ Such as C） slow .
PyPy yes CPython A fast and powerful alternative to . Use it to run scripts , You can greatly increase speed without changing the code . But it's not everything , There are some limitations .
In this tutorial , You learned ：
-
PyPy What is it? ？
-
How to install PyPy And use it to run scripts
-
PyPy And CPython The comparison in terms of speed
-
PyPy How to improve the program speed
-
In which cases PyPy There will be limitations
If your Python Scripts need to be a little bit faster , Welcome to try PyPy！
Be careful ： If you're looking for python Well paid jobs . I suggest you write more about real enterprise projects and accumulate experience . Or you won't find a job , Of course, a lot of people have never been in a business , How can there be project experience ？ So you have to find more enterprise projects and practice more . If you're lazy and don't want to find , You can also enter my Python Circle of communication ：1156465813. There are some real enterprise project cases that I have written before in the group file . You can take it to study , If you don't understand, you can find me in your skirt , I'll answer you patiently when I have time .
The following is useless , For this blog to be crawled by search engines
(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)
python What is it Zero basis to learn python How long will it take? python Why is it called a reptile
python Novice reptile tutorial python Crawler universal code python How do reptiles make money
python Basic course Web crawler python python Classic examples of reptiles
python Reptiles
(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)(*￣︶￣)
The above is useless , For this blog to be crawled by search engines