Click on Machine learning algorithm and Python Study , Choose star
Wonderful content doesn't get lost
This paper first introduces the basic knowledge of virtual environment and how to use it , Then I will introduce the working principle behind the virtual environment .（ Environmental Science ： stay macOS Mojave Use the latest version of Python 3.7.x）
1. Why use virtual environments ？
Virtual environments provide simple solutions to a range of potential problems , Especially in the following aspects ：
It sounds convenient , isn't it? ？ When you start building more complex projects and working with others , The importance of virtual environment will be highlighted . Many data scientists also need to be familiar with the multilingual aspects of virtual environments Conda Environmental Science .
You can use it in order ！
2. What is a virtual environment ？
What is virtual environment ？
Virtual environments are used for dependency management and project isolation Python Tools , allow Python Site package （ Third party Library ） Installed in the isolated directory of a local specific project , Instead of global installation （ That is, as system wide Python Part of ）.
That sounds good , But what exactly is a virtual environment ？ A virtual environment is just a directory with three important components ：
The last point is that unexpected mistakes can occur , I'll talk about that later , But let's take a look at how to actually use virtual environments in practice .
3. Using virtual environments
Create a virtual environment
Suppose you want to create a project called test-project/ Virtual environment for , The project has the following tree ：
├── data ├── deliver # Final analysis, code, & presentations ├── develop # Notebooks for exploratory analysis ├── src # Scripts & local project modules └── tests
You need to perform venv modular , It is Python Part of the standard library .
% cd test-project/ % python3 -m venv venv/ # Creates an environment called venv/
Be careful ： You can replace... With a different environment name “venv/”.
look ！ Virtual environment was born . Now the project becomes ：
test-project/ ├── data ├── deliver ├── develop ├── src ├── tests └── venv # There it is!
remind ： The virtual environment itself is a directory .
The only thing to do is to run the script mentioned earlier “ Activate ” Environmental Science .
% source venv/bin/activate (venv) % # Fancy new command prompt
Now we're in the active virtual environment （ Indicated by a command prompt , Prefix with the name of the active environment ）.
We will deal with the project as usual , Make sure that the project is completely isolated from the rest of the system . In a virtual environment , We can't access system wide site packages , And cannot access the installation package outside of the virtual environment .
On completion of project work , You can exit the environment with the following code ：
(venv) % deactivate % # Old familiar command prompt
By default , Install only in new environments pip and setuptools.
(venv) % pip list # Inside an active environmentPackage Version ---------- ------- pip 19.1.1 setuptools 40.8.0
If you want to install a specific version of the third-party library , such as numpyv1.15.3, Can be used as usual pip.
(venv) % pip install numpy==1.15.3 (venv) % pip listPackage Version ---------- ------- numpy 1.15.3 pip 19.1.1 setuptools 40.8.0
Now available in script or active Python shell Import numpy. for example , Suppose the project contains the following lines of script tests / imports-test.py.
#!/usr/bin/env python3 import numpy as np
When you run this script directly from the command line , available ：
(venv) % tests/imports-test.py (venv) % # Look, Ma, no errors!
success . Script import numpy No fault .
4. management environment
The easiest way to make our work available for reuse is at the root of the project （ Top level directory ） Add a requirement document to . So , Need to run pip freeze, The installed third-party packages and their version numbers are listed below ：
(venv) % pip freeze numpy==1.15.3
And write the output to a file , We call it requirements.txt.
(venv) % pip freeze > requirements.txt
When updating a package or installing a new package , You can rewrite the requirements file with the same command .
Now? , Anyone who shares a project can use requirements.txt file , Run the project on the system by copying the environment .
Copy the environment
wait —— How did it work ？
Imagine , Our teammates Sara From the team's GitHub Test project deleted from Repository . On her system , The directory tree for the project is shown below ：
test-project/ ├── data ├── deliver ├── develop ├── requirements.txt ├── src └── tests
Notice something unusual ？ Yes , you 're right ！ No, venv / Folder .
We've taken it from the team's GitHub Delete from repository , Because its existence may cause trouble . This is the use of requirements.txt One reason why files are critical to copying project code .
To run the test project on the machine ,Sara All you need to do is create a virtual environment in the root of the project ：
Sara% cd test-project/ Sara% python3 -m venv venv/
And use pip install -r requirements.txt Install the project's dependencies in an active virtual environment .
Sara% source venv/bin/activate (venv) Sara% pip install -r requirements.txt Collecting numpy==1.15.3 (from -r i (line 1)) Installing collected packages: numpy Successfully installed numpy-1.15.3
Now? ,Sara The project environment on the system is exactly the same as ours . It's neat , isn't it? ？
Unfortunately things don't always go as planned , There are always problems . Maybe by mistake updating a specific site package and finding myself in Dependency Hell The Ninth level of , Cannot run single line project code . Maybe it's not that bad , Maybe you will find yourself in the seventh level .
No matter how much you find yourself , The easiest way to solve the problem and see hope again is to recreate the virtual environment of the project .
% rm -r venv/ # Nukes the old environment % python3 -m venv venv/ # Makes a blank new one % pip install -r requirements.txt # Re-installs dependencies
Be accomplished , Thanks a lot requirements.txt file , It's back to normal again . However, another reason is that the requirements document should always be included in the project .
5. How virtual environments do this ？
Want to learn more about virtual environments ？ such as , How to use the right environment Python Explain the program and find the right third-party library ？
echo $ PATH
It all boils down to PATH The value of , It tells shell What to use Python Examples and where to find website packages . On the basis of shell in ,PATH It seems more or less like this .
% echo $PATH /usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin
call Python Interpreter or run .py Script time ,shell Will search in order PATH The contents listed in , Until I met Python example . To see PATH First found Python example , Please run which python3.
% which python3 /usr/local/bin/python3 # Your output may differ
Through the site module （ This is a Python Part of the standard library ） Find this Python It's also easy for instance to find the location of the site package .
% python3 # Activates a Python shell >>> import site >>> site.getsitepackages() # Points to site-packages folder[ /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages ]
Run script venv / bin / activate modify PATH, In order to shell Search the project's local binaries before searching the system's global binaries .
% cd ~/test-project/ % source venv/bin/activate (ven) % echo $PATH~/test-project/venv/bin:/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin
Now? shell Know how to use the project's local Python example ：
(venv) % which python3 ~/test-project/venv/bin/python3
Where can I find the local site package for the project ？
(venv) % python3 >>> import site >>> site.getsitepackages()[ ~/test-project/venv/lib/python3.7/site-packages ] # Ka-ching
Remember the old tests / imports-test.py Script ？ It looks like the following ：
import numpy as np
We can run this script in an active environment , There's no problem , It's because of the environment Python Instances can access the project's local site package .
What happens if you run the same script from outside the project's virtual environment ？
% tests/imports-test.py # Look, no active environmentTraceback (most recent call last): File "tests/imports-test.py", line 3, in <module> import numpy as npModuleNotFoundError: No module named numpy
Yes , An error occurred , But we should do this . If we don't , That means we can access the project's local site package from outside the project , And thus undermines the whole purpose of having a virtual environment . The fact that something went wrong proved that our project was completely isolated from the rest of the system .
The directory tree of the environment
One thing can help you organize all this information , That is, to understand the appearance of the environment tree clearly .
test-project/venv/ # Our environment s root directory ├── bin │ ├── activate # Scripts to activate │ ├── activate.csh # our project s │ ├── activate.fish # virtual environment. │ ├── easy_install │ ├── easy_install-3.7 │ ├── pip │ ├── pip3 │ ├── pip3.7 │ ├── python -> /usr/local/bin/python # Symlinks to system-wide │ └── python3 -> python3.7 # Python instances. ├── include ├── lib │ └── python3.7 │ └── site-packages # Stores local site packages └── pyvenv.cfg
This article is from WeChat official account. - Machine learning algorithm and Python Study （MLPython）
The source and reprint of the original text are detailed in the text , If there is any infringement , Please contact the firstname.lastname@example.org Delete .
Original publication time ： 2020-11-11
Participation of this paper Tencent cloud media sharing plan , You are welcome to join us , share .