Four Python project management build tools

Chen Chen 135 2022-06-23 17:50:48 阅读数:1,008

pythonprojectmanagementbuildtools

Python For so long, there has not been a de facto standard project management and construction tool , So as to cause Python The structure and construction methods of the project are diverse . This may reflect Python Free will .

Unlike Java After the initial manual construction , To semi automated Ant, Until then Maven It's basically the de facto standard . meanwhile Maven Also accepted other Gradle(Android Project main push ), SBT( Mainly Scala project ), Ant+Ivy, Buildr And so on , But it's hard to shake Maven In the Jianghu , And the others almost follow Maven Table of contents layout .

go back to Python, Produced pip, pipenv, conda Package management tools like that , However, there is no agreement on the directory layout of the project .

A lot about building continues the tradition Makefile The way , Plus setup.py and build.py Install and build with program code . About project catalog layout , There are project templates , Then make a tool to apply the project template .

Let's take a look at the use of four tools

  1. CookieCutter
  2. PyScaffold
  3. PyBuilder
  4. Poetry

 CookieCutter A classic Python Project directory structure

$ pip install cookiecutter
$ cookiecutter gh:audreyr/cookiecutter-pypackage   
#  With  github  Upper  audreyr/cookiecutter-pypackage  As a template , Answer a bunch of questions to generate a  Python  project
......
project_name [Python Boilerplate]: sample
......

Finally by cookiecutter The generated project template looks like the following :

$ tree sample
sample
├── AUTHORS.rst
├── CONTRIBUTING.rst
├── HISTORY.rst
├── LICENSE
├── MANIFEST.in
├── Makefile
├── README.rst
├── docs
│   ├── Makefile
│   ├── authors.rst
│   ├── conf.py
│   ├── contributing.rst
│   ├── history.rst
│   ├── index.rst
│   ├── installation.rst
│   ├── make.bat
│   ├── readme.rst
│   └── usage.rst
├── requirements_dev.txt
├── sample
│   ├── __init__.py
│   ├── cli.py
│   └── sample.py
├── setup.cfg
├── setup.py
├── tests
│   ├── __init__.py
│   └── test_sample.py
└── tox.ini
3 directories, 26 files

This is probably the main framework of the popular directory structure , The main elements are :

$ tree sample
sample
├── Makefile
├── README.rst
├── docs
│   └── index.rst
├── requirements.txt
├── sample
│   ├── __init__.py
│   └── sample.py
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    └── test_sample.py

project sample Duplicate in directory sample Put... In the directory Python Source file ,tests  In the directory is the test file , Add one  docs  Put the directory into the document ,README.rst, Others used to build setup, setup.cfg and Makefile file .

This is actually a very classic Python Project structure , The next build uses  make  The command , Input  make  You will see the definition in Makefile Instruction in file

$ make
clean                remove all build, test, coverage and Python artifacts
clean-build          remove build artifacts
clean-pyc            remove Python file artifacts
clean-test           remove test and coverage artifacts
lint                 check style
test                 run tests quickly with the default Python
test-all             run tests on every Python version with tox
coverage             check code coverage quickly with the default Python
docs                 generate Sphinx HTML documentation, including API docs
servedocs            compile the docs watching for changes
release              package and upload a release
dist                 builds source and wheel package
install              install the package to the active Python's site-packages

To use the above build process , The corresponding package needs to be installed , Such as  toxwheelcoveragesphinxflake8, They all pass through  pip  To install . Then you can  make testmake coveragemake docs,make dist  etc. . among  make docs  Can generate a very beautiful Web file .

 PyScaffold Create a project

PyScaffold seeing the name of a thing one thinks of its function , It is used to create Python Tools for project scaffolding , Installation and use :

$ pip install pyscaffold
$ putup sample

This creates a Python project , The directory structure is the same as before  cookiecutter The selected template is similar , It just puts the source file in  src  Catalog , Instead of  sample  Catalog .

$ tree sample
sample
├── AUTHORS.rst
├── CHANGELOG.rst
├── CONTRIBUTING.rst
├── LICENSE.txt
├── README.rst
├── docs
│   ├── Makefile
│   ├── _static
│   ├── authors.rst
│   ├── changelog.rst
│   ├── conf.py
│   ├── contributing.rst
│   ├── index.rst
│   ├── license.rst
│   ├── readme.rst
│   └── requirements.txt
├── pyproject.toml
├── setup.cfg
├── setup.py
├── src
│   └── sample
│       ├── __init__.py
│       └── skeleton.py
├── tests
│   ├── conftest.py
│   └── test_skeleton.py
└── tox.ini

The construction of the whole project will use  tox  This tool .tox  Is an automated test and build tool , It can be created during the build process Python A virtual environment , This allows a clean environment for testing and building .

tox -av  Can show the definition in  tox.ini  All the tasks in :

$ tox -av
default environments:
default   -> Invoke pytest to run automated tests
additional environments:
build     -> Build the package in isolation according to PEP517, see https://github.com/pypa/build
clean     -> Remove old distribution files and temporary build artifacts (./build and ./dist)
docs      -> Invoke sphinx-build to build the docs
doctests  -> Invoke sphinx-build to run doctests
linkcheck -> Check for broken links in the documentation
publish   -> Publish the package you have been developing to a package index server. By default, it uses testpypi. If you really want to publish your package to be publicly accessible in PyPI, use the `-- --repository pypi` option.

To execute which command, use  tox -e buildtox -e docs  etc.

In my experience tox In the course of the order , Every step seems to be slow , It should take some time to create a virtual machine .

 PyBuilder

It's best to look at another build tool PyBuilder, The directory structure it creates is very close to Maven, Let's take a look

$ pip install pybuilder
$ mkdir sample && cd sample    #  The project directory needs to be created manually
$ pyb --start-project          #  After answering some questions, create the required directories and files 

After that, look at its directory structure :

$ tree sample
.
├── build.py
├── docs
├── pyproject.toml
├── setup.py
└── src
    ├── main
    │   ├── python
    │   └── scripts
    └── unittest
        └── python

The build process is still done with  pyb  command , You can use  pyb -h  view help ,pyb -t  List all tasks , PyBuilder The task of is added as a plug-in , The plug-in is configured in  build.py  In file .

$ pyb -t sample
Tasks found for project "sample":
                  analyze -  Execute analysis plugins.
                            depends on tasks: prepare run_unit_tests
                    clean - Cleans the generated output.
          compile_sources - Compiles source files that need compilation.
                            depends on tasks: prepare
                 coverage - <no description available>
                            depends on tasks: verify
                  install - Installs the published project.
                            depends on tasks: package publish(optional)
                  package - Packages the application. Package a python application.
                            depends on tasks: compile_sources run_unit_tests(optional)
                  prepare - Prepares the project for building. Creates target VEnvs
        print_module_path - Print the module path.
       print_scripts_path - Print the script path.
                  publish - Publishes the project.
                            depends on tasks: package verify(optional) coverage(optional)
    run_integration_tests - Runs integration tests on the packaged application.
                            depends on tasks: package
           run_unit_tests - Runs all unit tests. Runs unit tests based on Python's unittest module
                            depends on tasks: compile_sources
                   upload - Upload a project to PyPi.
                   verify - Verifies the project and possibly integration tests.
                            depends on tasks: run_integration_tests(optional)
$ pyb run_unit_tests sample

PyBuilder Also create a virtual environment before building or testing , from 0.12.9 The version can be started through the parameter  --no-venvs  Skip the step of creating a virtual environment . Used  --no-venvs  Words Python The code will be running  pyb  The current Python Execution in the environment , The required dependencies will be installed manually .

Project dependencies should also be defined in  build.py  In file

@init
def set_properties(project):
    project.depends_on('boto3', '>=1.18.52')
    project.build_depends_on('mock')

Then in execution  pyb  The above dependencies will be installed when creating a virtual environment , And run the test and build .

 Poetry

the last one Poetry, It feels like a more mature , Project activity is also higher Python structure , It has more powerful trust management function , use  poetry add boto3  You can add dependencies ,poetry show --tree  Show dependency tree . See how to install and create a project

$ pip install poetry
$ poetry new sample

It creates projects that are simpler than the above

$ tree sample
sample
├── README.rst
├── pyproject.toml
├── sample
│   └── __init__.py
└── tests
    ├── __init__.py
    └── test_sample.py

If  poetry new  close  --src  Parameters , Then the source file directory  sample  It will be placed in  src  Under the table of contents , namely  sample/src/sample. poetry init  Will generate... In the current directory  pyproject.toml  file , The generation of directories, etc. needs to be completed manually .

It doesn't focus on document generation , Check of code specification , No code coverage . Its project configuration is more centralized , All in  pyproject.toml  In file ,toml  What is it? ? It is a configuration file format Tom's Obvious, Minimal Language (https://github.com/toml-lang/toml).

pyproject.toml  Some similar NodeJS Of  package.json  file , such as poetry add, poetry install Command line

#  Go to  pyproject.toml  Add a pair of   boto3  And install (add  Also from local or  git  To install dependencies  ),
poetry add boto3    
 #  Will be in accordance with  pyproject.toml  The file defines the corresponding dependencies of the installation to the current  Python  In a virtual environment
 #  For example  <test-venv>/lib/python3.9/site-packages  Directory , After the module is installed, the test case can also be used
poetry install       

Other major

1.  poetry build    #  Build installable  *.whl  and  tar.gz  file
2.  poetry shell    #  Will be defined in  pyproject.toml  Create and use virtual environments based on dependencies in files
3.  poetry run pytest    #  Run using  pytest  Test cases for , Such as  tests/test_sample.py
4.  poetry run python -m unittest tests/sample_tests.py  #  function  unittest  The test case
5.  poetry export --without-hashes --output requirements.txt  #  export  requirements.txt  file , --dev   Export contains  dev  Dependence , Or use  poetry export --without-hashes > requirements.txt

poetry run  Can execute any system command , But it will execute in the virtual environment it wants . So you can imagine meeting ,poetry  To generate documents or coverage for your project, you must use  poetry run ...  Command to support  sphinxcoverage  or  flake8.

stay sample Catalog ( And pyproject.toml Document level ) Create file in  my_module.py, The content is

def main():
    print('hello poetry')

And then in  pyproject.toml  Write in

[tool.poetry.scripts]
my-script="sample.my_module:main"

Re execution

$ poetry run my-script

Will be output "hello poetry".

Through the understanding of the above four tools , The complexity of the project structure is determined by cookiecutter-pyproject -> PyScaffold -> PyBuilder -> Poetry Lower in turn , The difficulty of using is roughly the same order .

版权声明:本文为[Chen Chen 135]所创,转载请带上原文链接,感谢。 https://pythonmana.com/2022/01/202201051711022101.html