Using pip to Manage Python Packages
Like many useful programming ecosystems, Python provides a powerful and easy-to-use package management system called pip. It is written to replace an older tool called easy_install. From a high-level point of view, pip has the following advantages over easy_install:
- All packages are downloaded before installation to prevent partial (thus broken) installation.
- Output information is pre-processed, so it’s more useful than eccentric messages.
- It keeps record of why actions are performed. For example, the reason why a package was required is recorded for future reference.
- Packages can be installed as flat modules, which makes library-code debugging much easier than egg archives.
- Native support for other version control systems. For example, you could install a Python package directly from a GitHub repository if it’s setup properly.
- Packages can be uninstalled. This is a huge advantage over easy_install, which requires the programmer to manually uninstall packages.
- It’s simple to define sets of requirements, thus easy to replicate a set of packages across different environments.
Setting up virtualenv and pip to perform simple operations
virtualenv is a tool for sand-boxing Python environments. Instead of modifying the global Python environment, virtualenv allows a programmer to setup independent Python contexts that are similar to separate sandboxes. One advantage of sandboxing your Python environment is that you can effortlessly test your code under different Python versions and package dependencies without switching between virtual machines.
- How to Install SQLAlchemy
- How to Install Django on Windows, Mac and Linux
- Writing Your First Python Django Application
To install and set up pip and virtualenv, run the following commands:
# easy_install is the default package manager in CPython % easy_install pip # Install virtualenv using pip % pip install virtualenv % virtualenv python-workspace # create a virtual environment under the folder 'python-workspace'
If your current Python environment does not contain any package manager, you could download one Python file from here and run it:
# Create a virtual environment under the folder 'python-workspace' % python virtualenv.py python-workspace
Once you have set up the new virtualenv under ‘python-workspace’, you need to activate it so that the Python environment in the current shell will transition into the virtualenv:
% cd python-workspace % source ./bin/activate # activate the virtualenv 'python-workspace' % python # enter an interpreter shell by executing the current virtual python program under 'python-workspace/bin/python'
If you take a look at the content of the folder ‘python-workspace/bin’, you will see a list of executable programs such as ‘python’, ‘pip’ and ‘easy_install’, etc. Instead of executing programs under the system-default Python environments, the virtualenv’s Python programs are being executed.
Adding/installing Packages with pip
To install a package, use the ‘install’ command.
% pip install sqlalchemy # install the package 'sqlalchemy' and its dependencies
Sometimes, you may want to inspect the source code of a package before installing it. For example, you may want to inspect the source code of a newer version of a package before installing it to make sure it will work with your current code.
% pip install --download sqlalchemy_download sqlalchemy # download the package 'sqlalchemy' archives into 'sqlalchemy_download' instead of installing it % pip install --no-install sqlalchemy # unpack the downloaded package archives into 'python-workspace/build' for inspection % pip install --no-download sqlalchemy # install the unpacked package archives
If you want to install the bleeding-edge version of a package, you can install it directly from its Git or Subversion repository:
% p% pip install git+https://github.com/simplejson/simplejson.git % pip install svn+svn://svn.zope.org/repos/main/zope.interface/trunk
Upgrading a Package with pip
For installed packages, you can upgrade them by:
% pip install --upgrade sqlalchemy # upgrade sqlalchemy if there’s a newer version available. Notice that --upgrade will recursively upgrade sqlalchemy and all of its dependencies.
If upgrading a package along with its dependencies is not desirable (sometimes you may want to test the package’s backward compatibility), you can perform a non-recursive upgrade by:
% pip install --upgrade --no-deps sqlalchemy # only upgrade sqlalchemy but leave its dependencies alone
Saving your pip package list
So far, you should have installed a bunch of packages using pip. In order to test the installed packages under a different environment, you can save or “freeze” the installed package list into a requirements file and re-install all the packages using the requirements file in another environment:
% pip freeze > my-awesome-env-req.txt # create a requirement file that contains a list of all installed packages in the current virtualenv 'python-workspace' % virtualenv ../another-python-workspace # create a new virtualenv 'another-python-workspace' % cd ../another-python-workspace % source ./bin/activate # activate the new empty virtualenv 'another-python-workspace' % pip install -r ../python-workspace/my-awesome-env-req.txt # install all packages specified in 'my-awesome-env-req.txt'
Removing/Uninstalling pip Packages
If somehow, you decide certain packages are not good for your project anymore and you want to remove them to clean up the virtualenv. You could remove a package by simply typing:
% pip uninstall sqlalchemy # uninstall the package 'sqlalchemy'
Or to remove a list of packages by:
% pip uninstall -r my-awesome-env-req.txt # uninstall all packages specified in 'my-awesome-env-req.txt'
Notice that pip does not know how to uninstall two types of packages:
- Packages installed with pure distutils: ‘
python setup.py install‘ - Packages installed with script wrappers: ‘
python setup.py develop‘
Because these two types of installed packages do not contain any metadata, pip does not know which files should be removed to uninstall them.
Searching for pip Packages
If you want to search for packages that solve specific types of problems, you can perform a search by:
% pip search database # search package titles and descriptions that contain the word 'database'
Searching packages is very useful to retrieve an overview over all the packages in a problem domain so that you can compare and choose the packages that are most suitable for what you need to do.
Tips on Using pip
1. To prevent accidentally running pip to install unwanted packages into the global environment, you can tell pip to run only if a virtualenv is currently active by setting an shell environment variable:
% export PIP_REQUIRE_VIRTUALENV=true
2. Usually, packages will be installed under the ‘site-packages’ directory. However if you want to make changes and debug a package, it makes sense to run the package directly from its source tree. You can put the package into “Edit mode” by telling pip to install it with the “-e” option/argument, as follows:
# create a .pth file for sqlalchemy in 'site-packages' instead of installing it into 'site-packages' # so that you can make changes to the package and debug the changes immediately % pip install -e path/to/sqlalchemy
Package Index, and More Info on pip
Python has a website, containing many useful packages you might want to checkout, known as the “Python Package Index”, or “PyPI”. This is a great repository of many commonly used Python libraries/modules, which are all pre-packaged and easily installable.
For more information on Python’s pip – checkout this website.