Python’s accessible and vast library of user-made packages is arguably one of its most powerful features. When developing a project with Python, it is often not long before you find your project dependent on a large list of libraries, and specific versions of those libraries, at that. Communicating such a long list of libraries to potential users of your own code base (or even a Python package of your own) thus could have been an arduous process, were it not for requirements.txt. By generating a
requirements.txt file and distributing it with your project, you can make it far, far easier for other users or developers to install the correct versions of the packages required for your code to execute properly. In this guide, I’ll break down how to generate a
requirements.txt and share two particularly useful scripts to automate working with them.
To generate a
requirements.txt file for your project, open your project’s directory in the terminal. Ensure that you have installed the currently-required packages in your current Python environment (virtual environment, system-wide installation, or otherwise) using pip. Then, simply execute the following command:
pip freeze > requirements.txt
Here, you could replace
requirements.txt with the path to the file you want to generate, but conventionally the generated file is named
requirements.txt and placed within the root directory of the project. If you open the file, you should find something in this format:
cycler==0.10.0 # via matplotlib kiwisolver==1.2.0 # via matplotlib matplotlib==3.2.1 numpy==1.18.5 pandas==1.0.4 pyparsing==2.4.7 # via matplotlib python-dateutil==2.8.1 # via matplotlib, pandas pytz==2020.1 # via pandas scipy==1.4.1 # via seaborn seaborn==0.10.1 six==1.15.0 # via cycler, python-dateutil
Note how the specific versions of the packages are listed, but keep in mind that version numbers could also be listed using
<=, and other operators.
Installing All Required Packages
To install the specified versions of all required packages, once again open your project’s directory in the terminal and ensure that you have the proper Python environment opened. Then, run the following command:
pip install -r requirements.txt
This will install the proper version of all packages listed in
requirements.txt, all in one command. Isn’t that convenient?
Python packages are updated quite frequently. As anyone who frequently uses
requirements.txt on GitHub can tell you, frequent updates are often required to fix security flaws in many widely-used packages. However, as the version of each package is explicitly defined in
requirements.txt, updating is a multi-step process.
To update all of your project’s packages, open your project’s directory in the terminal and ensure that you are inside the proper Python environment. Then, execute the following command:
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
This will fetch and install any available updates to the project’s packeges, along with any new package dependencies. However, you may notice that
requirements.txt remained untouched. Double check that your code still executes properly and, if it does, regenerate the requirements file by repeating the command from above:
pip freeze > requirements.txt
You should now be able to see the new version numbers reflected in the file.
Automating These Processes
You can use the scripts that I have published to this GitHub repository to simplify the processes of installing packages from
requirements.txt and updating your packages. The following shell scripts are included:
#! /bin/bash pip install -r requirements.txt
./install_packages.sh- output of installation from
requirements.txtwill be shown.
#! /bin/bash pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U # Prompt read -p "Would you like to update requirements.txt? (Y/N)" -n 1 -r echo # move to new line if [[ ! $REPLY =~ ^[Yy]$ ]] then exit 1 fi pip freeze > requirements.txt
./update_all_packages.sh- enter Y or N after the updating has been completed if you would like to update the versions of your
Implementing these scripts into your workflow will greatly simplify your Python package updates and installs. I hope you find them as useful as I do!