Beginner’s Guide to Use Python Virtual Environment with Raspberry PI (venv)
Last Updated on 5th April 2024 by peppe8o
This tutorial will show you how to use Python Virtual Environment with Raspberry PI computer boards, with some examples to better understand the logic of Python virtual environments.
This tutorial has the scope to help all the people who can’t install Python packages with the PIP (Package Installer for Python) with the new Bookworm release of Raspberry PI OS, resulting to the following error for any pip installation:
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
For more information visit http://rptl.io/venv
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.Of course, for this tutorial you will need pip, which can be installed with the following terminal command:
sudo apt install python3-pip -yWhat is a Python Virtual Environment
A virtual environment in Python is an isolated space which includes all the required packages for a specific project. Every package added to a virtual environment will work only inside that environment. The good side of this approach is that your project installations will not affect your system-global Python packages, so getting a more clean system.
Moreover, with virtual environments, you can create projects with a clean list of packages and thus get an improved reproducibility of your Python project as you will include only the packages required for that project.
What We Need
As usual, I suggest adding from now to your favourite e-commerce shopping cart all the needed hardware, so that at the end you will be able to evaluate overall costs and decide if to continue with the project or remove them from the shopping cart. So, hardware will be only:
- Raspberry PI Computer Board (including proper power supply or using a smartphone micro USB charger with at least 3A)
- high speed micro SD card (at least 16 GB, at least class 10)
Step-by-Step Procedure to Use Python Virtual Environments with Raspberry PI
There are a few main actions you may need to know when dealing with Python Virtual Environments on Raspberry PI:
- VE Creation
- VE Activation
- VE Deactivation
- Export/Re-install VE packages
- VE Remotion
The following chapters will briefly show you how to perform these actions.
Virtual Environment Creation
Assuming that you want to create a virtual environment named “test” (you can name it as you want, but I suggest avoiding space characters inside the name), you can create it with the following command:
python3 -m venv testThis will create a folder named “test” in your current path, including the following content (blue are folders, the lower will include other subfolders/files):
test/
├── bin
│ ├── activate
│ ├── activate.csh
│ ├── activate.fish
│ ├── Activate.ps1
│ ├── pip
│ ├── pip3
│ ├── pip3.11
│ ├── python -> python3
│ ├── python3 -> /usr/bin/python3
│ └── python3.11 -> python3
├── include
│ └── python3.11
├── lib
│ └── python3.11
│ └── site-packages
├── lib64 -> lib
└── pyvenv.cfgThe most important are the following:
- The bin directory contains the main executable scripts.
- The include directory contains header files potentially needed by Python packages.
- The lib/pythonX.Y/site-packages contains all the third-party Python packages you install using pip.
The venv utility also enables you to create virtual environments with some specific options regarding the included packages. The full venv help can be shown with the following command:
python3 -m venv -hIt will show you the following options:
-h, --help show this help message and exit
--system-site-packages
Give the virtual environment access to the system site-packages dir.
--symlinks Try to use symlinks rather than copies, when symlinks are not the default for
the platform.
--copies Try to use copies rather than symlinks, even when symlinks are the default
for the platform.
--clear Delete the contents of the environment directory if it already exists, before
environment creation.
--upgrade Upgrade the environment directory to use this version of Python, assuming
Python has been upgraded in-place.
--without-pip Skips installing or upgrading pip in the virtual environment (pip is
bootstrapped by default)
--prompt PROMPT Provides an alternative prompt prefix for this environment.
--upgrade-deps Upgrade core dependencies: pip setuptools to the latest version in PyPIAn important option is the “–system-site-packages”: Raspberry PI OS has some python packages pre-installed used to manage the GPIOs which can hardly be re-installed into a clean virtual environment. This option will create a virtual environment able to use the system packages. For this reason, you can create a virtual environment with your Raspberry PI pre-built python packages with the following command:
python3 -m venv my_project --system-site-packagesVirtual Environment Activation
If you want to use your Python virtual environment you must activate it. The activation process requires only to source the “activate” script inside the virtual environment bin folder:
source ./my_project/bin/activatePlease note that this command works if you are in the same folder where your virtual environment has been created. Otherwise, you will need to use the full path. In my case, with the old “pi” user:
source /home/pi/my_project/bin/activateAt any time, you can check your current path with the “pwd” command.
You can be aware of what virtual environment you activated from the new prompt including the virtual environment name at the beginning:
pi@raspberrypi:~ $ source ./my_project/bin/activate
(my_project) pi@raspberrypi:~ $Virtual Environment Deactivation
When you need to exit from a virtual environment, you can do it with the following simple command:
deactivateDifferently from the activation command, the “deactivate” doesn’t need to specify the bin path of your virtual environment and it will work from any folder.
After the deactivate command, you will get back the classic prompt without the environment name:
(my_project) pi@raspberrypi:~ $ deactivate
pi@raspberrypi:~ $Export/Re-install VE packages
When you move your project to a different PC (or Raspberry PI) you will need to re-create the same virtual environment with the same packages installed with pip3 in your original one. For this task we can use the pip3 “freeze” option, which collects a list of the installed packages. To test it, let’s activate our “my_project” virtual environment:
source ./my_project/bin/activateWith the Python virtual environment activated, we can list the installed packages by issuing the “pip3 freeze” command:
(my_project) pi@raspberrypi:~ $ pip3 freeze
certifi==2022.9.24
chardet==5.1.0
charset-normalizer==3.0.1
colorzero==2.0
distro==1.8.0
gpiozero==2.0
idna==3.3
lgpio==0.2.2.0
pigpio==1.78
python-apt==2.6.0
requests==2.28.1
RPi.GPIO==0.7.1a4
six==1.16.0
smbus2==0.4.2
spidev==3.5
ssh-import-id==5.10
toml==0.10.2
urllib3==1.26.12The pip3 freeze command also allows us to better understand the differences between the system packages and virtual environments. Executing this command outside the environments, inside the test environment (created without the “–system-site-packages” option), and the “my_project” environment (created with the “–system-site-packages” option), we’ll get the following results:
| System | “my_project” virtual environment | “test” virtual environment |
|---|---|---|
| certifi==2022.9.24 | certifi==2022.9.24 | |
| chardet==5.1.0 | chardet==5.1.0 | |
| charset-normalizer==3.0.1 | charset-normalizer==3.0.1 | |
| colorzero==2.0 | colorzero==2.0 | |
| distro==1.8.0 | distro==1.8.0 | |
| gpiozero==2.0 | gpiozero==2.0 | |
| idna==3.3 | idna==3.3 | |
| lgpio==0.2.2.0 | lgpio==0.2.2.0 | |
| pigpio==1.78 | pigpio==1.78 | |
| python-apt==2.6.0 | python-apt==2.6.0 | |
| requests==2.28.1 | requests==2.28.1 | |
| RPi.GPIO==0.7.1a4 | RPi.GPIO==0.7.1a4 | |
| six==1.16.0 | six==1.16.0 | |
| smbus2==0.4.2 | smbus2==0.4.2 | |
| spidev==3.5 | spidev==3.5 | |
| ssh-import-id==5.10 | ssh-import-id==5.10 | |
| toml==0.10.2 | toml==0.10.2 | |
| urllib3==1.26.12 | urllib3==1.26.12 |
As you could expect, the “–system-site-packages” option enables you to use a copy of the same packages installed at system level. On the other hand, the “test” virtual environment, created without that option, creates a clean environment.
Now, you can save these packages into a txt file with the following command. With the “my_project” environment activated, please use the
(my_project) pi@raspberrypi:~ $ pip3 freeze > requirements.txtA good practice is saving this file inside the virtual environment root folder to address multiple requirement files to the correct Virtual Environments when you work with multiple virtual environments.
You can align the packages of your Python virtual environment on Raspberry PI into a new virtual environment by moving to the destination environment and using the following terminal command:
pip3 install -r requirements.txtLet’s look at the whole process.
Activate the source environment:
source ./my_project/bin/activateInstall a simple pip package: BpyTOP: an Htop and Top Alternative for Raspberry PI:
pip3 install bpytopExport the packages of “your “my_project” environment:
pip3 freeze > requirements.txtDeactivate the current environment:
deactivateCreate a new virtual environment with the system packages:
python3 -m venv my_project_copy --system-site-packagesActivate the destination virtual environment:
source ./my_project_copy/bin/activateAlign the packages by installing those listed in our requirements.txt (please check that you activated the correct environment):
(my_project_copy) pi@raspberrypi:~ $ pip3 install -r requirements.txtBy comparing the system, my_project, my_project_copy, and test packages you’ll get these results:
| System | my_project virtual environment | my_project_copy virtual environment | test virtual environment |
|---|---|---|---|
| bpytop==1.0.68 | bpytop==1.0.68 | ||
| certifi==2022.9.24 | certifi==2022.9.24 | certifi==2022.9.24 | |
| chardet==5.1.0 | chardet==5.1.0 | chardet==5.1.0 | |
| charset-normalizer==3.0.1 | charset-normalizer==3.0.1 | charset-normalizer==3.0.1 | |
| colorzero==2.0 | colorzero==2.0 | colorzero==2.0 | |
| distro==1.8.0 | distro==1.8.0 | distro==1.8.0 | |
| gpiozero==2.0 | gpiozero==2.0 | gpiozero==2.0 | |
| idna==3.3 | idna==3.3 | idna==3.3 | |
| lgpio==0.2.2.0 | lgpio==0.2.2.0 | lgpio==0.2.2.0 | |
| pigpio==1.78 | pigpio==1.78 | pigpio==1.78 | |
| psutil==5.9.8 | psutil==5.9.8 | ||
| python-apt==2.6.0 | python-apt==2.6.0 | python-apt==2.6.0 | |
| requests==2.28.1 | requests==2.28.1 | requests==2.28.1 | |
| RPi.GPIO==0.7.1a4 | RPi.GPIO==0.7.1a4 | RPi.GPIO==0.7.1a4 | |
| six==1.16.0 | six==1.16.0 | six==1.16.0 | |
| smbus2==0.4.2 | smbus2==0.4.2 | smbus2==0.4.2 | |
| spidev==3.5 | spidev==3.5 | spidev==3.5 | |
| ssh-import-id==5.10 | ssh-import-id==5.10 | ssh-import-id==5.10 | |
| toml==0.10.2 | toml==0.10.2 | toml==0.10.2 | |
| urllib3==1.26.12 | urllib3==1.26.12 | urllib3==1.26.12 |
As you can see from this comparison, the installation of BpyTOP into the my_project environment added 2 packages only to that environment, without affecting the system and the test packages. Installing the requirements.txt packages list in our new my_project_copy environment enabled us to get a new environment with the exact copy of the packages list.
NOTE 1: running the requirements.txt installation in your test environment (created without the “–system-site-packages” option) will give you errors as some specific Raspberry PI packages seem to be unavailable from the pip installation process. For this reason, I suggest using this option if you want to create projects interacting with the GPIOs.
NOTE 2: the requirements method doesn’t import the scripts that you created into the virtual environment folder, it installs only the packages installed with pip3. For this reason, you will need to move your Python scripts of your project manually.
VE Remotion
Once you deactivated your Python virtual environment, you can delete it by deleting the whole folder. For example, let’s deactivate the my_project_copy virtual environment:
deactivateand we can remove the Python Virtual Environment from Raspberry PI with the remove (rm) command, using the “-r” option to delete also all the subfolders:
rm -r my_project_copy/Using Python Virtual Environments in Raspberry PI with Cron
When automating your scripts to run at a specific frequency, you may need to use a crontab line to execute the Python script without any user intervention. In these cases, you will need to concatenate the Virtual Environment activation command with your script command. The command concatenation can be performed with the “&&” operator in somewhat similar to the following:
command1 && command2This executes the command2 script only at the end of the command1 script. Your crontab line will appear like the following:
0 0 * * * source /home/pi/my_project/bin/activate && python3 /home/pi/my_script.pyWhat’s Next
If you want to discover many other projects for your Raspberry PI, you can take a look at peppe8o Raspberry PI computer tutorials.
Enjoy!
