Beginner’s Guide to Use Python Virtual Environment with Raspberry PI (venv)

Last Updated on 5th April 2024 by peppe8o

This tutorial will show you how to use Python Virtual Environment with Raspberry PI computer boards, with some examples to better understand the logic of Python virtual environments.

This tutorial has the scope to help all the people who can’t install Python packages with the PIP (Package Installer for Python) with the new Bookworm release of Raspberry PI OS, resulting to the following error for any pip installation:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Of course, for this tutorial you will need pip, which can be installed with the following terminal command:

sudo apt install python3-pip -y

What is a Python Virtual Environment

A virtual environment in Python is an isolated space which includes all the required packages for a specific project. Every package added to a virtual environment will work only inside that environment. The good side of this approach is that your project installations will not affect your system-global Python packages, so getting a more clean system.

Moreover, with virtual environments, you can create projects with a clean list of packages and thus get an improved reproducibility of your Python project as you will include only the packages required for that project.

What We Need

As usual, I suggest adding from now to your favourite e-commerce shopping cart all the needed hardware, so that at the end you will be able to evaluate overall costs and decide if to continue with the project or remove them from the shopping cart. So, hardware will be only:

• Raspberry PI Computer Board (including proper power supply or using a smartphone micro USB charger with at least 3A)
• high speed micro SD card (at least 16 GB, at least class 10)

Step-by-Step Procedure to Use Python Virtual Environments with Raspberry PI

There are a few main actions you may need to know when dealing with Python Virtual Environments on Raspberry PI:

• VE Creation
• VE Activation
• VE Deactivation
• Export/Re-install VE packages
• VE Remotion

The following chapters will briefly show you how to perform these actions.

Virtual Environment Creation

Assuming that you want to create a virtual environment named “test” (you can name it as you want, but I suggest avoiding space characters inside the name), you can create it with the following command:

python3 -m venv test

This will create a folder named “test” in your current path, including the following content (blue are folders, the lower will include other subfolders/files):

test/
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── Activate.ps1
│   ├── pip
│   ├── pip3
│   ├── pip3.11
│   ├── python -> python3
│   ├── python3 -> /usr/bin/python3
│   └── python3.11 -> python3
├── include
│   └── python3.11
├── lib
│   └── python3.11
│       └── site-packages
├── lib64 -> lib
└── pyvenv.cfg

The most important are the following:

• The bin directory contains the main executable scripts.
• The include directory contains header files potentially needed by Python packages.
• The lib/pythonX.Y/site-packages contains all the third-party Python packages you install using pip.

The venv utility also enables you to create virtual environments with some specific options regarding the included packages. The full venv help can be shown with the following command:

python3 -m venv -h

It will show you the following options:

  -h, --help            show this help message and exit
  --system-site-packages
                        Give the virtual environment access to the system site-packages dir.
  --symlinks            Try to use symlinks rather than copies, when symlinks are not the default for
                        the platform.
  --copies              Try to use copies rather than symlinks, even when symlinks are the default
                        for the platform.
  --clear               Delete the contents of the environment directory if it already exists, before
                        environment creation.
  --upgrade             Upgrade the environment directory to use this version of Python, assuming
                        Python has been upgraded in-place.
  --without-pip         Skips installing or upgrading pip in the virtual environment (pip is
                        bootstrapped by default)
  --prompt PROMPT       Provides an alternative prompt prefix for this environment.
  --upgrade-deps        Upgrade core dependencies: pip setuptools to the latest version in PyPI

An important option is the “–system-site-packages”: Raspberry PI OS has some python packages pre-installed used to manage the GPIOs which can hardly be re-installed into a clean virtual environment. This option will create a virtual environment able to use the system packages. For this reason, you can create a virtual environment with your Raspberry PI pre-built python packages with the following command:

python3 -m venv my_project --system-site-packages

Virtual Environment Activation

If you want to use your Python virtual environment you must activate it. The activation process requires only to source the “activate” script inside the virtual environment bin folder:

source ./my_project/bin/activate

Please note that this command works if you are in the same folder where your virtual environment has been created. Otherwise, you will need to use the full path. In my case, with the old “pi” user:

source /home/pi/my_project/bin/activate

At any time, you can check your current path with the “pwd” command.

You can be aware of what virtual environment you activated from the new prompt including the virtual environment name at the beginning:

pi@raspberrypi:~ $ source ./my_project/bin/activate
(my_project) pi@raspberrypi:~ $

Virtual Environment Deactivation

When you need to exit from a virtual environment, you can do it with the following simple command:

deactivate

Differently from the activation command, the “deactivate” doesn’t need to specify the bin path of your virtual environment and it will work from any folder.

After the deactivate command, you will get back the classic prompt without the environment name:

(my_project) pi@raspberrypi:~ $ deactivate
pi@raspberrypi:~ $

Export/Re-install VE packages

When you move your project to a different PC (or Raspberry PI) you will need to re-create the same virtual environment with the same packages installed with pip3 in your original one. For this task we can use the pip3 “freeze” option, which collects a list of the installed packages. To test it, let’s activate our “my_project” virtual environment:

source ./my_project/bin/activate

With the Python virtual environment activated, we can list the installed packages by issuing the “pip3 freeze” command:

(my_project) pi@raspberrypi:~ $ pip3 freeze
certifi==2022.9.24
chardet==5.1.0
charset-normalizer==3.0.1
colorzero==2.0
distro==1.8.0
gpiozero==2.0
idna==3.3
lgpio==0.2.2.0
pigpio==1.78
python-apt==2.6.0
requests==2.28.1
RPi.GPIO==0.7.1a4
six==1.16.0
smbus2==0.4.2
spidev==3.5
ssh-import-id==5.10
toml==0.10.2
urllib3==1.26.12

The pip3 freeze command also allows us to better understand the differences between the system packages and virtual environments. Executing this command outside the environments, inside the test environment (created without the “–system-site-packages” option), and the “my_project” environment (created with the “–system-site-packages” option), we’ll get the following results:

System	“my_project” virtual environment	“test” virtual environment
certifi==2022.9.24	certifi==2022.9.24
chardet==5.1.0	chardet==5.1.0
charset-normalizer==3.0.1	charset-normalizer==3.0.1
colorzero==2.0	colorzero==2.0
distro==1.8.0	distro==1.8.0
gpiozero==2.0	gpiozero==2.0
idna==3.3	idna==3.3
lgpio==0.2.2.0	lgpio==0.2.2.0
pigpio==1.78	pigpio==1.78
python-apt==2.6.0	python-apt==2.6.0
requests==2.28.1	requests==2.28.1
RPi.GPIO==0.7.1a4	RPi.GPIO==0.7.1a4
six==1.16.0	six==1.16.0
smbus2==0.4.2	smbus2==0.4.2
spidev==3.5	spidev==3.5
ssh-import-id==5.10	ssh-import-id==5.10
toml==0.10.2	toml==0.10.2
urllib3==1.26.12	urllib3==1.26.12

As you could expect, the “–system-site-packages” option enables you to use a copy of the same packages installed at system level. On the other hand, the “test” virtual environment, created without that option, creates a clean environment.

Now, you can save these packages into a txt file with the following command. With the “my_project” environment activated, please use the

(my_project) pi@raspberrypi:~ $ pip3 freeze > requirements.txt

A good practice is saving this file inside the virtual environment root folder to address multiple requirement files to the correct Virtual Environments when you work with multiple virtual environments.

You can align the packages of your Python virtual environment on Raspberry PI into a new virtual environment by moving to the destination environment and using the following terminal command:

pip3 install -r requirements.txt

Let’s look at the whole process.

Activate the source environment:

source ./my_project/bin/activate

Install a simple pip package: BpyTOP: an Htop and Top Alternative for Raspberry PI:

pip3 install bpytop

Export the packages of “your “my_project” environment:

pip3 freeze > requirements.txt

Deactivate the current environment:

deactivate

Create a new virtual environment with the system packages:

python3 -m venv my_project_copy --system-site-packages

Activate the destination virtual environment:

source ./my_project_copy/bin/activate

Align the packages by installing those listed in our requirements.txt (please check that you activated the correct environment):

(my_project_copy) pi@raspberrypi:~ $ pip3 install -r requirements.txt

By comparing the system, my_project, my_project_copy, and test packages you’ll get these results:

System	my_project virtual environment	my_project_copy virtual environment	test virtual environment
	bpytop==1.0.68	bpytop==1.0.68
certifi==2022.9.24	certifi==2022.9.24	certifi==2022.9.24
chardet==5.1.0	chardet==5.1.0	chardet==5.1.0
charset-normalizer==3.0.1	charset-normalizer==3.0.1	charset-normalizer==3.0.1
colorzero==2.0	colorzero==2.0	colorzero==2.0
distro==1.8.0	distro==1.8.0	distro==1.8.0
gpiozero==2.0	gpiozero==2.0	gpiozero==2.0
idna==3.3	idna==3.3	idna==3.3
lgpio==0.2.2.0	lgpio==0.2.2.0	lgpio==0.2.2.0
pigpio==1.78	pigpio==1.78	pigpio==1.78
	psutil==5.9.8	psutil==5.9.8
python-apt==2.6.0	python-apt==2.6.0	python-apt==2.6.0
requests==2.28.1	requests==2.28.1	requests==2.28.1
RPi.GPIO==0.7.1a4	RPi.GPIO==0.7.1a4	RPi.GPIO==0.7.1a4
six==1.16.0	six==1.16.0	six==1.16.0
smbus2==0.4.2	smbus2==0.4.2	smbus2==0.4.2
spidev==3.5	spidev==3.5	spidev==3.5
ssh-import-id==5.10	ssh-import-id==5.10	ssh-import-id==5.10
toml==0.10.2	toml==0.10.2	toml==0.10.2
urllib3==1.26.12	urllib3==1.26.12	urllib3==1.26.12

As you can see from this comparison, the installation of BpyTOP into the my_project environment added 2 packages only to that environment, without affecting the system and the test packages. Installing the requirements.txt packages list in our new my_project_copy environment enabled us to get a new environment with the exact copy of the packages list.

NOTE 1: running the requirements.txt installation in your test environment (created without the “–system-site-packages” option) will give you errors as some specific Raspberry PI packages seem to be unavailable from the pip installation process. For this reason, I suggest using this option if you want to create projects interacting with the GPIOs.

NOTE 2: the requirements method doesn’t import the scripts that you created into the virtual environment folder, it installs only the packages installed with pip3. For this reason, you will need to move your Python scripts of your project manually.

VE Remotion

Once you deactivated your Python virtual environment, you can delete it by deleting the whole folder. For example, let’s deactivate the my_project_copy virtual environment:

deactivate

and we can remove the Python Virtual Environment from Raspberry PI with the remove (rm) command, using the “-r” option to delete also all the subfolders:

rm -r my_project_copy/

Using Python Virtual Environments in Raspberry PI with Cron

When automating your scripts to run at a specific frequency, you may need to use a crontab line to execute the Python script without any user intervention. In these cases, you will need to concatenate the Virtual Environment activation command with your script command. The command concatenation can be performed with the “&&” operator in somewhat similar to the following:

command1 && command2

This executes the command2 script only at the end of the command1 script. Your crontab line will appear like the following:

0 0 * * * source /home/pi/my_project/bin/activate && python3 /home/pi/my_script.py

What’s Next

If you want to discover many other projects for your Raspberry PI, you can take a look at peppe8o Raspberry PI computer tutorials.

Enjoy!