Adding more Python libraries

Although Python’ „Batteries included“ philosophy means that you can already do a lot with the default installation of Python, there will inevitably come a situation where you need functionality that is not included in Python.

If you need a third-party module that is not pre-built for your platform, you must use its source distribution. However, this causes two problems:

• To install the source distribution, you need to find and download it.

• Certain Python paths and authorisations of your system are expected.

Python offers pip as a current solution for both problems. pip tries to find the module in the Python Package Index (PyPI), downloads it and all dependencies and takes care of the installation. You can also call pypi.org directly and search for packages or filter the packages by category.

Warning

Never install anything with pip into the global Python, not even with the --user flag. Always use venv. This way you avoid contaminating your Python installation with libraries that you install and then forget about. Every time you need to do something new, you should create a new virtual environment. This will also avoid library conflicts between different projects.

Tip

We recommend that you configure pip so that it is not possible to install Python packages globally. To do this, you can enter the following in your ~/.config/pip/pip.conf:

[global]
require-virtualenv = true

venv

A virtual environment (virtualenv) is a self-contained directory structure that contains both an installation of Python and the additional packages. Since the entire Python environment is contained in this directory, the libraries and modules installed there cannot collide with those in the main system or in other virtual environments, so that different applications can use different versions of Python and its packages. Creating and using a virtual environment is a two-step process:

1. First we create a project directory and then the virtual environment in it:

   Linux/macOS

   $ mkdir myproj
   $ cd myproj
   $ python3 -m venv .venv
   

   Windows

   > mkdir myproj
   > cd myproj
   > py -m venv .venv
   

   This creates the environment with Python and pip in a directory called .venv.

2. You can then activate this environment so that the next time you call python, it will use the Python from your new environment:

   Linux/macOS

   $ . .venv/bin/activate
   

   Windows

   > .venv\Scripts\activate
   

3. Install Python packages only for for this virtual environment, for example, the popular pandas library:

   Linux/macOS

   (.venv) $ python -m pip install pandas
   

   Windows

   (.venv) > python.exe -m pip install pandas
   

4. If you want to finish your work on this project, you can deactivate the virtual environment again with

   Linux/macOS

   (.venv) $ deactivate
   

   Windows

   (.venv) > deactivate
   

See also

• Virtual Environments and Packages

pip

The basic syntax of pip is quite simple:

(.venv) $ python -m pip install pandas

If you want to specify a particular version of a package, you can simply append the version numbers:

(.venv) $ python -m pip install pandas==2.2.2

or

(.venv) $ python -m pip install "pandas>=2"

Proxy server

To install Python packages via a proxy server, you can enter the following:

python -m pip install --proxy http://USER_NAME:{PASSWORD}@PROXYSERVER_NAME:PORT PKG_NAME

You can also save the proxy server permanently as an environment variable:

Linux

for example in the ~/.bashrc with:

$ export HTTP_PROXY=http://{USER_NAME}:{PASSWORD}@{PROXYSERVER_NAME}:{PORT}

Windows

Add the following line to the environment variables:

> set HTTP_PROXY={PROXYSERVER_NAME}:{PORT}

Pinning …

… of Python

In contrast to applications, our packages usually support more than one Python version. Nevertheless, we usually add the current standard version in .python-version to packages as well:

.python-version

3.14

The nice thing about this is that we can use the same file in GitHub Actions as input for setup-python:

.github/workflows/ci.yml

jobs:
  docs:
    name: Build docs and check links
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
      with:
        persist-credentials: false
    - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
      with:
        # Keep in sync with .readthedocs.yaml
        python-version-file: .python-version

In our GitLab CI/CD pipelines, however, we use requires-python from the pyproject.toml file to build Docker containers with the appropriate Python version.

… of packages

To ensure a stable environment, it is advisable to specify the exact versions of the dependencies.

Tip

In none of our library projects does so much happen that the Git history should mainly consist of updates. We only restrict the version numbers to be used in the event of problems. For apps, however, we specify the version numbers.

To pin versions for our applications and maintain cross-platform lock files, we usually use uv. uv also helps us to ensure reproducible Python environments.

However, packages can also be pinned using pip≥26.1:

$ python -m pip install --upgrade pip
$ python -m pip --version

PEP 751 defined the format for the pylock.toml file, which can be generated using the following command, for example:

$ python -m pip lock -e . -o pylock.prod.toml

Alternatively, a pylock.toml file can be generated from a requirements.txt file:

$ python -m pip lock -r requirements.txt -o pylock.prod.toml

Warning

In both cases, however, only the packages for the current platform and Python version are specified.

Tip

If pip is not the only tool available, the cross-platform output of uv export is the smoother option.

The packages specified in pylock.NAME.toml can then be installed in a different environment using:

$ python -m pip install -r pylock.prod.toml --no-deps

--no-deps

ensures that transitive dependencies are not resolved in addition to the pylock.NAME.toml.

See also

• pip lock

• pylock.toml Specification

uv

uv simplifies the creation of an initial project structure and the management of your dependencies.

Note

Many coding agents typically use pip when installing packages or running scripts. We therefore need to configure them first to use uv:

AGENTS.md

- Use `uv` to manage Python environments and dependencies.
- Use `uv run` to execute Python scripts and commands.
- Don't edit `pyproject.toml` directly. Instead, use `uv add` and `uv add --dev` to manage dependencies.

See also

• uv

Installation

uv does not depend on Python. Pre-compiled, standalone binaries can be installed on Linux, macOS and Windows:

Linux/macOS

$ curl -LsSf https://astral.sh/uv/install.sh | sh

Windows

> powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

uv updates itself regularly with this installation.

Automatic shell completion

To activate automatic shell completion for uv commands, carry out one of the following steps:

Linux/macOS

Specify your shell, for example with echo $SHELL, then execute one of the following commands:

$ echo 'eval "$(uv generate-shell-completion bash)"' >> ~/.bashrc
$ echo 'eval "$(uv generate-shell-completion zsh)"' >> ~/.zshrc
$ echo 'uv generate-shell-completion fish | source' > ~/.config/fish/completions/uv.fish
$ echo 'eval (uv generate-shell-completion elvish | slurp)' >> ~/.elvish/rc.elv

$ echo 'eval "$(uvx --generate-shell-completion bash)"' >> ~/.bashrc
$ echo 'eval "$(uvx --generate-shell-completion zsh)"' >> ~/.zshrc
$ echo 'uvx --generate-shell-completion fish | source' > ~/.config/fish/completions/uvx.fish
$ echo 'eval (uvx --generate-shell-completion elvish | slurp)' >> ~/.elvish/rc.elv

Windows

if (!(Test-Path -Path $PROFILE)) {
  New-Item -ItemType File -Path $PROFILE -Force
}
Add-Content -Path $PROFILE -Value '(& uv generate-shell-completion powershell) | Out-String | Invoke-Expression'

if (!(Test-Path -Path $PROFILE)) {
  New-Item -ItemType File -Path $PROFILE -Force
}
Add-Content -Path $PROFILE -Value '(& uvx --generate-shell-completion powershell) | Out-String | Invoke-Expression'

Then restart the shell or call up source with your shell configuration file.

Update

You can easily update uv with:

$ uv self update
info: Checking for updates...
success: Upgraded uv from v0.8.12 to v0.8.13! https://github.com/astral-sh/uv/releases/tag/0.8.13

Python installation

The current Python version can be installed with uv python install. Alternatively, a specific version can be installed with uv python install 3.14. However, you can install not only older CPython versions, but also PyPy with uv python install pypy@3.14 or Free-threaded Python with uv python install --python-preferenc only-managed 3.14t. You can see the Python versions that are already installed with uv python list. You can call up an installed Python version with uv run --python 3.14 python.

Create project structure

Depending on whether you want to create a library or an application, uv can create a suitable project structure.

Installing dependencies

Using uv sync --frozen you can install your project’s dependencies in the exact variants specified in the uv.lock file.

See also

• uv.lock file

• Reproducing the Python environment

Using uv pip install --pylock pylock.NAME.toml, you can also install dependencies from an existing pylock.NAME.toml file.

Adding dependencies

Using uv add PACKAGE, you can add further dependencies to your project. This adds the package to the dependencies section of the pyproject.toml file and writes the exact variant to the uv.lock file.