Set up a virtual environment or install pipenv to cope with the new Ubuntu runner

[Kodiologist]

Justification:
GitHub is in the process of updating ubuntu-latest to Ubuntu 24.04 (actions/runner-images#10636), which leads to pip install crashing with error: externally-managed-environment.

Description:
Prevent people's jobs from breaking by making pip install work (presumably by either setting up a virtual environment or by implicitly enabling --break-system-packages). Or perhaps run sudo apt install pipenv and tell people to use pipenv instead of pip, but then everyone will still have to change their jobs.

Are you willing to submit a PR?
Nah.

[mahabaleshwars]

Hi @Kodiologist,
Thank you for creating this issue. We will investigate it and provide feedback as soon as we have some updates.

[Kodiologist]

Apparently, this already works, at least sometimes, but I remember having trouble with it earlier. I'm not sure what's going on.

[tcvall86]

Hmm it works for our workloads using ubuntu 24.04. we run

      - name: Set up Python
        uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5
        with:
          python-version: '3.12'
          cache: 'pip'
          cache-dependency-path: '**/requirements.txt'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r path/to/requirements/requirements.txt

Tested it hard against ubuntu-24.04 runner just to be sure

[devanubis]

This morning we've had a load of dependabot PRs run into this issue.

Each of our repos actions are running against ubuntu-latest and now that's running as ubuntu-2024.04 (specifically 24.04.1), where as on Friday our last actions run was against 22.04.5.

We're using https://github.com/abatilo/actions-poetry but it's failing at the most basic step: pip install --user pipx

Run abatilo/actions-poetry@v3
  with:
    poetry-version: latest
Run pip install --user pipx
  pip install --user pipx
  pipx ensurepath
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Error: Process completed with exit code 1.

For now I'm setting our various actions runners to ubuntu-22.04.

If the intention is to keep the PEP 668 behaviour with actions/setup-pytohn then that's fine, please just let us know so we can re-write our actions to use a virtual environment.

Thanks for any help.

[docktermj]

We're seeing the requirement for a virtual environment in dependabot pull requests, too.

Before we make massive changes to our GitHub Actions, we seek guidance from the authors of actions/setup-python to know what direction they are heading.

Thank you for your good work!

[docktermj]

As a response to the requirements of the new ubuntu-latest runner, we're considering the following GitHub action pattern. Should we be using a different approach?

[repository-name]/.github/workflows/check-dependencies.yaml

name: Check dependencies

on: [pull_request]

permissions:
  contents: read

jobs:
  check-dependencies:
    name: Check dependencies
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5

      - name: Install dependencies
        run: |
          python -m venv ./venv
          source ./venv/bin/activate
          echo "PATH=${PATH}" >> "${GITHUB_ENV}"
          python -m pip install --upgrade pip
          python -m pip install --requirement requirements.txt

[mxmlnkn]

As a response to the requirements of the new ubuntu-latest runner, we're considering the following GitHub action pattern. Should we be using a different approach?

Note that venv/activate also sets VIRTUAL_ENV, not just PATH. I'm not sure how important it is though.

Personally, I use this on all my systems.

export PIP_BREAK_SYSTEM_PACKAGES=1
echo "export PIP_BREAK_SYSTEM_PACKAGES=1" >> "$HOME/.bashrc"

I have added this step to my workflow YAML:

jobs:
  Tests:
    steps:
    - name: Fix pip
      run: |
        echo "PIP_BREAK_SYSTEM_PACKAGES=1" >> "${GITHUB_ENV}"

Alternatively, it can be done via pip.conf:

echo '[global]\nbreak-system-packages = true' >> "$HOME/.config/pip/pip.conf"

MacOS:

echo '[global]\nbreak-system-packages = true' >> "$HOME/Library/Application Support/pip/pip.conf"