Cannot create new virtualenv with "system" Python if it's 3.11+

[millerdev]

pyenv-virtualenv is unable to create new virtualenvs using the "system" Python since upgrading to Ubuntu 23.04. Example output:

$ pyenv virtualenv system cloud
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

This worked on earlier versions of Ubuntu, presumably because they had Python 3.10 or earlier.

It is possible to work around the issue by creating the virtualenv manually, but it seems like pyenv-virtualenv should be able to do this automatically:

python3 -m venv ~/.pyenv/versions/cloud

System details:

• Platform: Ubuntu 23.04
• OS architecture: x86_64
• pyenv version: pyenv 2.3.0-14-g0930533d
• pyenv-virtualenv version: pyenv-virtualenv 1.2.1 (virtualenv unknown)
• Python version (system): Python 3.11.4
• Debug log: https://gist.github.com/millerdev/f9c4370dab04d88a0ff6538fad101f18

[laur89]

Underlying cause is PEP 668 – Marking Python base environments as “externally managed”

Note this only happens when referencing system version - doesn't reproduce using py version managed by pyenv:

$ pyenv virtualenv  3.9.7  my-env
Looking in links: /tmp/tmp7dv5m49y
Requirement already satisfied: setuptools in /home/laur89/.pyenv/versions/3.9.7/envs/my-env/lib/python3.9/site-packages (57.4.0)
Requirement already satisfied: pip in /home/laur89/.pyenv/versions/3.9.7/envs/my-env/lib/python3.9/site-packages (21.2.3)