Drop support for Python 3.6

[jeremystretch]

Proposed Changes

Drop support for Python 3.6 in the next major release.

Justification

Python 3.6 will become unsupported by PSF in December 2021, and dropping support for it will allow us to begin using Python 3.7 features.

[steffann]

Python 3.6 is the default version shipped with Ubuntu 18.04.5 LTS. It has packages for 3.7 and 3.8, but the upgrade.sh script only discovers /usr/bin/python3, which is 3.6. To avoid breaking on LTS releases it may be a good idea to test for specific version (for python in /usr/bin/python3.9 /usr/bin/python3.8 /usr/bin/python3.7; do etc).

[bluikko]

This would make Red Hat users' life very difficult. Red Hat 7 and derivatives have 3.6 supported until 2024 - and those users would be forced to compile Python 3.7 from source or move to 8. I have not seen 3.7 RPMs anywhere.

[candlerb]

TL;DR: I propose that the Netbox doesn't drop support for python 3.6 until the first major release of Netbox which requires Django 4.0.

Long version:

Moving to python 3.7 is painful for users who are currently on Ubuntu 18.04 or RedHat, as already described.

I don't see any compelling python language features in 3.7 (*); ISTM the main driver is when Django drops 3.6 support.

The next LTS release of Django will be 3.2, due April 2021 and supported to April 2024. The FAQ says:

Typically, we will support a Python version up to and including the first Django LTS release whose security support ends after security support for that version of Python ends. For example, Python 3.3 security support ended September 2017 and Django 1.8 LTS security support ended April 2018. Therefore Django 1.8 is the last version to support Python 3.3.

If I understand it correctly, this means that Django 3.2 will likely support python 3.6 until April 2024, which is a year past the EOL of Ubuntu 18.04.

What if Netbox wants to move to Django 4.0? That isn't due until December 2021, around the same time as python 3.6 officially ends support. But I don't expect adoption of 4.0 until well into 2022 after it's had time to stabilise.

That is - some time in mid to late 2022, there may be a major release of Netbox which depends on Django 4.0 features. I think that would be the time to drop python 3.6 support.

---

(*) apart from the insertion-order preserving nature of dicts, but AFAIK this is already the case in 3.6, it's just "official" in 3.7.

---

@steffann: if the venv already exists then it could deference the symlink venv/bin/python to pick up the same version as was used previously.

Of course, the logic you suggest would still be needed the first time that upgrade.sh is run. Another option is to run python3 -V and only go hunting if the version is too low - or prompt the user to pick the right version.

[tyler-8]

I think it does make sense to stop supporting Python 3.6 - but I don't agree with the timing proposed in the OP.

At the earliest I think it should be done in step with the PSF dropping support (end of 2021).

Additionally, the announcement of the chosen timeframe should be added to all NetBox release notes up until that point. That will give the community plenty of opportunities to be informed of the coming change as well as give time to plan & make infrastructure changes on the user side.

[jeremystretch]

Moving to python 3.7 is painful for users who are currently on Ubuntu 18.04 or RedHat, as already described.

I think there's some confusion regarding the default Python version and available versions. I've had Python 3.7 installed on Ubuntu for a while now; it's quite easy to do. NetBox runs in its own virtual environment, which can use Python 3.7 (or any other version) independently from the system default. If there's some limitation preventing you from installing a version of Python that was released over two years ago, that's something to take up with your distro.

But I don't expect adoption of 4.0 until well into 2022 after it's had time to stabilise.

We always adopt the latest stable Django release during each major NetBox release. This won't change with Django 4.0.

[candlerb]

Sure, but even if Netbox adopts Django 4.0 at the earliest opportunity, this won't be until the first major Netbox release after December 2021.

I also note that if Netbox requires a non-default python3, there will be some housekeeping required - e.g. changing "python3" to "python3.7" at appropriate points in the documentation and in installation scripts.

I just don't see what the push is to drop 3.6 right now, rather than concurrently with adopting Django 4.0.

[jeremystretch]

For context, "right now" is two and a half years after Python 3.7's release, and roughly ten months until EOL for Python 3.6. This change wouldn't take effect until the release of NetBox 2.11 anyway, which given the current state of things likely won't even come out until the second half of 2021.

[candlerb]

Right now is also >2 years before EOL of Ubuntu 18.04. As you say, it's possible to install another python version, and I wouldn't have much difficulty doing this on my own instances; but it is going to complicate the upgrade process, and I would expect more support requests generated as a result. Leaving it later means that organically more people will be running on more recent OSes anyway.

Those running CentOS are in a worse situation, especially now that IBM have announced they will abandon CentOS 8 at the end of this year, so there will be a longer tail of people sticking to CentOS 7.

I have no issue with dropping 3.6 when Django, or any other key dependency, drops 3.6.

What's the plus side? Faster merge of pull requests because the CI pipeline can drop 3.6 for tests?

[jeremystretch]

What's the plus side?

We can start using Python 3.7 features, as stated above. I understand that this probably isn't an appreciable difference unless you're actively engaged in the project's development, but that's how it goes. Plus, we can move testing up from 3.6-3.8 to 3.7-3.9, to better accommodate the portion of our user base that isn't restricted to a nearly-EOL Python version.

Users who for whatever reason are unwilling or unable to install Python 3.7 on their NetBox deployment have the option of sticking with NetBox 2.10 until an upgrade path becomes available to them.

[bluikko]

Users who for whatever reason are unwilling or unable to install Python 3.7 on their NetBox deployment have the option of sticking with NetBox 2.10 until an upgrade path becomes available to them.

Does this mean there will be security and other important updates for 2.10 even after 2.11 is released?

I tried to find release lifecycle policy but failed. A cursory look at releases since 1.9.x shows that after a new minor version is released there has been no more patch releases for the old minor version.

Edit: "after 2.11", not "after 2.9".

[jeremystretch]

Does this mean there will be security and other important updates for 2.10 even after 2.11 is released?

No, in keeping with established precedence there will be no further releases of 2.10 once 2.11 is released, much like Python 3.6 in December.

[candlerb]

We can start using Python 3.7 features, as stated above

Any feature in particular that's desirable to incorporate?

I just think this needs to be balanced against the inconvenience this will cause to the user base.

[jeremystretch]

I'm not going to keep going in circles with this. The question is not "why should we drop old software?" It's "why shouldn't we drop old software?"

The only argument provided in opposition is that it will be moderately inconvenient - though far from infeasible - for certain users on older distributions to install Python 3.7. This is not sufficient justification for delaying the move to 3.7. Further, these users are still going to have to migrate eventually anyway, whether we make the move in NetBox 2.11 or 2.12 or 2.13: It's still going to happen one way or the other, so might as well move forward now so we can at least start to reap the benefits.

[bluikko]

inb4 locked

I'm not going to keep going in circles with this. The question is not "why should we drop old software?" It's "why shouldn't we drop old software?"

I don't think that was the question at all. It was "what is the benefit in 3.7".

[jeremystretch]

Locking as this has been settled.

[jeremystretch]

Bumping this to v2.12 as v2.11 will be released sooner than anticipated.

[yrro]

This would make Red Hat users' life very difficult. Red Hat 7 and derivatives have 3.6 supported until 2024 - and those users would be forced to compile Python 3.7 from source or move to 8. I have not seen 3.7 RPMs anywhere.

FWIW, Red Hat Software Collections 3.5 introduced Python 3.8, which will be supported until May 2023.