set_env substiution regression in tox 4.x #2972
Comments
|
PR welcome. 👍 |
|
So, question which would guide a potential PR: I've observed another behavior discrepancy between tox 3 and tox 4, and am not sure if I should try to restore the tox 3 behavior or maintain the behavior in tox 4 (I could not find anything in either the tox 3 or tox 4 documentation that specifies the expected behavior). In tox 3, the order of section substitution relative to explicit keys is significant: [tox]
envlist =
main
[testenv:base]
setenv =
FOO = 2
[testenv:main]
setenv =
FOO = 3
{[testenv:base]setenv}will set the environment variable [tox]
envlist =
main
[testenv:base]
setenv =
FOO = 2
[testenv:main]
setenv =
{[testenv:base]setenv}
FOO = 3will set the environment variable However, in tox 4, for both of the examples, Any input regarding what the desired behavior should be is welcome. |
|
This is expected 🤔 though can be convinced that last found key should win over first found 🤷 |
|
So another question, if you don't mind: commit 4cf619a introduced what looks to me like a level of indirection using |
|
The core idea is to make configuration definition and manifestation lazy. So tox is now faster because it only does work when it must. |
setenv defined explicitly until regression fixed in tox: * tox-dev/tox#2972 * tox-dev/tox#2872
setenv defined explicitly until regression fixed in tox: * tox-dev/tox#2972 * tox-dev/tox#2872
setenv defined explicitly until regression fixed in tox: * tox-dev/tox#2972 * tox-dev/tox#2872
|
I had to put this issue on the back burner due to other responsibilities, but today I had a bit of time to look at it again. So it looks like tox is making use of futures in |
|
You could try to remove it 🤔 in a PR. |
gaborbernat
added
the
help:wanted
|
I also faced the same issue. The substitution done inside tox file works as expected, but the environment of commands is not set properly. A workaround is to move all the variables setting that I want exported out of the common environment. |
Issue
In tox 4.x, with a
set_envkey that contains a substitution from another section as well as an environment variable substitution, the values that should have been substituted from another section are lost. For more information, see the example below.Environment
Provide at least:
OS: Linux
`pip list` of the host Python where `tox` is installed:
Output of running tox
Provide the output of `tox -rvv`:
Minimal example
If possible, provide a minimal reproducer for the issue:
The following tox.ini works on tox 3.x (the
BARenvironment variable is present as expected) but fails on tox 4.x (theBARenvironment variable should be inherited from[testenv:base]set_envbut is absent):The issue is that
SetEnv.__iter__()gets called recursively, which effectively causes some of the keys to be lost. The first time it is called viaVirtualEnv.session(), which ends up invokingToxEnv.environment_variables(). At this point, theSetEnv._rawdict has one key,FOO, andSetEnv._needs_replacementis['{[testenv:base]set_env}'].ToxEnv.environment_variables()calls theSetEnv.load()method on each key thatSetEnv.__iter__()yields, so onceSetEnv.__iter__()yieldsFOO(as a result of this line),ToxEnv.environment_variables()callsset_env.load('FOO')(see here). In the process of resolving the value forFOO, thereplace_env()function ends up being called. This in turn callsSetEnv.__contains__()which in turn callsSetEnv.__iter__()a second time. Because the key being tested (in this caseFOO2) is not present, the second call toSetEnv.__iter__()will iterate over all the keys, including expanding the entries from theself._needs_replacementattribute. Those keys will be stored in theself._rawattribute, but since the the first call toSetEnv.__iter__()has already made a copy of theself._rawattribute to protect against concurrent mutation, it cannot yield them. Furthermore, once it is done yielding from its copy of theself._rawkeys, the first call finds thatself._needs_replacementis empty, because the second call toSetEnv.__iter__()has already processed all of those entries. Thus the firstSetEnv.__iter__()call never yields the keys fromself._needs_replacementwhich were expanded by the second.__iter__()call, and from the perspective of theToxEnv.environment_variables()call, they are lost.Below is an annotated call stack from when the problem occurs:
The text was updated successfully, but these errors were encountered: