Intermittent errors with tests on x86_64 / amd64

[johanneskastl]

When packaging this for openSUSE Tumbleweed, we get intermittent build errors due to failing tests.

Today this one errored out:

[  126s] =================================== FAILURES ===================================
[  126s] ___________________________ test_capture_pane_start ____________________________
[  126s] 
[  126s] session = Session($1 libtmux_udfg_nz8)
[  126s] 
[  126s]     def test_capture_pane_start(session: Session) -> None:
[  126s]         env = shutil.which("env")
[  126s]         assert env is not None, "Cannot find usable `env` in PATH."
[  126s]     
[  126s]         session.new_window(
[  126s]             attach=True,
[  126s]             window_name="capture_pane_start",
[  126s]             window_shell=f"{env} PS1='$ ' sh",
[  126s]         )
[  126s]         pane = session.attached_window.attached_pane
[  126s]         assert pane is not None
[  126s]         pane_contents = "\n".join(pane.capture_pane())
[  126s]         assert pane_contents == "$"
[  126s]         pane.send_keys(r'printf "%s"', literal=True, suppress_history=False)
[  126s]         pane_contents = "\n".join(pane.capture_pane())
[  126s]         assert pane_contents == '$ printf "%s"\n$'
[  126s]         pane.send_keys("clear -x", literal=True, suppress_history=False)
[  126s]         pane_contents = "\n".join(pane.capture_pane())
[  126s] >       assert pane_contents == "$"
[  126s] E       assert '$ printf "%s"\n$ clear -x' == '$'
[  126s] E         - $
[  126s] E         + $ printf "%s"
[  126s] E         + $ clear -x
[  126s] 
[  126s] tests/test_pane.py:113: AssertionError
[  126s] =============================== warnings summary ===============================
[...]

tmux version was tmux-3.3a-2.2.

[johanneskastl]

And another one:

[  242s] =================================== FAILURES ===================================
[  242s] ___________________________ test_function_times_out ____________________________
[  242s] 
[  242s]     def test_function_times_out() -> None:
[  242s]         ini = time()
[  242s]     
[  242s]         def never_true() -> bool:
[  242s]             return False
[  242s]     
[  242s]         with pytest.raises(WaitTimeout):
[  242s]             retry_until(never_true, 1)
[  242s]     
[  242s]         end = time()
[  242s]     
[  242s] >       assert abs((end - ini) - 1.0) < 0.01
[  242s] E       assert 0.010947227478027344 < 0.01
[  242s] E        +  where 0.010947227478027344 = abs(((1679552325.5506115 - 1679552324.5396643) - 1.0))
[  242s] 
[  242s] tests/test_test.py:41: AssertionError
[  242s] =============================== warnings summary ===============================

[tony]

@johanneskastl Have you seen other packages have this issue?

I am not quite sure how to help - but interested in finding a solution.

[johanneskastl]

@johanneskastl Have you seen other packages have this issue?

Hi Tony, none of the few python packages in openSUSE that I maintain has similar errors.

If I am not mistaken this also happen for all of the python versions in use (3.9/3.10/3.11).

[tony]

I will look into a longterm solution or fix

Short term:

One possibility is I can disable the test on openSUSE - is there a reliable marker to check for OpenSUSE machines?

Also, if OpenSUSE packages can support patches, @pytest.mark.skip above test_function_times_out should skip it.

[johanneskastl]

I will look into a longterm solution or fix

Thank you!

Short term:

One possibility is I can disable the test on openSUSE - is there a reliable marker to check for OpenSUSE machines?

Also, if OpenSUSE packages can support patches, @pytest.mark.skip above test_function_times_out should skip it.

Thanks for the offer, but there is no need for a short term solution, I already excluded the failing tests in the package build.

[johanneskastl]

I got another one today: test_new_window_with_environment[environment0]

[  116s] =================================== FAILURES ===================================
[  116s] ________________ test_new_window_with_environment[environment0] ________________
[  116s] 
[  116s] session = Session($1 libtmux_imz1fuo6), environment = {'ENV_VAR': 'window'}
[  116s] 
[  116s]     @pytest.mark.skipif(
[  116s]         has_lt_version("3.0"),
[  116s]         reason="needs -e flag for new-window which was introduced in 3.0",
[  116s]     )
[  116s]     @pytest.mark.parametrize(
[  116s]         "environment",
[  116s]         [
[  116s]             {"ENV_VAR": "window"},
[  116s]             {"ENV_VAR_1": "window_1", "ENV_VAR_2": "window_2"},
[  116s]         ],
[  116s]     )
[  116s]     def test_new_window_with_environment(
[  116s]         session: Session,
[  116s]         environment: t.Dict[str, str],
[  116s]     ) -> None:
[  116s]         env = shutil.which("env")
[  116s]         assert env is not None, "Cannot find usable `env` in PATH."
[  116s]     
[  116s]         window = session.new_window(
[  116s]             attach=True,
[  116s]             window_name="window_with_environment",
[  116s]             window_shell=f"{env} PS1='$ ' sh",
[  116s]             environment=environment,
[  116s]         )
[  116s]         pane = window.attached_pane
[  116s]         assert pane is not None
[  116s]         for k, v in environment.items():
[  116s]             pane.send_keys(f"echo ${k}")
[  116s] >           assert pane.capture_pane()[-2] == v
[  116s] E           IndexError: list index out of range
[  116s] 
[  116s] tests/legacy_api/test_session.py:291: IndexError

[matoro]

Observed this test_capture_pane_start failure on Gentoo also.

================================================================= FAILURES ==================================================================
__________________________________________________________ test_capture_pane_start __________________________________________________________

session = Session($1 libtmux_9bmi4d2z)

    def test_capture_pane_start(session: Session) -> None:
        """Assert Pane.capture_pane() with ``start`` param."""
        env = shutil.which("env")
        assert env is not None, "Cannot find usable `env` in PATH."

        session.new_window(
            attach=True,
            window_name="capture_pane_start",
            window_shell=f"{env} PS1='$ ' sh",
        )
        pane = session.active_window.active_pane
        assert pane is not None
        pane_contents = "\n".join(pane.capture_pane())
        assert pane_contents == "$"
        pane.send_keys(r'printf "%s"', literal=True, suppress_history=False)
        pane_contents = "\n".join(pane.capture_pane())
        assert pane_contents == '$ printf "%s"\n$'
        pane.send_keys("clear -x", literal=True, suppress_history=False)
        pane_contents = "\n".join(pane.capture_pane())
>       assert pane_contents == "$"
E       assert '$ printf "%s"\n$ clear -x' == '$'
E
E         - $
E         + $ printf "%s"
E         + $ clear -x

env        = '/usr/bin/env'
pane       = Pane(%2 Window(@2 2:capture_pane_start, Session($1 libtmux_9bmi4d2z)))
pane_contents = '$ printf "%s"\n$ clear -x'
session    = Session($1 libtmux_9bmi4d2z)

tests/test_pane.py:104: AssertionError