Skip to content

warnings.filterwarnings do not allow filtering warning that starts with empty line #114426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Czaki opened this issue Jan 22, 2024 · 3 comments
Open

warnings.filterwarnings do not allow filtering warning that starts with empty line #114426

Czaki opened this issue Jan 22, 2024 · 3 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@Czaki
Copy link

Czaki commented Jan 22, 2024 •
edited by github-actions bot
Loading

Bug report

Bug description:

# Add a code block here, if required

I have found that I cannot filter warnings that start with an empty line. Like this warning in pandas: https://github.com/pandas-dev/pandas/blob/488f6bd5c08111603549d42effc6dbb7fec935f0/pandas/__init__.py#L221-L231

On a project that I work (napari) we have a rule "error:::napari" in pytest that changes all warning into errors. If it is possible, then we solve warnings immediately, but some of them needs to be ignored (like linked one).

Based on documentation, I assume that rule "ignore:Pyarrow will become a required dependency of pandas in the next major release of pandas.*:DeprecationWarning", should work. But it don't.

Rule is effectively calling warnings.filterwarnings("ignore", "Pyarrow will become a required dependency of pandas in the next major release of pandas.*", DeprecationWarning).

I have found that in filterwarnings the message is changed to regexp using only re.IGNORECASE flag

cpython/Lib/warnings.py

Line 160 in d1b031c

message = re.compile(message, re.I)
without re.MULTILINE flag.

And then the match of the regexp is used

cpython/Lib/warnings.py

Line 375 in d1b031c

if ((msg is None or msg.match(text)) and
.

Based on my knowledge, the problem could be solved by either adding re.MULTILINE to compile flags or using msg.search during filtering.

Which solution is preferred? I could make a PR if someone point which one I should select.

CPython versions tested on:

3.11

Operating systems tested on:

Linux
@Czaki Czaki added the type-bug An unexpected behavior, bug, or error label Jan 22, 2024
This was referenced Jan 22, 2024
Closed
Merged
@serhiy-storchaka
Copy link
Member

Try to use
"\nPyarrow will become a required dependency of pandas"
or
"(?s:.*)Pyarrow will become a required dependency of pandas"

@Czaki
Copy link
Author

Czaki commented Jan 22, 2024

First do not work,
Second fail, because pytest uses : as separtor but I found that "ignore:(?s).*Pyarrow will become a required dependency of pandas:DeprecationWarning" works.

So maybe I should think about possible documentation update?

Czaki added a commit to napari/napari that referenced this issue Jan 22, 2024
@Czaki @psobolewskiPhD @brisvag
Ignore pandas pyarrow warning (#6609)
41dcb89 
# Description

The #6608 shows that new pandas warning crash our tests. 
```
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
      exec(co, module.__dict__)
  napari/layers/base/base.py:28: in <module>
      from napari.layers.utils.layer_utils import (
  <frozen importlib._bootstrap>:1007: in _find_and_load
      ???
  <frozen importlib._bootstrap>:986: in _find_and_load_unlocked
      ???
  <frozen importlib._bootstrap>:680: in _load_unlocked
      ???
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
      exec(co, module.__dict__)
  napari/layers/utils/layer_utils.py:10: in <module>
      import pandas as pd
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/pandas/__init__.py:221: in <module>
      warnings.warn(
  E   DeprecationWarning: 
  E   Pyarrow will become a required dependency of pandas in the next major release of pandas (pandas 3.0),
  E   (to allow more performant data types, such as the Arrow string type, and better interoperability with other libraries)
  E   but was not found to be installed on your system.
  E   If this would cause problems for you,
  E   please provide us feedback at pandas-dev/pandas#54466
```

This PR adds filter warnings to prevent this failure. Because warnings
start with an empty line, this fix is fragile and heavily depends on the
order of imports.

Pull request to fix warning message in pandas 
pandas-dev/pandas#57003

cPython issue:
python/cpython#114426

This PR also fixes usage of chained assignment.

---------

Co-authored-by: Peter Sobolewski <[email protected]>
Co-authored-by: Lorenzo Gaifas <[email protected]>
Czaki added a commit to napari/napari that referenced this issue Jan 23, 2024
@Czaki @psobolewskiPhD @brisvag
Ignore pandas pyarrow warning (#6609)
ec5798b 
# Description

The #6608 shows that new pandas warning crash our tests. 
```
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
      exec(co, module.__dict__)
  napari/layers/base/base.py:28: in <module>
      from napari.layers.utils.layer_utils import (
  <frozen importlib._bootstrap>:1007: in _find_and_load
      ???
  <frozen importlib._bootstrap>:986: in _find_and_load_unlocked
      ???
  <frozen importlib._bootstrap>:680: in _load_unlocked
      ???
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
      exec(co, module.__dict__)
  napari/layers/utils/layer_utils.py:10: in <module>
      import pandas as pd
  /tmp/.tox/py39-linux-pyqt5/lib/python3.9/site-packages/pandas/__init__.py:221: in <module>
      warnings.warn(
  E   DeprecationWarning: 
  E   Pyarrow will become a required dependency of pandas in the next major release of pandas (pandas 3.0),
  E   (to allow more performant data types, such as the Arrow string type, and better interoperability with other libraries)
  E   but was not found to be installed on your system.
  E   If this would cause problems for you,
  E   please provide us feedback at pandas-dev/pandas#54466
```

This PR adds filter warnings to prevent this failure. Because warnings
start with an empty line, this fix is fragile and heavily depends on the
order of imports.

Pull request to fix warning message in pandas 
pandas-dev/pandas#57003

cPython issue:
python/cpython#114426

This PR also fixes usage of chained assignment.

---------

Co-authored-by: Peter Sobolewski <[email protected]>
Co-authored-by: Lorenzo Gaifas <[email protected]>
@antonymilne
Copy link

The following works for me - note the \\:

\\nPyarrow will become a required dependency of pandas

For anyone coming from the pandas issue, in this case the (?s).* solution is better than \\n since it will also work when pandas-dev/pandas#57003 gets released.

Thanks for figuring all this out @Czaki - I was faced with the same difficulty and your work on it is very helpful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@serhiy-storchaka @Czaki @antonymilne