Closed
Description
SQLAlchemy CI is now broken due to the unconditional inclusion of pytest-warnings in py.test 3.1.0. I need at least to know how to disable this plugin entirely.
In our test suite we make use of the warnings filter internally in order to propagate our own warnings to errors. It seems like warnings.filter() is now non-functional when this plugin is installed.
Demo test case:
import warnings
class MyWarning(Warning):
pass
warnings.filterwarnings("error", category=MyWarning)
class TestWarnings(object):
def test_my_warning(self):
try:
warnings.warn(MyWarning("warn!"))
assert False
except MyWarning:
assert True
with py.test 3.0.7:
$ py.test test_warning.py
======================================================== test session starts ========================================================
platform linux2 -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/classic/Desktop/tmp, inifile:
plugins: xdist-1.15.0, cov-2.4.0
collected 1 items
test_warning.py .
===================================================== 1 passed in 0.01 seconds ======================================================
with py.test 3.1.0:
$ py.test test_warning.py
======================================================== test session starts ========================================================
platform linux2 -- Python 2.7.13, pytest-3.1.0, py-1.4.33, pluggy-0.4.0
rootdir: /home/classic/Desktop/tmp, inifile:
plugins: xdist-1.15.0, cov-2.4.0
collected 1 items
test_warning.py F
============================================================= FAILURES ==============================================================
___________________________________________________ TestWarnings.test_my_warning ____________________________________________________
self = <test_warning.TestWarnings object at 0x7fecb08a5290>
def test_my_warning(self):
try:
warnings.warn(MyWarning("warn!"))
> assert False
E assert False
test_warning.py:12: AssertionError
========================================================= warnings summary ==========================================================
test_warning.py::TestWarnings::()::test_my_warning
/home/classic/Desktop/tmp/test_warning.py:11: MyWarning: warn!
warnings.warn(MyWarning("warn!"))
-- Docs: http://doc.pytest.org/en/latest/warnings.html
=============================================== 1 failed, 1 warnings in 0.03 seconds ================================================
I have tried everything I can think of with the new -W flag, disable-warnings, no luck.
Please advise on the correct options to allow Python stdlib warnings.filter() to work again, thanks!
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Participants
Activity
- pin py.test at 3.0.7 due to pytest-dev/pytest#2430
- pin py.test at 3.0.7 due to pytest-dev/pytest#2430
[-]forcing pytest-warnings breaks the world[/-][+]Warning integration breaks warnings.filterwarnings[/+]zzzeek commentedon May 23, 2017
how do I get a helper to help triage my bugs and change nasty headlines to nice ones? :(
nicoddemus commentedon May 23, 2017
@zzzeek you can disable the warnings plugin by passing
-p no:warnings; this will disable the plugin completely and you should get green builds again. You can choose to add this to yourpytest.inifile:As for the problem itself, I will take a more careful look later.
zzzeek commentedon May 23, 2017
great, thanks!
- add option to disable py.test warnings plugin;
- add option to disable py.test warnings plugin;
RonnyPfannschmidt commentedon May 23, 2017
i believe https://github.com/pytest-dev/pytest/blob/master/_pytest/warnings.py#L54 is gutting all prior setups
we should just call reset_warning instead when we do something like that and give people a opt-out to allow respecting in code specifications
nicoddemus commentedon May 24, 2017
@zzzeek thanks for posting the example, it helped understand the issue.
Can I ask why you add that warnings filter? If the objective is to turn certain warnings into errors, you can now configure your
pytest.inito the same effect. Raising more awareness about warnings was the motivation for this new feature.Also, do you think we should add a blurb to the warnings docs making it easy for users to find how to disable it in case they run into trouble?
@RonnyPfannschmidt
Actually the implementation uses
catch_warnings, which saves the previous filters and restores them after the call (see https://github.com/python/cpython/blob/master/Lib/warnings.py#L454). Unfortunately this has the effect of silencing errors configured manually by otherwarnings.simplefiltercalls, which is the heart of this issue. Also, by default Python will install a number of default filters silencingDeprecationWarnings,ImportWarningsand others, which are precisely the kind of warnings we believe should be displayed in test suites.@RonnyPfannschmidt
resetwarnings()will actually throw out all filters:(https://docs.python.org/3/library/warnings.html#warnings.resetwarnings)
Which is not what we want, I think.
I'm not sure how to proceed here, so I would like to hear some opinions.
RonnyPfannschmidt commentedon May 24, 2017
@nicoddemus pushing a new once simple filter is invalidating all other filter s due to matching first
so its literally pushing a new filter to be there
i beleive that by pushing a unfiltered once for all warning setups literally breaks all expectations,
people should opt in into breaking filter changes, not have to opt out
in retrospect the warnings addition is implemented in a way that warrants a major release, and the way we should fix it warrants a major release as well because to fix this issue we need to do a breaking change, and we never adopted a process of publishing experiments that we may change our mind on
53 remaining items