| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
Issue #12086 suggests another possible benefit of removing pseudo fixtures from the --fixtures-per-test output. If a user is expecting their fixture, user_defined, to be parametrised in the following manner: @pytest.fixture
def user_defined(request):
return request.param * 3
@pytest.mark.parametrize('user_defined', [1])
def test_function(user_defined):
assert user_defined == 3This test will fail: user_defined = 1
@pytest.mark.parametrize('user_defined', [1])
def test_scenario_1(user_defined):
> assert user_defined == 3
E assert 1 == 3
file.py:9: AssertionError
Yet the current output of --fixtures-per-test does not provide a clue about what is happening. Even though the user-defined fixture has been shadowed by parametrization and is no longer in the test function's scope, it doesn't look like that has happened: --------------------------- fixtures used by test_scenario_1[1] ----------------------------
--------------------------------------- (file.py:8) ----------------------------------------
user_defined -- src/_pytest/python.py:1113
no docstring available
But if pseudo fixtures are excluded from the output, there is a stronger message about the user-defined fixtures's non-use because no fixture use will be reported. Documentation needs to clarifyFrom one point of view, one mystery has been replaced with another, so I definitely think documentation needs to be available to help clarify the matter for new users. Devil's advocateShould --fixtures-per-test still include pseudo fixtures but add a docstring that explains what they are? In my opinion, this is very explicit, which is good, and it will provide some comfort to users when they use --fixtures-per-test and have their intuitions about fixtures in pytest violated, but I suppose it still has the problem of exposing internal details. |
Sorry, something went wrong.
| if verbose <= 0 and argname.startswith("_"): | ||
| return |
There was a problem hiding this comment.
I believe this condition needs to be moved into write_item before the occurrence of the terminal writes
tw.sep("-", f"fixtures used by {item.name}")
tw.sep("-", f"({get_best_relpath(item.function)})") and adapted to filter fixturedefs of such private fixtures (assuming verbosity less than or equal to zero).
Otherwise there is a circumstance where all an item's fixtures have been excluded from the output yet the terminal still starts being written to as if fixture output is about to follow.
For example, if all of test_private_fixtures's fixtures start with _ and verbosity is zero or less, then no fixtures will be shown but fixtures used by test_private_fixtures. . . etc will be shown.
TODO:
I could be wrong about all of this. I will write up the test tonight or tomorrow and see.
Sorry, something went wrong.
There was a problem hiding this comment.
I have confirmed that in this PR private fixtures are not shown when verbosity is zero or less but the fixtures used by . . . is still printed. Is this desired behaviour?
I don't know. It is how pytest currently handles reporting on private fixtures.
import pytest
# DEFINE FIXTURES
#################
@pytest.fixture
def _private():
"""Private fixture"""
pass
@pytest.fixture
def public():
"""Public fixture."""
pass
# TESTS
#################
def test_private_fixture(_private):
pass
def test_public_fixture(public):
pass
@pytest.mark.parametrize("pseudo", [1])
def test_pseudo_fixture(pseudo):
pass
def test_no_fixture():
passRunning pytest --fixtures-per-test on the current version of pytest shows:
----------- fixtures used by test_private_fixture ------------
-------------------- (test_private.py:14) --------------------
------------ fixtures used by test_public_fixture ------------
-------------------- (test_private.py:17) --------------------
public -- test_private.py:9
Public fixture.
---------- fixtures used by test_pseudo_fixture[1] -----------
-------------------- (test_private.py:20) --------------------
pseudo -- .../_pytest/python.py:1112
no docstring available
I'm proposing it shows:
--------------------- fixtures used by test_public_fixture ----------------------
-------------------------- (test_fixture_marker.py:23) --------------------------
public -- test_fixture_marker.py:12
Public fixture.
Notable differences:
The output this PR currently produces is:
--------------------- fixtures used by test_private_fixture ---------------------
-------------------------- (test_fixture_marker.py:20) --------------------------
--------------------- fixtures used by test_public_fixture ----------------------
-------------------------- (test_fixture_marker.py:23) --------------------------
public -- test_fixture_marker.py:12
Public fixture.
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks for the PR @WarrenTheRabbit. IMO it is indeed better not to show params as fixtures in --fixtures-per-test output, for the reasons you stated, and also because they are somewhat already reflected in the test id (the part in [...]).
For the implementation you've used _get_direct_parametrize_args to exclude the params. But at this point we've already gathered this info for the item (test), which can be found in the callspec field. So it's better to use that.
However, I'd even go a step further and just exclude "pseudo" fixtures directly. This would involve adding an indication on the FixtureDef whether it's a "pseudo" fixture (we really ought to find a better name for this...). As long as it's only used for "information" purposes such as --fixtures-per-test and not for the actual business logic of fixtures, I have no objection to adding such an indication. Then, the change could be to just skip over such FixtureDefs.
Sorry, something went wrong.
…zation helper fixtures No functional changes. This is intended for next commit (wants to skip showing these fixtures in `--fixtures-per-test`). These fixtures were previously called "pseudo fixtures". I took the opportunity to rename them "direct param fixture def", which, while less catchy, is more self-descriptive and less judgmental.
There was a problem hiding this comment.
So should be good to go now.
Sorry, something went wrong.
Fix pytest-dev#11295 by excluding from the --fixtures-per-test output any 'pseudo fixture' that results from directy parametrizing a test e.g. with ``@pytest.mark.parametrize``. The justification for removing these fixtures from the report is that a) They are unintuitive. Their appearance in the fixtures-per-test report confuses new users because the fixtures created via ``@pytest.mark.parametrize`` do not confrom to the expectations established in the documentation; namely, that fixtures are - richly reusable - provide setup/teardown features - created via the ``@pytest.fixture` decorator b) They are an internal implementation detail. It is not the explicit goal of the direct parametrization mark to create a fixture; instead, pytest's internals leverages the fixture system to achieve the explicit goal: a succinct batch execution syntax. Consequently, exposing the fixtures that implement the batch execution behaviour reveal more about pytest's internals than they do about the user's own design choices and test dependencies.
| Back | FazBrowse Home | New Git URL |
Closes #11295 by excluding 'pseudo fixtures' from the --fixtures-per-test output.
For example, when pytest --fixtures-per-test is run, this test
will no longer produce output such as this:
------------------------- fixtures used by test_one[1] ------------------------- ----------------------------------- (test_file.py:5) ------------------------------ monkeypatch -- src/_pytest/monkeypatch.py:33 A convenient fixture for monkey-patching. x -- src/_pytest/python.py:1113 no docstring available ------------------------- fixtures used by test_one[2] ------------------------- ----------------------------------- (test_file.py:5) ------------------------------ monkeypatch -- src/_pytest/monkeypatch.py:33 A convenient fixture for monkey-patching. x -- src/_pytest/python.py:1113 no docstring availableInstead, it will be:
------------------------- fixtures used by test_one[1] ------------------------- ----------------------------------- (test_file.py:5) ------------------------------ monkeypatch -- src/_pytest/monkeypatch.py:33 A convenient fixture for monkey-patching. ------------------------- fixtures used by test_one[2] ------------------------- ----------------------------------- (test_file.py:5) ------------------------------ monkeypatch -- src/_pytest/monkeypatch.py:33 A convenient fixture for monkey-patching.Justification for changing the output
The original output did not match with new users' intuitions and expectations
As a new user, I found it unintuitive to see the @pytest.mark.parametrize variables appear in my --fixtures-per-test report. I am of the opinion that the inclusion of pseudo fixtures in the output confuses new users because they do not conform to the expectations established in the documentation. Namely, that fixtures are
The original output puts attention on internal implementation details
The purpose of --fixtures-per-test is to create a summary of the user's fixture decisions and dependencies. Yet creating a fixture for the user is not the goal of the direct parametrization mark. Instead, _pytest'_s internals just leverage the fixture system to achieve the actual goal: a succinct batch execution syntax. I believe that including the pseudo fixtures in the output exposes internal implementation details unnecessarily and distracts from the user-side summary.
Checklist