See also
About fixtures
See also
Fixtures reference
“Requesting” fixtures¶
At a basic level, test functions request fixtures they require by declaringthem as arguments.
When pytest goes to run a test, it looks at the parameters in that testfunction’s signature, and then searches for fixtures that have the same names asthose parameters. Once pytest finds them, it runs those fixtures, captures whatthey returned (if anything), and passes those objects into the test function asarguments.
Quick example¶
import pytestclass Fruit: def __init__(self, name): self.name = name self.cubed = False def cube(self): self.cubed = Trueclass FruitSalad: def __init__(self, *fruit_bowl): self.fruit = fruit_bowl self._cube_fruit() def _cube_fruit(self): for fruit in self.fruit: fruit.cube()# Arrange@pytest.fixturedef fruit_bowl(): return [Fruit("apple"), Fruit("banana")]def test_fruit_salad(fruit_bowl): # Act fruit_salad = FruitSalad(*fruit_bowl) # Assert assert all(fruit.cubed for fruit in fruit_salad.fruit)
In this example, test_fruit_salad “requests” fruit_bowl (i.e.def test_fruit_salad(fruit_bowl):), and when pytest sees this, it willexecute the fruit_bowl fixture function and pass the object it returns intotest_fruit_salad as the fruit_bowl argument.
Here’s roughlywhat’s happening if we were to do it by hand:
def fruit_bowl(): return [Fruit("apple"), Fruit("banana")]def test_fruit_salad(fruit_bowl): # Act fruit_salad = FruitSalad(*fruit_bowl) # Assert assert all(fruit.cubed for fruit in fruit_salad.fruit)# Arrangebowl = fruit_bowl()test_fruit_salad(fruit_bowl=bowl)
Fixtures can request other fixtures¶
One of pytest’s greatest strengths is its extremely flexible fixture system. Itallows us to boil down complex requirements for tests into more simple andorganized functions, where we only need to have each one describe the thingsthey are dependent on. We’ll get more into this further down, but for now,here’s a quick example to demonstrate how fixtures can use other fixtures:
# contents of test_append.pyimport pytest# Arrange@pytest.fixturedef first_entry(): return "a"# Arrange@pytest.fixturedef order(first_entry): return [first_entry]def test_string(order): # Act order.append("b") # Assert assert order == ["a", "b"]
Notice that this is the same example from above, but very little changed. Thefixtures in pytest request fixtures just like tests. All the samerequesting rules apply to fixtures that do for tests. Here’s how thisexample would work if we did it by hand:
def first_entry(): return "a"def order(first_entry): return [first_entry]def test_string(order): # Act order.append("b") # Assert assert order == ["a", "b"]entry = first_entry()the_list = order(first_entry=entry)test_string(order=the_list)
Fixtures are reusable¶
One of the things that makes pytest’s fixture system so powerful, is that itgives us the ability to define a generic setup step that can be reused over andover, just like a normal function would be used. Two different tests can requestthe same fixture and have pytest give each test their own result from thatfixture.
This is extremely useful for making sure tests aren’t affected by each other. Wecan use this system to make sure each test gets its own fresh batch of data andis starting from a clean state so it can provide consistent, repeatable results.
Here’s an example of how this can come in handy:
# contents of test_append.pyimport pytest# Arrange@pytest.fixturedef first_entry(): return "a"# Arrange@pytest.fixturedef order(first_entry): return [first_entry]def test_string(order): # Act order.append("b") # Assert assert order == ["a", "b"]def test_int(order): # Act order.append(2) # Assert assert order == ["a", 2]
Each test here is being given its own copy of that list object,which means the order fixture is getting executed twice (the sameis true for the first_entry fixture). If we were to do this by hand aswell, it would look something like this:
def first_entry(): return "a"def order(first_entry): return [first_entry]def test_string(order): # Act order.append("b") # Assert assert order == ["a", "b"]def test_int(order): # Act order.append(2) # Assert assert order == ["a", 2]entry = first_entry()the_list = order(first_entry=entry)test_string(order=the_list)entry = first_entry()the_list = order(first_entry=entry)test_int(order=the_list)
A test/fixture can request more than one fixture at a time¶
Tests and fixtures aren’t limited to requesting a single fixture at a time.They can request as many as they like. Here’s another quick example todemonstrate:
# contents of test_append.pyimport pytest# Arrange@pytest.fixturedef first_entry(): return "a"# Arrange@pytest.fixturedef second_entry(): return 2# Arrange@pytest.fixturedef order(first_entry, second_entry): return [first_entry, second_entry]# Arrange@pytest.fixturedef expected_list(): return ["a", 2, 3.0]def test_string(order, expected_list): # Act order.append(3.0) # Assert assert order == expected_list
Fixtures can be requested more than once per test (return values are cached)¶
Fixtures can also be requested more than once during the same test, andpytest won’t execute them again for that test. This means we can requestfixtures in multiple fixtures that are dependent on them (and even again in thetest itself) without those fixtures being executed more than once.
# contents of test_append.pyimport pytest# Arrange@pytest.fixturedef first_entry(): return "a"# Arrange@pytest.fixturedef order(): return []# Act@pytest.fixturedef append_first(order, first_entry): return order.append(first_entry)def test_string_only(append_first, order, first_entry): # Assert assert order == [first_entry]
If a requested fixture was executed once for every time it was requestedduring a test, then this test would fail because both append_first andtest_string_only would see order as an empty list (i.e. []), butsince the return value of order was cached (along with any side effectsexecuting it may have had) after the first time it was called, both the test andappend_first were referencing the same object, and the test saw the effectappend_first had on that object.
Autouse fixtures (fixtures you don’t have to request)¶
Sometimes you may want to have a fixture (or even several) that you know allyour tests will depend on. “Autouse” fixtures are a convenient way to make alltests automatically request them. This can cut out alot of redundant requests, and can even provide more advanced fixture usage(more on that further down).
We can make a fixture an autouse fixture by passing in autouse=True to thefixture’s decorator. Here’s a simple example for how they can be used:
# contents of test_append.pyimport pytest@pytest.fixturedef first_entry(): return "a"@pytest.fixturedef order(first_entry): return []@pytest.fixture(autouse=True)def append_first(order, first_entry): return order.append(first_entry)def test_string_only(order, first_entry): assert order == [first_entry]def test_string_and_int(order, first_entry): order.append(2) assert order == [first_entry, 2]
In this example, the append_first fixture is an autouse fixture. Because ithappens automatically, both tests are affected by it, even though neither testrequested it. That doesn’t mean they can’t be requested though; justthat it isn’t necessary.
Scope: sharing fixtures across classes, modules, packages or session¶
Fixtures requiring network access depend on connectivity and areusually time-expensive to create. Extending the previous example, wecan add a scope="module" parameter to the@pytest.fixture invocationto cause a smtp_connection fixture function, responsible to create a connection to a preexisting SMTP server, to only be invokedonce per test module (the default is to invoke once per test function).Multiple test functions in a test module will thuseach receive the same smtp_connection fixture instance, thus saving time.Possible values for scope are: function, class, module, package or session.
The next example puts the fixture function into a separate conftest.py fileso that tests from multiple test modules in the directory canaccess the fixture function:
# content of conftest.pyimport smtplibimport pytest@pytest.fixture(scope="module")def smtp_connection(): return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
# content of test_module.pydef test_ehlo(smtp_connection): response, msg = smtp_connection.ehlo() assert response == 250 assert b"smtp.gmail.com" in msg assert 0 # for demo purposesdef test_noop(smtp_connection): response, msg = smtp_connection.noop() assert response == 250 assert 0 # for demo purposes
Here, the test_ehlo needs the smtp_connection fixture value. pytestwill discover and call the @pytest.fixturemarked smtp_connection fixture function. Running the test looks like this:
$ pytest test_module.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.yrootdir: /home/sweet/projectcollected 2 itemstest_module.py FF [100%]================================= FAILURES =================================________________________________ test_ehlo _________________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0001> def test_ehlo(smtp_connection): response, msg = smtp_connection.ehlo() assert response == 250 assert b"smtp.gmail.com" in msg> assert 0 # for demo purposesE assert 0test_module.py:7: AssertionError________________________________ test_noop _________________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0001> def test_noop(smtp_connection): response, msg = smtp_connection.noop() assert response == 250> assert 0 # for demo purposesE assert 0test_module.py:13: AssertionError========================= short test summary info ==========================FAILED test_module.py::test_ehlo - assert 0FAILED test_module.py::test_noop - assert 0============================ 2 failed in 0.12s =============================
You see the two assert 0 failing and more importantly you can also seethat the exactly same smtp_connection object was passed into thetwo test functions because pytest shows the incoming argument values in thetraceback. As a result, the two test functions using smtp_connection runas quick as a single one because they reuse the same instance.
If you decide that you rather want to have a session-scoped smtp_connectioninstance, you can simply declare it:
@pytest.fixture(scope="session")def smtp_connection(): # the returned fixture value will be shared for # all tests requesting it ...
Fixture scopes¶
Fixtures are created when first requested by a test, and are destroyed based on their scope:
function: the default scope, the fixture is destroyed at the end of the test.class: the fixture is destroyed during teardown of the last test in the class.module: the fixture is destroyed during teardown of the last test in the module.package: the fixture is destroyed during teardown of the last test in the package where the fixture is defined, including sub-packages and sub-directories within it.session: the fixture is destroyed at the end of the test session.
Note
Pytest only caches one instance of a fixture at a time, whichmeans that when using a parametrized fixture, pytest may invoke a fixture more than once inthe given scope.
Dynamic scope¶
Added in version 5.2.
In some cases, you might want to change the scope of the fixture without changing the code.To do that, pass a callable to scope. The callable must return a string with a valid scopeand will be executed only once - during the fixture definition. It will be called with twokeyword arguments - fixture_name as a string and config with a configuration object.
This can be especially useful when dealing with fixtures that need time for setup, like spawninga docker container. You can use the command-line argument to control the scope of the spawnedcontainers for different environments. See the example below.
def determine_scope(fixture_name, config): if config.getoption("--keep-containers", None): return "session" return "function"@pytest.fixture(scope=determine_scope)def docker_container(): yield spawn_container()
Teardown/Cleanup (AKA Fixture finalization)¶
When we run our tests, we’ll want to make sure they clean up after themselves sothey don’t mess with any other tests (and also so that we don’t leave behind amountain of test data to bloat the system). Fixtures in pytest offer a veryuseful teardown system, which allows us to define the specific steps necessaryfor each fixture to clean up after itself.
This system can be leveraged in two ways.
1. yield fixtures (recommended)¶
“Yield” fixtures yield instead of return. With thesefixtures, we can run some code and pass an object back to the requestingfixture/test, just like with the other fixtures. The only differences are:
returnis swapped out foryield.Any teardown code for that fixture is placed after the
yield.
Once pytest figures out a linear order for the fixtures, it will run each one upuntil it returns or yields, and then move on to the next fixture in the list todo the same thing.
Once the test is finished, pytest will go back down the list of fixtures, but inthe reverse order, taking each one that yielded, and running the code insideit that was after the yield statement.
As a simple example, consider this basic email module:
# content of emaillib.pyclass MailAdminClient: def create_user(self): return MailUser() def delete_user(self, user): # do some cleanup passclass MailUser: def __init__(self): self.inbox = [] def send_email(self, email, other): other.inbox.append(email) def clear_mailbox(self): self.inbox.clear()class Email: def __init__(self, subject, body): self.subject = subject self.body = body
Let’s say we want to test sending email from one user to another. We’ll have tofirst make each user, then send the email from one user to the other, andfinally assert that the other user received that message in their inbox. If wewant to clean up after the test runs, we’ll likely have to make sure the otheruser’s mailbox is emptied before deleting that user, otherwise the system maycomplain.
Here’s what that might look like:
# content of test_emaillib.pyfrom emaillib import Email, MailAdminClientimport pytest@pytest.fixturedef mail_admin(): return MailAdminClient()@pytest.fixturedef sending_user(mail_admin): user = mail_admin.create_user() yield user mail_admin.delete_user(user)@pytest.fixturedef receiving_user(mail_admin): user = mail_admin.create_user() yield user user.clear_mailbox() mail_admin.delete_user(user)def test_email_received(sending_user, receiving_user): email = Email(subject="Hey!", body="How's it going?") sending_user.send_email(email, receiving_user) assert email in receiving_user.inbox
Because receiving_user is the last fixture to run during setup, it’s the first to runduring teardown.
There is a risk that even having the order right on the teardown side of thingsdoesn’t guarantee a safe cleanup. That’s covered in a bit more detail inSafe teardowns.
$ pytest -q test_emaillib.py. [100%]1 passed in 0.12s
Handling errors for yield fixture¶
If a yield fixture raises an exception before yielding, pytest won’t try to runthe teardown code after that yield fixture’s yield statement. But, for everyfixture that has already run successfully for that test, pytest will stillattempt to tear them down as it normally would.
2. Adding finalizers directly¶
While yield fixtures are considered to be the cleaner and more straightforwardoption, there is another choice, and that is to add “finalizer” functionsdirectly to the test’s request-context object. It brings a similar result asyield fixtures, but requires a bit more verbosity.
In order to use this approach, we have to request the request-context object(just like we would request another fixture) in the fixture we need to addteardown code for, and then pass a callable, containing that teardown code, toits addfinalizer method.
We have to be careful though, because pytest will run that finalizer once it’sbeen added, even if that fixture raises an exception after adding the finalizer.So to make sure we don’t run the finalizer code when we wouldn’t need to, wewould only add the finalizer once the fixture would have done something thatwe’d need to teardown.
Here’s how the previous example would look using the addfinalizer method:
# content of test_emaillib.pyfrom emaillib import Email, MailAdminClientimport pytest@pytest.fixturedef mail_admin(): return MailAdminClient()@pytest.fixturedef sending_user(mail_admin): user = mail_admin.create_user() yield user mail_admin.delete_user(user)@pytest.fixturedef receiving_user(mail_admin, request): user = mail_admin.create_user() def delete_user(): mail_admin.delete_user(user) request.addfinalizer(delete_user) return user@pytest.fixturedef email(sending_user, receiving_user, request): _email = Email(subject="Hey!", body="How's it going?") sending_user.send_email(_email, receiving_user) def empty_mailbox(): receiving_user.clear_mailbox() request.addfinalizer(empty_mailbox) return _emaildef test_email_received(receiving_user, email): assert email in receiving_user.inbox
It’s a bit longer than yield fixtures and a bit more complex, but itdoes offer some nuances for when you’re in a pinch.
$ pytest -q test_emaillib.py. [100%]1 passed in 0.12s
Note on finalizer order¶
Finalizers are executed in a first-in-last-out order.For yield fixtures, the first teardown code to run is from the right-most fixture, i.e. the last test parameter.
# content of test_finalizers.pyimport pytestdef test_bar(fix_w_yield1, fix_w_yield2): print("test_bar")@pytest.fixturedef fix_w_yield1(): yield print("after_yield_1")@pytest.fixturedef fix_w_yield2(): yield print("after_yield_2")
$ pytest -s test_finalizers.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.yrootdir: /home/sweet/projectcollected 1 itemtest_finalizers.py test_bar.after_yield_2after_yield_1============================ 1 passed in 0.12s =============================
For finalizers, the first fixture to run is last call to request.addfinalizer.
# content of test_finalizers.pyfrom functools import partialimport pytest@pytest.fixturedef fix_w_finalizers(request): request.addfinalizer(partial(print, "finalizer_2")) request.addfinalizer(partial(print, "finalizer_1"))def test_bar(fix_w_finalizers): print("test_bar")
$ pytest -s test_finalizers.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.yrootdir: /home/sweet/projectcollected 1 itemtest_finalizers.py test_bar.finalizer_1finalizer_2============================ 1 passed in 0.12s =============================
This is so because yield fixtures use addfinalizer behind the scenes: when the fixture executes, addfinalizer registers a function that resumes the generator, which in turn calls the teardown code.
Safe teardowns¶
The fixture system of pytest is very powerful, but it’s still being run by acomputer, so it isn’t able to figure out how to safely teardown everything wethrow at it. If we aren’t careful, an error in the wrong spot might leave stufffrom our tests behind, and that can cause further issues pretty quickly.
For example, consider the following tests (based off of the mail example fromabove):
# content of test_emaillib.pyfrom emaillib import Email, MailAdminClientimport pytest@pytest.fixturedef setup(): mail_admin = MailAdminClient() sending_user = mail_admin.create_user() receiving_user = mail_admin.create_user() email = Email(subject="Hey!", body="How's it going?") sending_user.send_email(email, receiving_user) yield receiving_user, email receiving_user.clear_mailbox() mail_admin.delete_user(sending_user) mail_admin.delete_user(receiving_user)def test_email_received(setup): receiving_user, email = setup assert email in receiving_user.inbox
This version is a lot more compact, but it’s also harder to read, doesn’t have avery descriptive fixture name, and none of the fixtures can be reused easily.
There’s also a more serious issue, which is that if any of those steps in thesetup raise an exception, none of the teardown code will run.
One option might be to go with the addfinalizer method instead of yieldfixtures, but that might get pretty complex and difficult to maintain (and itwouldn’t be compact anymore).
$ pytest -q test_emaillib.py. [100%]1 passed in 0.12s
Safe fixture structure¶
The safest and simplest fixture structure requires limiting fixtures to onlymaking one state-changing action each, and then bundling them together withtheir teardown code, as the email examples above showed.
The chance that a state-changing operation can fail but still modify state isnegligible, as most of these operations tend to be transaction-based (at least at thelevel of testing where state could be left behind). So if we make sure that anysuccessful state-changing action gets torn down by moving it to a separatefixture function and separating it from other, potentially failingstate-changing actions, then our tests will stand the best chance at leavingthe test environment the way they found it.
For an example, let’s say we have a website with a login page, and we haveaccess to an admin API where we can generate users. For our test, we want to:
Create a user through that admin API
Launch a browser using Selenium
Go to the login page of our site
Log in as the user we created
Assert that their name is in the header of the landing page
We wouldn’t want to leave that user in the system, nor would we want to leavethat browser session running, so we’ll want to make sure the fixtures thatcreate those things clean up after themselves.
Here’s what that might look like:
Note
For this example, certain fixtures (i.e. base_url andadmin_credentials) are implied to exist elsewhere. So for now, let’sassume they exist, and we’re just not looking at them.
from uuid import uuid4from urllib.parse import urljoinfrom selenium.webdriver import Chromeimport pytestfrom src.utils.pages import LoginPage, LandingPagefrom src.utils import AdminApiClientfrom src.utils.data_types import User@pytest.fixturedef admin_client(base_url, admin_credentials): return AdminApiClient(base_url, **admin_credentials)@pytest.fixturedef user(admin_client): _user = User(name="Susan", username=f"testuser-{uuid4()}", password="P4$$word") admin_client.create_user(_user) yield _user admin_client.delete_user(_user)@pytest.fixturedef driver(): _driver = Chrome() yield _driver _driver.quit()@pytest.fixturedef login(driver, base_url, user): driver.get(urljoin(base_url, "/login")) page = LoginPage(driver) page.login(user)@pytest.fixturedef landing_page(driver, login): return LandingPage(driver)def test_name_on_landing_page_after_login(landing_page, user): assert landing_page.header == f"Welcome, {user.name}!"
The way the dependencies are laid out means it’s unclear if the userfixture would execute before the driver fixture. But that’s ok, becausethose are atomic operations, and so it doesn’t matter which one runs firstbecause the sequence of events for the test is still linearizable. But what does matter isthat, no matter which one runs first, if the one raises an exception while theother would not have, neither will have left anything behind. If driverexecutes before user, and user raises an exception, the driver willstill quit, and the user was never made. And if driver was the one to raisethe exception, then the driver would never have been started and the user wouldnever have been made.
Running multiple assert statements safely¶
Sometimes you may want to run multiple asserts after doing all that setup, whichmakes sense as, in more complex systems, a single action can kick off multiplebehaviors. pytest has a convenient way of handling this and it combines a bunchof what we’ve gone over so far.
All that’s needed is stepping up to a larger scope, then having the actstep defined as an autouse fixture, and finally, making sure all the fixturesare targeting that higher level scope.
Let’s pull an example from above, and tweak it abit. Let’s say that in addition to checking for a welcome message in the header,we also want to check for a sign out button, and a link to the user’s profile.
Let’s take a look at how we can structure that so we can run multiple assertswithout having to repeat all those steps again.
Note
For this example, certain fixtures (i.e. base_url andadmin_credentials) are implied to exist elsewhere. So for now, let’sassume they exist, and we’re just not looking at them.
# contents of tests/end_to_end/test_login.pyfrom uuid import uuid4from urllib.parse import urljoinfrom selenium.webdriver import Chromeimport pytestfrom src.utils.pages import LoginPage, LandingPagefrom src.utils import AdminApiClientfrom src.utils.data_types import User@pytest.fixture(scope="class")def admin_client(base_url, admin_credentials): return AdminApiClient(base_url, **admin_credentials)@pytest.fixture(scope="class")def user(admin_client): _user = User(name="Susan", username=f"testuser-{uuid4()}", password="P4$$word") admin_client.create_user(_user) yield _user admin_client.delete_user(_user)@pytest.fixture(scope="class")def driver(): _driver = Chrome() yield _driver _driver.quit()@pytest.fixture(scope="class")def landing_page(driver, login): return LandingPage(driver)class TestLandingPageSuccess: @pytest.fixture(scope="class", autouse=True) def login(self, driver, base_url, user): driver.get(urljoin(base_url, "/login")) page = LoginPage(driver) page.login(user) def test_name_in_header(self, landing_page, user): assert landing_page.header == f"Welcome, {user.name}!" def test_sign_out_button(self, landing_page): assert landing_page.sign_out_button.is_displayed() def test_profile_link(self, landing_page, user): profile_href = urljoin(base_url, f"/profile?id={user.profile_id}") assert landing_page.profile_link.get_attribute("href") == profile_href
Notice that the methods are only referencing self in the signature as aformality. No state is tied to the actual test class as it might be in theunittest.TestCase framework. Everything is managed by the pytest fixturesystem.
Each method only has to request the fixtures that it actually needs withoutworrying about order. This is because the act fixture is an autouse fixture,and it made sure all the other fixtures executed before it. There’s no morechanges of state that need to take place, so the tests are free to make as manynon-state-changing queries as they want without risking stepping on the toes ofthe other tests.
The login fixture is defined inside the class as well, because not every oneof the other tests in the module will be expecting a successful login, and the act may need tobe handled a little differently for another test class. For example, if wewanted to write another test scenario around submitting bad credentials, wecould handle it by adding something like this to the test file:
class TestLandingPageBadCredentials: @pytest.fixture(scope="class") def faux_user(self, user): _user = deepcopy(user) _user.password = "badpass" return _user def test_raises_bad_credentials_exception(self, login_page, faux_user): with pytest.raises(BadCredentialsException): login_page.login(faux_user)
Fixtures can introspect the requesting test context¶
Fixture functions can accept the request objectto introspect the “requesting” test function, class or module context.Further extending the previous smtp_connection fixture example, let’sread an optional server URL from the test module which uses our fixture:
# content of conftest.pyimport smtplibimport pytest@pytest.fixture(scope="module")def smtp_connection(request): server = getattr(request.module, "smtpserver", "smtp.gmail.com") smtp_connection = smtplib.SMTP(server, 587, timeout=5) yield smtp_connection print(f"finalizing {smtp_connection} ({server})") smtp_connection.close()
We use the request.module attribute to optionally obtain ansmtpserver attribute from the test module. If we just executeagain, nothing much has changed:
$ pytest -s -q --tb=no test_module.pyFFfinalizing <smtplib.SMTP object at 0xdeadbeef0002> (smtp.gmail.com)========================= short test summary info ==========================FAILED test_module.py::test_ehlo - assert 0FAILED test_module.py::test_noop - assert 02 failed in 0.12s
Let’s quickly create another test module that actually sets theserver URL in its module namespace:
# content of test_anothersmtp.pysmtpserver = "mail.python.org" # will be read by smtp fixturedef test_showhelo(smtp_connection): assert 0, smtp_connection.helo()
Running it:
$ pytest -qq --tb=short test_anothersmtp.pyF [100%]================================= FAILURES =================================______________________________ test_showhelo _______________________________test_anothersmtp.py:6: in test_showhelo assert 0, smtp_connection.helo()E AssertionError: (250, b'mail.python.org')E assert 0------------------------- Captured stdout teardown -------------------------finalizing <smtplib.SMTP object at 0xdeadbeef0003> (mail.python.org)========================= short test summary info ==========================FAILED test_anothersmtp.py::test_showhelo - AssertionError: (250, b'mail....
voila! The smtp_connection fixture function picked up our mail server namefrom the module namespace.
Using markers to pass data to fixtures¶
Using the request object, a fixture can also accessmarkers which are applied to a test function. This can be useful to pass datainto a fixture from a test:
import pytest@pytest.fixturedef fixt(request): marker = request.node.get_closest_marker("fixt_data") if marker is None: # Handle missing marker in some way... data = None else: data = marker.args[0] # Do something with the data return data@pytest.mark.fixt_data(42)def test_fixt(fixt): assert fixt == 42
Factories as fixtures¶
The “factory as fixture” pattern can help in situations where the resultof a fixture is needed multiple times in a single test. Instead of returningdata directly, the fixture instead returns a function which generates the data.This function can then be called multiple times in the test.
Factories can have parameters as needed:
@pytest.fixturedef make_customer_record(): def _make_customer_record(name): return {"name": name, "orders": []} return _make_customer_recorddef test_customer_records(make_customer_record): customer_1 = make_customer_record("Lisa") customer_2 = make_customer_record("Mike") customer_3 = make_customer_record("Meredith")
If the data created by the factory requires managing, the fixture can take care of that:
@pytest.fixturedef make_customer_record(): created_records = [] def _make_customer_record(name): record = models.Customer(name=name, orders=[]) created_records.append(record) return record yield _make_customer_record for record in created_records: record.destroy()def test_customer_records(make_customer_record): customer_1 = make_customer_record("Lisa") customer_2 = make_customer_record("Mike") customer_3 = make_customer_record("Meredith")
Parametrizing fixtures¶
Fixture functions can be parametrized in which case they will be calledmultiple times, each time executing the set of dependent tests, i.e. thetests that depend on this fixture. Test functions usually do not needto be aware of their re-running. Fixture parametrization helps towrite exhaustive functional tests for components which themselves can beconfigured in multiple ways.
Extending the previous example, we can flag the fixture to create twosmtp_connection fixture instances which will cause all tests using the fixtureto run twice. The fixture function gets access to each parameterthrough the special request object:
# content of conftest.pyimport smtplibimport pytest@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])def smtp_connection(request): smtp_connection = smtplib.SMTP(request.param, 587, timeout=5) yield smtp_connection print(f"finalizing {smtp_connection}") smtp_connection.close()
The main change is the declaration of params with@pytest.fixture, a list of valuesfor each of which the fixture function will execute and can accessa value via request.param. No test function code needs to change.So let’s just do another run:
$ pytest -q test_module.pyFFFF [100%]================================= FAILURES =================================________________________ test_ehlo[smtp.gmail.com] _________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0004> def test_ehlo(smtp_connection): response, msg = smtp_connection.ehlo() assert response == 250 assert b"smtp.gmail.com" in msg> assert 0 # for demo purposesE assert 0test_module.py:7: AssertionError________________________ test_noop[smtp.gmail.com] _________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0004> def test_noop(smtp_connection): response, msg = smtp_connection.noop() assert response == 250> assert 0 # for demo purposesE assert 0test_module.py:13: AssertionError________________________ test_ehlo[mail.python.org] ________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0005> def test_ehlo(smtp_connection): response, msg = smtp_connection.ehlo() assert response == 250> assert b"smtp.gmail.com" in msgE AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8\nCHUNKING'test_module.py:6: AssertionError-------------------------- Captured stdout setup ---------------------------finalizing <smtplib.SMTP object at 0xdeadbeef0004>________________________ test_noop[mail.python.org] ________________________smtp_connection = <smtplib.SMTP object at 0xdeadbeef0005> def test_noop(smtp_connection): response, msg = smtp_connection.noop() assert response == 250> assert 0 # for demo purposesE assert 0test_module.py:13: AssertionError------------------------- Captured stdout teardown -------------------------finalizing <smtplib.SMTP object at 0xdeadbeef0005>========================= short test summary info ==========================FAILED test_module.py::test_ehlo[smtp.gmail.com] - assert 0FAILED test_module.py::test_noop[smtp.gmail.com] - assert 0FAILED test_module.py::test_ehlo[mail.python.org] - AssertionError: asser...FAILED test_module.py::test_noop[mail.python.org] - assert 04 failed in 0.12s
We see that our two test functions each ran twice, against the differentsmtp_connection instances. Note also, that with the mail.python.orgconnection the second test fails in test_ehlo because adifferent server string is expected than what arrived.
pytest will build a string that is the test ID for each fixture valuein a parametrized fixture, e.g. test_ehlo[smtp.gmail.com] andtest_ehlo[mail.python.org] in the above examples. These IDs canbe used with -k to select specific cases to run, and they willalso identify the specific case when one is failing. Running pytestwith --collect-only will show the generated IDs.
Numbers, strings, booleans and None will have their usual stringrepresentation used in the test ID. For other objects, pytest willmake a string based on the argument name. It is possible to customisethe string used in a test ID for a certain fixture value by using theids keyword argument:
# content of test_ids.pyimport pytest@pytest.fixture(params=[0, 1], ids=["spam", "ham"])def a(request): return request.paramdef test_a(a): passdef idfn(fixture_value): if fixture_value == 0: return "eggs" else: return None@pytest.fixture(params=[0, 1], ids=idfn)def b(request): return request.paramdef test_b(b): pass
The above shows how ids can be either a list of strings to use ora function which will be called with the fixture value and thenhas to return a string to use. In the latter case if the functionreturns None then pytest’s auto-generated ID will be used.
Running the above tests results in the following test IDs being used:
$ pytest --collect-only=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.yrootdir: /home/sweet/projectcollected 12 items<Dir fixtures.rst-215> <Module test_anothersmtp.py> <Function test_showhelo[smtp.gmail.com]> <Function test_showhelo[mail.python.org]> <Module test_emaillib.py> <Function test_email_received> <Module test_finalizers.py> <Function test_bar> <Module test_ids.py> <Function test_a[spam]> <Function test_a[ham]> <Function test_b[eggs]> <Function test_b[1]> <Module test_module.py> <Function test_ehlo[smtp.gmail.com]> <Function test_noop[smtp.gmail.com]> <Function test_ehlo[mail.python.org]> <Function test_noop[mail.python.org]>======================= 12 tests collected in 0.12s ========================Using marks with parametrized fixtures¶
pytest.param() can be used to apply marks in values sets of parametrized fixtures in the same waythat they can be used with @pytest.mark.parametrize.
Example:
# content of test_fixture_marks.pyimport pytest@pytest.fixture(params=[0, 1, pytest.param(2, marks=pytest.mark.skip)])def data_set(request): return request.paramdef test_data(data_set): pass
Running this test will skip the invocation of data_set with value 2:
$ pytest test_fixture_marks.py -v=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/pythoncachedir: .pytest_cacherootdir: /home/sweet/projectcollecting ... collected 3 itemstest_fixture_marks.py::test_data[0] PASSED [ 33%]test_fixture_marks.py::test_data[1] PASSED [ 66%]test_fixture_marks.py::test_data[2] SKIPPED (unconditional skip) [100%]======================= 2 passed, 1 skipped in 0.12s =======================
Modularity: using fixtures from a fixture function¶
In addition to using fixtures in test functions, fixture functionscan use other fixtures themselves. This contributes to a modular designof your fixtures and allows re-use of framework-specific fixtures acrossmany projects. As a simple example, we can extend the previous exampleand instantiate an object app where we stick the already definedsmtp_connection resource into it:
# content of test_appsetup.pyimport pytestclass App: def __init__(self, smtp_connection): self.smtp_connection = smtp_connection@pytest.fixture(scope="module")def app(smtp_connection): return App(smtp_connection)def test_smtp_connection_exists(app): assert app.smtp_connection
Here we declare an app fixture which receives the previously definedsmtp_connection fixture and instantiates an App object with it. Let’s run it:
$ pytest -v test_appsetup.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/pythoncachedir: .pytest_cacherootdir: /home/sweet/projectcollecting ... collected 2 itemstest_appsetup.py::test_smtp_connection_exists[smtp.gmail.com] PASSED [ 50%]test_appsetup.py::test_smtp_connection_exists[mail.python.org] PASSED [100%]============================ 2 passed in 0.12s =============================
Due to the parametrization of smtp_connection, the test will run twice with twodifferent App instances and respective smtp servers. There is noneed for the app fixture to be aware of the smtp_connectionparametrization because pytest will fully analyse the fixture dependency graph.
Note that the app fixture has a scope of module and uses amodule-scoped smtp_connection fixture. The example would still work ifsmtp_connection was cached on a session scope: it is fine for fixtures to use“broader” scoped fixtures but not the other way round:A session-scoped fixture could not use a module-scoped one in ameaningful way.
Automatic grouping of tests by fixture instances¶
pytest minimizes the number of active fixtures during test runs.If you have a parametrized fixture, then all the tests using it willfirst execute with one instance and then finalizers are calledbefore the next fixture instance is created. Among other things,this eases testing of applications which create and use global state.
The following example uses two parametrized fixtures, one of which isscoped on a per-module basis, and all the functions perform print callsto show the setup/teardown flow:
# content of test_module.pyimport pytest@pytest.fixture(scope="module", params=["mod1", "mod2"])def modarg(request): param = request.param print(" SETUP modarg", param) yield param print(" TEARDOWN modarg", param)@pytest.fixture(scope="function", params=[1, 2])def otherarg(request): param = request.param print(" SETUP otherarg", param) yield param print(" TEARDOWN otherarg", param)def test_0(otherarg): print(" RUN test0 with otherarg", otherarg)def test_1(modarg): print(" RUN test1 with modarg", modarg)def test_2(otherarg, modarg): print(f" RUN test2 with otherarg {otherarg} and modarg {modarg}")
Let’s run the tests in verbose mode and with looking at the print-output:
$ pytest -v -s test_module.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/pythoncachedir: .pytest_cacherootdir: /home/sweet/projectcollecting ... collected 8 itemstest_module.py::test_0[1] SETUP otherarg 1 RUN test0 with otherarg 1PASSED TEARDOWN otherarg 1test_module.py::test_0[2] SETUP otherarg 2 RUN test0 with otherarg 2PASSED TEARDOWN otherarg 2test_module.py::test_1[mod1] SETUP modarg mod1 RUN test1 with modarg mod1PASSEDtest_module.py::test_2[mod1-1] SETUP otherarg 1 RUN test2 with otherarg 1 and modarg mod1PASSED TEARDOWN otherarg 1test_module.py::test_2[mod1-2] SETUP otherarg 2 RUN test2 with otherarg 2 and modarg mod1PASSED TEARDOWN otherarg 2test_module.py::test_1[mod2] TEARDOWN modarg mod1 SETUP modarg mod2 RUN test1 with modarg mod2PASSEDtest_module.py::test_2[mod2-1] SETUP otherarg 1 RUN test2 with otherarg 1 and modarg mod2PASSED TEARDOWN otherarg 1test_module.py::test_2[mod2-2] SETUP otherarg 2 RUN test2 with otherarg 2 and modarg mod2PASSED TEARDOWN otherarg 2 TEARDOWN modarg mod2============================ 8 passed in 0.12s =============================
You can see that the parametrized module-scoped modarg resource caused anordering of test execution that lead to the fewest possible “active” resources.The finalizer for the mod1 parametrized resource was executed before themod2 resource was setup.
In particular notice that test_0 is completely independent and finishes first.Then test_1 is executed with mod1, then test_2 with mod1, then test_1with mod2 and finally test_2 with mod2.
The otherarg parametrized resource (having function scope) was set up beforeand teared down after every test that used it.
Use fixtures in classes and modules with usefixtures¶
Sometimes test functions do not directly need access to a fixture object.For example, tests may require to operate with an empty directory as thecurrent working directory but otherwise do not care for the concretedirectory. Here is how you can use the standard tempfileand pytest fixtures toachieve it. We separate the creation of the fixture into a conftest.pyfile:
# content of conftest.pyimport osimport tempfileimport pytest@pytest.fixturedef cleandir(): with tempfile.TemporaryDirectory() as newpath: old_cwd = os.getcwd() os.chdir(newpath) yield os.chdir(old_cwd)
and declare its use in a test module via a usefixtures marker:
# content of test_setenv.pyimport osimport pytest@pytest.mark.usefixtures("cleandir")class TestDirectoryInit: def test_cwd_starts_empty(self): assert os.listdir(os.getcwd()) == [] with open("myfile", "w", encoding="utf-8") as f: f.write("hello") def test_cwd_again_starts_empty(self): assert os.listdir(os.getcwd()) == []
Due to the usefixtures marker, the cleandir fixturewill be required for the execution of each test method, just as ifyou specified a “cleandir” function argument to each of them. Let’s run itto verify our fixture is activated and the tests pass:
$ pytest -q.. [100%]2 passed in 0.12s
You can specify multiple fixtures like this:
@pytest.mark.usefixtures("cleandir", "anotherfixture")def test(): ...
and you may specify fixture usage at the test module level using pytestmark:
pytestmark = pytest.mark.usefixtures("cleandir")
It is also possible to put fixtures required by all tests in your projectinto an ini-file:
# content of pytest.ini[pytest]usefixtures = cleandir
Warning
Note this mark has no effect in fixture functions. For example,this will not work as expected:
@pytest.mark.usefixtures("my_other_fixture")@pytest.fixturedef my_fixture_that_sadly_wont_use_my_other_fixture(): ...
This generates a deprecation warning, and will become an error in Pytest 8.
Overriding fixtures on various levels¶
In relatively large test suite, you most likely need to override a global or root fixture with a locallydefined one, keeping the test code readable and maintainable.
Override a fixture on a folder (conftest) level¶
Given the tests file structure is:
tests/ conftest.py # content of tests/conftest.py import pytest @pytest.fixture def username(): return 'username' test_something.py # content of tests/test_something.py def test_username(username): assert username == 'username' subfolder/ conftest.py # content of tests/subfolder/conftest.py import pytest @pytest.fixture def username(username): return 'overridden-' + username test_something_else.py # content of tests/subfolder/test_something_else.py def test_username(username): assert username == 'overridden-username'
As you can see, a fixture with the same name can be overridden for certain test folder level.Note that the base or super fixture can be accessed from the overridingfixture easily - used in the example above.
Override a fixture on a test module level¶
Given the tests file structure is:
tests/ conftest.py # content of tests/conftest.py import pytest @pytest.fixture def username(): return 'username' test_something.py # content of tests/test_something.py import pytest @pytest.fixture def username(username): return 'overridden-' + username def test_username(username): assert username == 'overridden-username' test_something_else.py # content of tests/test_something_else.py import pytest @pytest.fixture def username(username): return 'overridden-else-' + username def test_username(username): assert username == 'overridden-else-username'
In the example above, a fixture with the same name can be overridden for certain test module.
Override a fixture with direct test parametrization¶
Given the tests file structure is:
tests/ conftest.py # content of tests/conftest.py import pytest @pytest.fixture def username(): return 'username' @pytest.fixture def other_username(username): return 'other-' + username test_something.py # content of tests/test_something.py import pytest @pytest.mark.parametrize('username', ['directly-overridden-username']) def test_username(username): assert username == 'directly-overridden-username' @pytest.mark.parametrize('username', ['directly-overridden-username-other']) def test_username_other(other_username): assert other_username == 'other-directly-overridden-username-other'
In the example above, a fixture value is overridden by the test parameter value. Note that the value of the fixturecan be overridden this way even if the test doesn’t use it directly (doesn’t mention it in the function prototype).
Override a parametrized fixture with non-parametrized one and vice versa¶
Given the tests file structure is:
tests/ conftest.py # content of tests/conftest.py import pytest @pytest.fixture(params=['one', 'two', 'three']) def parametrized_username(request): return request.param @pytest.fixture def non_parametrized_username(request): return 'username' test_something.py # content of tests/test_something.py import pytest @pytest.fixture def parametrized_username(): return 'overridden-username' @pytest.fixture(params=['one', 'two', 'three']) def non_parametrized_username(request): return request.param def test_username(parametrized_username): assert parametrized_username == 'overridden-username' def test_parametrized_username(non_parametrized_username): assert non_parametrized_username in ['one', 'two', 'three'] test_something_else.py # content of tests/test_something_else.py def test_username(parametrized_username): assert parametrized_username in ['one', 'two', 'three'] def test_username(non_parametrized_username): assert non_parametrized_username == 'username'
In the example above, a parametrized fixture is overridden with a non-parametrized version, anda non-parametrized fixture is overridden with a parametrized version for certain test module.The same applies for the test folder level obviously.
Using fixtures from other projects¶
Usually projects that provide pytest support will use entry points,so just installing those projects into an environment will make those fixtures available for use.
In case you want to use fixtures from a project that does not use entry points, you candefine pytest_plugins in your top conftest.py file to register that moduleas a plugin.
Suppose you have some fixtures in mylibrary.fixtures and you want to reuse them into yourapp/tests directory.
All you need to do is to define pytest_plugins in app/tests/conftest.pypointing to that module.
pytest_plugins = "mylibrary.fixtures"
This effectively registers mylibrary.fixtures as a plugin, making all its fixtures andhooks available to tests in app/tests.
Note
Sometimes users will import fixtures from other projects for use, however this is notrecommended: importing fixtures into a module will register them in pytestas defined in that module.
This has minor consequences, such as appearing multiple times in pytest --help,but it is not recommended because this behavior might change/stop workingin future versions.
