Pytest Parameter Tests

Recently, I’ve been converting a small python script I wrote into an organized project. As part of that work, I started to setup an automated testing framework (like a good QE 😆). While writing tests, I have been parameterizing the code so that each test function can be provided multiple inputs, resulting in multiple tests (based from the same function). I talked about this previously, but used a different method then. This time, I opted to try the parametrize marker.

Conftest.py Method

The last time I wrote about pytest, I used parameterization. However, I implemented it a bit differently, using fixtures defined in the conftest.py file to do the parameterization.

For example, this fixture function loops through a list of post names, and returns the relative url for each post:

@pytest.fixture(params=POST_NAMES)
def post_url(request):
    """Returns the post urls for testing."""
    return BASE_URL + "/post/" + request.param.lower()

Next, I used that fixture as the single parameter in my test function:

def test_post_served(post_url):
    """Checks that the desired posts are available"""
    response = requests.get(post_url)
    assert response.status_code == 200

This causes the test to loop across each item the fixture returns from the POST_NAMES constant list, resulting in a test run for each one.

Using Pytest Parameterize

While that method works, tests can be parameterized without the overhead of setting up fixtures for each test data set. Instead, the parametrize pytest marker can be used.

I started by creating my set of test inputs:

example_resolution_ratios = [
    (1920, 1080, Fraction(16, 9)),
    (2560, 1440, Fraction(16, 9)),
    (3840, 2160, Fraction(16, 9)),
    (1440, 900, Fraction(8, 5)),
    (1600, 900, Fraction(16, 9)),
    (5120, 2880, Fraction(16, 9)),
    (6016, 3384, Fraction(16, 9)),
    (3440, 1440, Fraction(43, 18)),
]

Next, I added the @pytest.mark.parametrize marker to my test function:

# Example of parameters marker
@pytest.mark.parametrize(
    "horz_pixels,vert_pixels,expected_ratio",
    example_resolution_ratios,
    ids=[f"{pair[0]}x{pair[1]}" for pair in example_resolution_ratios],
)
def test_resolution_ratio(horz_pixels, vert_pixels, expected_ratio):
    """Tests resolution ratio with set of known calculation results."""
    assert resolution_ratio(horz_pixels, vert_pixels) == expected_ratio

While this marker looks intimidating (It’s larger than the actual test code!), it does a lot for us. It requires at least two parameters, a string containing parameter names to pass to the test ("horz_pixels,vert_pixels,expected_ratio" in this example), and the list of test items to iterate over (example_resolution_ratios). Notice, if the test items are a list of tuples, pytest can match each tuple value to a different parameter to pass to the test function.

Additional options can be provided to the parametrize marker. For example, I used the ids parameter to define how each parametrized test instance is identified when run. This test function verifies a ratio calculator, so using a list comprehension, I was able to have the resolution used for each test displayed in the run output:

...
/tests/test_calcs.py::test_resolution_ratio[1920x1080] PASSED [  3%]
/tests/test_calcs.py::test_resolution_ratio[2560x1440] PASSED [  6%]
/tests/test_calcs.py::test_resolution_ratio[3840x2160] PASSED [ 10%]
/tests/test_calcs.py::test_resolution_ratio[1440x900] PASSED [ 13%]
/tests/test_calcs.py::test_resolution_ratio[1600x900] PASSED [ 16%]
/tests/test_calcs.py::test_resolution_ratio[5120x2880] PASSED [ 20%]
/tests/test_calcs.py::test_resolution_ratio[6016x3384] PASSED [ 23%]
/tests/test_calcs.py::test_resolution_ratio[3440x1440] PASSED [ 26%]
...

This is extremely helpful when a few tests are failing, as you can easily identify what specific inputs are failing.

Conclusion

That’s really all there is to it. If you’re using pytest and want to parameterize the test functions, the parametrize marker is a simple, but powerful tool to do so. We use it often at work for some complicated setups, but I enjoyed getting to try here in a much simpler case!