Programming with Python | Chapter 19: Testing with pytest (Alternative Framework)

Chapter Objectives

• Understand the benefits of using pytest as an alternative to unittest.
• Learn how to install pytest using pip.
• Write simple test functions recognizable by pytest.
• Use Python’s standard assert statement for checking conditions in pytest.
• Understand pytest‘s test discovery mechanism.
• Learn how to run tests using the pytest command-line tool.
• Introduce fixtures using the @pytest.fixture decorator for setup and teardown.
• Learn how to parameterize tests using @pytest.mark.parametrize to run the same test with different inputs.
• Compare the syntax and philosophy of pytest with unittest.

Introduction

While Python‘s built-in unittest module (Chapter 18) provides a solid foundation for testing, the Python ecosystem offers powerful third-party alternatives. pytest is arguably the most popular testing framework for Python, favored for its simpler syntax, powerful features, and extensive plugin ecosystem. Instead of requiring test classes and specific self.assert* methods like unittest, pytest allows you to write tests as simple functions using the standard Python assert statement. It boasts excellent test discovery, informative tracebacks, and a highly flexible fixture system for managing test setup and teardown. This chapter introduces the basics of pytest, showing you how to write, run, and organize tests using its conventions.

Theory & Explanation

pytest Philosophy and Advantages

• Less Boilerplate: Tests are typically simple functions, not methods within classes inheriting from unittest.TestCase.
• Standard assert: Uses the standard Python assert statement for checks. pytest provides detailed introspection on assertion failures.
• Powerful Fixtures: Offers a more flexible and reusable way to manage test setup, teardown, and dependencies compared to unittest‘s setUp/tearDown.
• Test Discovery: Excellent automatic discovery of test files (test_*.py or *_test.py) and test functions (test_*).
• Rich Plugin Ecosystem: Numerous plugins available for enhanced reporting, coverage analysis, parallel execution, integration with web frameworks (like Django, Flask), etc.
• Readability: Often considered more readable due to less boilerplate and standard assertions.
• Compatibility: Can run many existing unittest and nose test suites without modification.

Installation

pytest is a third-party library, so you need to install it using pip:

pip install pytest

Writing Basic Tests

With pytest, a test is typically just a Python function whose name starts with test_. Test files should also follow naming conventions, usually test_*.py or *_test.py.

# File: test_example.py

# Function to be tested (could be imported from another module)
def add(x, y):
    return x + y

# Test function for pytest
def test_add_positive():
    """Tests adding two positive numbers."""
    result = add(2, 3)
    assert result == 5 # Use standard assert

def test_add_negative():
    """Tests adding two negative numbers."""
    assert add(-1, -2) == -3

def test_add_mixed():
    """Tests adding positive and negative numbers."""
    assert add(5, -3) == 2

# No special classes or imports needed for basic tests!

Assertions in pytest

You use the standard Python assert statement. When an assertion fails, pytest provides detailed output showing the values involved in the comparison, making debugging easier than unittest‘s basic failure messages (unless you provide custom messages in unittest).

# Example of pytest assertion output (conceptual)
def test_assertion_detail():
    x = [1, 2, 3]
    y = [1, 2, 4]
    assert x == y # This will fail

# pytest output might look like:
# E   AssertionError: assert [1, 2, 3] == [1, 2, 4]
# E     At index 2 diff: 3 != 4

For checking exceptions, use pytest.raises as a context manager:

import pytest

# Function that raises an error
def divide(a, b):
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

def test_divide_by_zero():
    """Tests that division by zero raises ValueError."""
    with pytest.raises(ValueError):
        divide(10, 0)

def test_divide_by_zero_message():
    """Tests the specific error message."""
    with pytest.raises(ValueError, match="Cannot divide by zero"):
        divide(5, 0)

The match argument allows checking the exception message using a regular expression.

Running Tests with pytest

Navigate to your project’s root directory (or the directory containing your tests) in the terminal and simply run:

pytest

pytest will automatically discover and run all test files matching the pattern test_*.py or *_test.py. It executes functions named test_*, and methods named test_* inside classes named Test*. It can also run unittest-style test classes.

graph TD
    A["Start: <br>Run <i>pytest</i> command"] --> B{Discover Files};
    B -- "Matches <i>test_*.py</i> or <i>*_test.py</i>" --> C{Discover Tests};
    B -- No Match --> F[End: No tests found];
    C -- "Matches <i>test_*</i> functions/methods" --> D["Execute Test Setup (Fixtures)"];
    C -- No Match --> B;
    D --> E[Run Test Function/Method];
    E --> G{Assertion Passed?};
    G -- Yes --> H["Execute Test Teardown (Fixtures)"];
    G -- No --> I[Record Failure];
    I --> H;
    H --> C;
    C -- All tests in file done --> B;
    B -- All files done --> J[Generate Report];
    J --> K[End: Display Results];

    style A fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px
    style K fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px
    style F fill:#FECACA,stroke:#B91C1C,stroke-width:2px

Common pytest Command-Line Options

• pytest -v
  Verbose output — shows test names and status.
• pytest -q
  Quiet output — minimal output.
• pytest -k "expression"
  Run tests matching the given expression. Example: pytest -k "add or string".
• pytest path/to/test_file.py
  Run tests in a specific file.
• pytest path/to/test_file.py::test_function_name
  Run a specific test function.
• pytest --maxfail=N
  Stop after N failures.
• pytest -x
  Stop instantly on the first failure.
• pytest --lf or --last-failed
  Run only the tests that failed last time.
• pytest --ff or --failed-first
  Run failed tests first, then the remaining ones.

Fixtures (@pytest.fixture)

Fixtures are a powerful pytest feature used for setup, teardown, and providing test data or dependencies. They are defined using the @pytest.fixture decorator. Test functions receive fixtures by including them as parameters.

Key Concepts:

• Scope: Controls how often the fixture is invoked (function, class, module, session). Default is function.
• Setup/Teardown: Code before the yield runs during setup; code after runs during teardown.
• Dependency Injection: pytest injects fixtures into test functions automatically.

# File: test_fixtures.py
import pytest

@pytest.fixture # Default scope is 'function'
def sample_list():
    """Fixture providing a sample list for tests."""
    print("\n(Setting up sample_list fixture)")
    data = [1, 2, 3, 4, 5]
    yield data # Provide the data to the test
    # Teardown code would go here after yield (if needed)
    print("(Tearing down sample_list fixture)")

@pytest.fixture(scope="module") # Runs once for all tests in this module
def expensive_resource():
    """Fixture simulating an expensive resource setup/teardown."""
    print("\n(Setting up expensive_resource - MODULE scope)")
    resource = {"id": 1, "data": "shared"}
    yield resource
    print("(Tearing down expensive_resource - MODULE scope)")


# Test functions receive fixtures by naming them as arguments
def test_list_length(sample_list):
    """Test using the sample_list fixture."""
    print("Running test_list_length...")
    assert len(sample_list) == 5

def test_list_content(sample_list):
    """Another test using the same sample_list fixture."""
    # Note: sample_list setup/teardown runs again because scope is 'function'
    print("Running test_list_content...")
    assert 3 in sample_list

def test_with_expensive_resource(expensive_resource):
    """Test using the module-scoped fixture."""
    print("Running test_with_expensive_resource...")
    assert expensive_resource["id"] == 1

def test_with_expensive_resource_again(expensive_resource):
    """Another test using the module-scoped fixture."""
    # Note: expensive_resource setup/teardown does NOT run again
    print("Running test_with_expensive_resource_again...")
    assert expensive_resource["data"] == "shared"

Running pytest -v -s (-s shows print output) on this file demonstrates how fixtures are set up and torn down based on their scope and usage.

graph TD
    subgraph Test Function Execution
        D["Test function <i>test_example(my_fixture)</i> starts"]
        E{"Fixture <i>my_fixture</i> requested"}
        F["pytest executes <i>my_fixture</i> setup code (before yield)"]
        G["Value from <i>yield</i> is passed to <i>test_example</i>"]
        H["Test function <i>test_example</i> runs its code"]
        I["Test function <i>test_example</i> finishes"]
        J["pytest executes <i>my_fixture</i> teardown code (after yield)"]
        K[Test function complete]
    end

    A["pytest runs test <i>test_example</i>"] --> D;
    D --> E;
    E --> F;
    F -- Yields data/resource --> G;
    G --> H;
    H --> I;
    I --> J;
    J --> K;

    style F fill:#D1FAE5,stroke:#047857,stroke-width:1px
    style J fill:#FEE2E2,stroke:#991B1B,stroke-width:1px
    style G fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1px

Parameterizing Tests (@pytest.mark.parametrize)

Often, you want to run the same test logic with different input values and expected outputs. @pytest.mark.parametrize allows you to do this concisely.

Syntax:

@pytest.mark.parametrize("arg_names", list_of_arg_values)
def test_function(arg_names):
    # Test logic using the arguments
    assert ...

• "arg_names": A comma-separated string of argument names that the test function expects.
• list_of_arg_values: A list of tuples (or single values if only one arg_name), where each tuple contains the values for the arguments for one test run.

# File: test_parameterized.py
import pytest

# Function to test
def add(x, y):
    return x + y

# Parameterized test function
@pytest.mark.parametrize("input1, input2, expected", [
    (1, 2, 3),       # Test case 1
    (-1, 1, 0),      # Test case 2
    (0, 0, 0),       # Test case 3
    (100, 200, 300), # Test case 4
    (-5, -10, -15)   # Test case 5
])
def test_add_parametrized(input1, input2, expected):
    """Tests the add function with multiple inputs."""
    assert add(input1, input2) == expected

Running pytest -v on this file will show 5 separate test runs for test_add_parametrized, one for each set of parameters.

graph TD
    A["pytest discovers <i>test_add_parametrized</i>"] --> B{"Decorator: <b>@pytest.mark.parametrize(<i>in1, in2, exp</i>, [(1,2,3), (-1,1,0), ...])</b>"};
    B --> C(Test Run 1);
    B --> D(Test Run 2);
    B --> E(Test Run ...);
    B --> F(Test Run N);

    subgraph Test Execution Iterations
        direction LR
        C -- Args: in1=1, in2=2, exp=3 --> G["test_add_parametrized(1, 2, 3)"];
        D -- Args: in1=-1, in2=1, exp=0 --> H["test_add_parametrized(-1, 1, 0)"];
        E -- Args: ... --> I["test_add_parametrized(...)"];
        F -- Args: ... --> J["test_add_parametrized(...)"];
        G --> K{"Assert <i>add(in1, in2) == exp</i>"};
        H --> K;
        I --> K;
        J --> K;
        K -- Pass/Fail --> L[Record Result];
    end

    style B fill:#FEF9C3,stroke:#854D0E,stroke-width:1px

Comparing pytest and unittest

Many developers prefer pytest for new projects due to its conciseness and powerful features, but unittest remains important as it’s part of the standard library and widely understood.

Code Examples

Example 1: Rewriting unittest Calculation Tests with pytest

File: calculations.py (Same as Chapter 18)

File: test_calculations_pytest.py

# test_calculations_pytest.py
import pytest
from calculations import add, subtract, multiply, divide # Import functions

# --- Test functions for pytest ---

def test_add():
    """Test the add function with various inputs."""
    assert add(2, 3) == 5
    assert add(-1, 1) == 0
    assert add(-1, -1) == -2
    assert add(0, 0) == 0

def test_subtract():
    """Test the subtract function."""
    assert subtract(10, 5) == 5
    assert subtract(-1, 1) == -2
    assert subtract(-1, -1) == 0

def test_multiply():
    """Test the multiply function."""
    assert multiply(3, 7) == 21
    assert multiply(-1, 1) == -1
    assert multiply(-1, -1) == 1
    assert multiply(5, 0) == 0

def test_divide():
    """Test the divide function for valid inputs."""
    assert divide(10, 2) == 5
    assert divide(-1, 1) == -1
    assert divide(-1, -1) == 1
    assert divide(5, 2) == 2.5

def test_divide_by_zero():
    """Test that divide raises ValueError for division by zero."""
    with pytest.raises(ValueError, match="Cannot divide by zero"):
        divide(10, 0)

# No classes or unittest.main() needed!

Running the tests (from terminal):

pytest test_calculations_pytest.py -v

This demonstrates the reduced boilerplate compared to the unittest version.

Example 2: Using Fixtures and Parameterization

File: test_data_processing.py

# test_data_processing.py
import pytest

# --- Code to be tested (Example) ---
def process_data(data_list):
    """Calculates sum and average of a list of numbers."""
    if not data_list:
        return 0, 0
    total = sum(data_list)
    average = total / len(data_list)
    return total, average

# --- Fixtures ---
@pytest.fixture
def empty_list():
    return []

@pytest.fixture
def list_with_positives():
    return [10, 20, 30]

@pytest.fixture
def list_with_mixed():
    return [10, -5, 15, 0]

# --- Test Functions ---
def test_process_empty(empty_list):
    """Test processing an empty list using a fixture."""
    total, average = process_data(empty_list)
    assert total == 0
    assert average == 0

def test_process_positives(list_with_positives):
    """Test processing positive numbers using a fixture."""
    total, average = process_data(list_with_positives)
    assert total == 60
    assert average == 20

def test_process_mixed(list_with_mixed):
    """Test processing mixed numbers using a fixture."""
    total, average = process_data(list_with_mixed)
    assert total == 20
    assert average == 5

# --- Parameterized Test ---
@pytest.mark.parametrize("input_list, expected_total, expected_average", [
    ([], 0, 0),                           # Test case 1: Empty
    ([10, 20, 30], 60, 20),               # Test case 2: Positives
    ([10, -5, 15, 0], 20, 5),             # Test case 3: Mixed
    ([7], 7, 7),                          # Test case 4: Single element
    ([-10, -20], -30, -15)                # Test case 5: Negatives
])
def test_process_data_parametrized(input_list, expected_total, expected_average):
    """Parameterized test for process_data function."""
    total, average = process_data(input_list)
    assert total == expected_total
    # Use pytest.approx for float comparison if necessary
    assert average == pytest.approx(expected_average)

Explanation:

• Fixtures (empty_list, list_with_positives, list_with_mixed) provide predefined test data. Test functions receive this data by naming the fixture as an argument.
• test_process_data_parametrized uses @pytest.mark.parametrize to run the same test logic with five different sets of inputs and expected outputs, reducing code duplication.
• pytest.approx is used for comparing floating-point numbers, handling potential small inaccuracies.

Common Mistakes or Pitfalls

• Forgetting to Install pytest: Running pytest without installing it first will result in a “command not found” error.
• Incorrect Naming Conventions: Test files not named test_*.py or *_test.py, or test functions not starting with test_, will not be discovered by default.
• Misusing assert: Forgetting that pytest uses the standard assert statement and trying to use unittest‘s self.assertEqual() etc. in pytest functions (unless running unittest classes via pytest).
• Fixture Scope Confusion: Not understanding how fixture scopes (function, class, module, session) affect when setup/teardown code runs, potentially leading to unexpected sharing or re-creation of resources.
• Overly Complex Fixtures: While powerful, very complex fixture dependencies can sometimes make tests harder to understand.

Chapter Summary

• pytest is a popular, powerful testing framework for Python, often preferred for its simpler syntax and features.
• Install using pip install pytest.
• Tests are typically functions named test_* in files named test_*.py or *_test.py.
• Use the standard Python assert statement for checks; pytest provides detailed failure reports.
• Use pytest.raises as a context manager to test for expected exceptions.
• Run tests using the pytest command in the terminal.
• Fixtures (@pytest.fixture) provide a flexible way to manage test setup, teardown, and dependencies. Tests receive fixtures as arguments.
• Parameterization (@pytest.mark.parametrize) allows running a single test function with multiple sets of input data.

Exercises & Mini Projects

Exercises

1. Install pytest: If you haven’t already, install pytest using pip.
2. Rewrite is_even Test: Take the is_even function and its unittest tests from Chapter 18 Exercise 1. Rewrite the tests in a new file (test_even_pytest.py) using pytest‘s function-based approach and standard assert statements. Run the tests using pytest.
3. Rewrite List Function Test: Rewrite the get_first_element tests from Chapter 18 Exercise 2 using pytest. Use pytest.raises(IndexError) for the empty list case.
4. pytest Fixture: Create a fixture named sample_dict that returns a simple dictionary (e.g., {"a": 1, "b": 2}). Write two test functions that accept sample_dict as an argument:
   • test_dict_keys: Assert that the key “a” is in the dictionary.
   • test_dict_value: Assert that the value associated with key “b” is 2.
5. Parameterize String Test: Write a parameterized pytest function test_string_length that takes two arguments, input_string and expected_length. Use @pytest.mark.parametrize to test it with at least three different strings and their expected lengths (e.g., “”, “abc”, ” pytest “). Inside the test, assert that len(input_string) equals expected_length.

Mini Project: Parameterizing BankAccount Tests with pytest

Goal: Rewrite some of the BankAccount tests from Chapter 18 using pytest and parameterization.

Steps:

1. Create Test File: Create test_bank_account_pytest.py.
2. Import: Import pytest and the BankAccount class.
3. Basic Fixture: Create a pytest fixture basic_account that yields a BankAccount("Test User", 100.0).
4. Test Initialization (using fixture): Write a test function test_initialization(basic_account) that uses the fixture and asserts the initial holder name and balance.
5. Parameterize Deposit Test: Create a parameterized test function test_deposit(basic_account, deposit_amount, expected_balance).
   • Use @pytest.mark.parametrize to provide test cases for:
     • Valid positive deposit (e.g., 50.50 -> 150.50)
     • Zero deposit (e.g., 0 -> 100.0)
     • Negative deposit (e.g., -20 -> 100.0)
   • Inside the test, call basic_account.deposit(deposit_amount) and assert that basic_account.get_balance() equals expected_balance.
6. Parameterize Withdrawal Test: Create a similar parameterized test function test_withdrawal(basic_account, withdraw_amount, expected_balance).
   • Use @pytest.mark.parametrize for cases:
     • Valid withdrawal (e.g., 30 -> 70.0)
     • Withdrawing exact balance (e.g., 100 -> 0.0)
     • Withdrawing zero (e.g., 0 -> 100.0)
     • Withdrawing negative (e.g., -50 -> 100.0)
     • Insufficient funds (e.g., 100.01 -> 100.0)
   • Inside the test, call basic_account.withdraw(withdraw_amount) and assert the expected_balance.
7. Run Tests: Execute pytest -v and verify the results. Notice how parameterization reduces the number of test functions needed compared to the unittest version for similar coverage.

Additional Sources:

• Pytest