Writing Property-Based Tests With Hypothesis: Bugs My Example Tests Missed
A date-range overlap function in a scheduling service had a full set of example-based unit tests, adjacent ranges, overlapping ranges, identical ranges, all green, and it still shipped a bug where a range with a zero-length duration, start and end on the same instant, broke the overlap calculation in a way none of my hand-picked examples happened to cover. Rewriting the test with Hypothesis, a property-based testing library for Python, generated exactly that edge case within seconds of the first run, because it wasn't relying on me to think of it.
Example-Based Tests Only Cover What You Thought Of
A standard unit test asserts a specific input produces a specific output, and its coverage is bounded entirely by the test author's imagination, every case not explicitly written is a case not tested. This isn't a knock on unit tests generally, they're clear and fast and good at documenting intended behavior, but for a function with a large input space, date ranges, numeric edge cases, string parsing, the set of inputs a human thinks to write by hand is reliably smaller than the set that actually exists.
What Property-Based Testing Actually Asserts
Instead of asserting a specific input produces a specific output, a property-based test asserts a general property that should hold for any valid input, and the framework generates many varied inputs, including deliberately weird edge cases, to try to find one that breaks the property. For the overlap function, the property isn't "this specific pair of dates overlaps," it's something more general: "a range always overlaps with itself," which should hold for every possible range, not just the ones I happened to write down.
from hypothesis import given, strategies as st
from datetime import date, timedelta
@given(
start=st.dates(min_value=date(2020, 1, 1), max_value=date(2030, 1, 1)),
duration=st.integers(min_value=0, max_value=365),
)
def test_range_always_overlaps_itself(start, duration):
end = start + timedelta(days=duration)
assert ranges_overlap((start, end), (start, end))How Hypothesis Actually Found the Zero-Duration Bug
Running the test above against the original implementation, Hypothesis's generator explored the space of start dates and durations, including duration=0 within the first handful of generated cases, since zero is exactly the kind of boundary value these generators are built to bias toward rather than uniformly random values that would rarely land exactly on an edge. The failing case, a range where start equals end, triggered a comparison in the original overlap logic using a strict less-than where it needed less-than-or-equal, a genuine off-by-one in the boundary condition that three hand-written example tests had all missed by coincidentally never testing a zero-length range.
Shrinking: Hypothesis Doesn't Just Find a Failure, It Minimizes It
The first failing input Hypothesis finds is often not the clearest example of the bug, maybe a range spanning 340 days with an oddly specific start date. Hypothesis automatically shrinks a failing case toward the simplest input that still reproduces the failure, repeatedly trying smaller or simpler variations and keeping the smallest one that still fails, so the actual reported failure was the cleanest possible version, a single-day range where start equals end, not a confusing 340-day edge case that would have taken longer to actually debug.
Writing Good Properties Is the Real Skill, Not the Library API
The library's API is genuinely simple, decorate a test with @given and a strategy describing the input shape. The actual difficulty is thinking in terms of properties instead of examples, which took real practice. Useful categories I now reach for: invariants (a sort should produce output of the same length as the input), round-trip properties (serializing then deserializing an object should return an equal object), and comparison against a simpler reference implementation (a fast overlap check should agree with a deliberately naive, obviously-correct brute-force version on every generated input).
@given(a=st.builds(DateRange, ...), b=st.builds(DateRange, ...))
def test_agrees_with_naive_implementation(a, b):
assert ranges_overlap(a, b) == naive_overlap_check(a, b)Where Property-Based Testing Doesn't Replace Example Tests
Hypothesis is excellent at finding edge cases in general properties, but it doesn't replace example-based tests that document specific, named business scenarios, "a same-day meeting request during business hours should auto-approve," a case worth asserting explicitly for its own sake and readable meaning, not just as an instance of a broader mathematical property. The two approaches complement each other in the same test suite rather than one obsoleting the other, and I kept a handful of clear example tests specifically because they read as documentation in a way a property assertion doesn't.
Final Verdict
The date-range bug that shipped despite passing tests is exactly the class of failure property-based testing is built to catch, an edge case genuinely nobody thought to write down by hand. It's not a wholesale replacement for example-based tests, but for any function with a real input space and clear invariants, adding a handful of property tests alongside the existing suite found a real bug in the first hour of trying it, which is a better return than most testing techniques offer that quickly.
