Prompt Engineering for QA: Property-Based, Mutation, and Contract Testing with AI

Most "AI-generated tests" are shallow example-based unit tests that mirror the happy path already in the code — useful for coverage numbers, less useful for actually catching bugs. QA and test engineers get far more value prompting for property-based tests, mutation testing, and contract tests, which are exactly the test styles most teams under-invest in because they're tedious to write by hand.

Property-based testing: describe invariants, not examples

Instead of asking for "a test for this function," prompt for the properties that must hold for all inputs — e.g. "serializing then deserializing always returns an equal object" or "sorting is idempotent." The model can then generate property-based tests (FsCheck, and equivalents in other ecosystems) that explore the input space far more broadly than hand-written examples.

Context: C# record `Money(decimal Amount, string CurrencyCode)` with an `Add(Money other)` method that throws if CurrencyCode differs.

Task: Identify the properties that should hold for Add, then generate FsCheck property-based tests for them.

Constraints: Cover at minimum: commutativity (a.Add(b) == b.Add(a) when currencies match), identity (a.Add(Money.Zero(a.CurrencyCode)) == a), and the exception case (mismatched currencies always throw, for any two distinct currency codes).

Output format: The list of properties in plain English first, then the FsCheck test code.

[<Property>]
let ``Add is commutative for matching currencies`` (amountA: decimal) (amountB: decimal) (currency: string) =
    let a = Money(amountA, currency)
    let b = Money(amountB, currency)
    a.Add(b) = b.Add(a)

[<Property>]
let ``Add throws on mismatched currencies`` (amount: decimal) (curA: string) (curB: string) =
    (curA <> curB) ==> lazy (
        let a = Money(amount, curA)
        let b = Money(amount, curB)
        Prop.throws<InvalidOperationException, _> (lazy a.Add(b)))

Mutation testing: let AI grade your test suite

Mutation testing (tools like Stryker) deliberately introduces small bugs into your code and checks whether your tests catch them. Prompting AI to analyze surviving mutants — mutations your suite failed to catch — surfaces exactly which assertions are missing, which is a much higher-signal exercise than chasing raw coverage percentage.

Context: Stryker.NET report shows 3 surviving mutants in DiscountCalculator.cs: (1) line 22, `>` changed to `>=` survived, (2) line 40, return value changed from `total * 0.9m` to `total` survived, (3) line 55, a boolean `&&` changed to `||` survived.

Task: For each surviving mutant, explain what test case would have killed it, and write that test.

Output format: One test method per mutant, named to reference which mutant it targets.

Consumer-driven contract tests for service boundaries

For microservices, AI is useful for drafting consumer-driven contracts (Pact-style) from an API's actual usage patterns, catching breaking changes before they reach staging rather than after a downstream consumer fails.

Context: The Checkout service calls GET /api/inventory/{sku} on the Inventory service and depends on fields `sku`, `quantityAvailable`, and `warehouseId` from the response. It ignores all other fields.

Task: Write a Pact consumer contract test from Checkout's perspective that pins down exactly this expectation — no more, no less — so the Inventory team can verify against it without being blocked on fields Checkout doesn't use.

These patterns are core to the Tester/QA role guide and the Testing Existing Systems module of the Existing Project Analysis & Modernization program.