How to test AI-generated code before merging

Treat AI-generated code and AI-generated tests as one untrusted proposal. Before running either, restate the behavior in terms a caller or user can observe.

The prompt is useful context, but it is not a test oracle. Derive expected behavior from the ticket, API contract, schema, existing callers, and production invariants.

Write down what must be preserved before looking for a fix. At minimum, cover:

• Valid boundary values: zero, false, empty strings, empty arrays, and explicit nulls where allowed.
• Invalid input: missing fields, malformed values, and combinations the schema should reject.
• Ordering: duplicate, delayed, retried, or out-of-order work.
• Ownership: which user, request, component, or transaction is allowed to change the state.
• Failure behavior: timeout, partial response, dependency error, and cleanup after failure.

If the patch uses input.limit || 20, do not just test a missing limit. Test an explicit 0. That single case distinguishes absence from a valid falsy value.

1. Baseline: run the existing suite against the unmodified patch. Record the first failure instead of editing immediately.
2. Adversarial case: add one test for the most plausible broken assumption. Make sure it fails for the right reason before changing code.
3. Regression: apply the smallest correction, rerun the focused test, then run the broader suite that protects neighboring behavior.

The highest-value cases are often ordinary inputs in an inconvenient order, not exotic fuzzing inputs.

• Falsy-value replacement: an explicit value is silently replaced by a default.
• Stale async state: an older request resolves after a newer one and overwrites the current result.
• Partial authorization: the route checks authentication but not ownership of the requested record.
• Mock-complete behavior: the test double always returns the exact shape the implementation expects.
• Scope drift: the patch fixes the ticket while changing unrelated public behavior.

For a search-as-you-type UI, deliberately resolve the second request before the first. A happy-path test where requests finish in order cannot prove stale-response protection.

A green suite is weak evidence when the same model produced both sides of the assertion. Reject a generated test when it:

• re-implements the production algorithm inside the expectation;
• asserts only that a mock returned its configured value;
• covers the happy path while ignoring documented boundaries;
• updates a snapshot without explaining the intended behavior;
• passes before the defect is fixed.

A useful regression test should fail on the broken patch, pass after the correction, and describe the contract in language a reviewer can understand.

Do not finish with a bare claim that tests pass. Leave a bounded merge record:

• the broken assumption;
• the failing case and expected behavior;
• the smallest code change that corrected it;
• the focused and regression suites run;
• the important behavior that remains unverified.

If you cannot explain those five points, the patch is not ready to merge even when the suite is green.