Test whether approval survives a changed refund

Test AI-generated approval code by changing the request after approval, then checking both the rejection result and the absence of a payment call. Pair that test with an unchanged request that succeeds, so rejecting everything cannot satisfy the requirement.

For example, approve a USD 100.00 refund, change it to USD 150.00, and try to execute it with the original approval. An implementation that checks only approved == true can accept the second amount without anyone reviewing it.

The decision is what approval authorizes: a mutable request record, or the exact action someone reviewed. For this refund flow, bind approval to an immutable snapshot of the action.

Make the approved action explicit

The authors of ShallowDefend propose an Assess → Decide → Respond framework in which a model commits to an explicit, parseable safety decision before generating its reply. That is a proposal about model safety behavior, not evidence that a software approval workflow is correct. Its useful connection here is the distinction between recording a decision and leaving that decision implicit in later behavior. ShallowDefend abstract on arXiv

For an AI-assisted refund implementation, an explicit approval is a starting point. You must also specify what that approval remains valid for.

Use this action tuple for the example:

RefundAction = (
    refund_id,
    customer_id,
    amount_cents,
    currency
)

The approval stores a separate, immutable copy of this tuple. It must not hold a reference to an object that later edits can mutate.

A boolean records a decision, but not necessarily its scope.

For this exercise, assume the customer ID determines the payment destination and all four fields are available in a canonical representation. If the destination can change independently, the tuple needs an additional destination identifier or version.

Write the requirement before reviewing the diff

The approval-binding requirement is:

Execution requires an approved action tuple that exactly matches the current action tuple. A mismatch must return approval_mismatch before any payment call.

This is a necessary condition, not the entire refund policy. An exact match does not replace permission checks, approval revocation rules, or duplicate-payment protection.

Within this example, approval behaves as follows:

An edit can invalidate approval eagerly by changing workflow state, or execution can detect the mismatch. Either way, execution must enforce the requirement.

Here is the counterexample the contract must reject:

Create   (refund-17, customer-8, 10000, USD)
Approve  (refund-17, customer-8, 10000, USD) → approval-1
Edit     (refund-17, customer-8, 15000, USD)
Execute  refund-17 using approval-1

Required result: approval_mismatch
Required payment calls: 0

The refund ID stayed the same. The authorized action did not.

Test the refusal and the side effect together

The following Python acceptance test defines a proposed adapter interface, not a library API. Supply a make_flow test fixture that connects these operations to the implementation under review and injects the payment spy as its only payment dependency. Use isolated test storage and no live payment credentials.

In this interface, save_request creates or updates a request, approve captures its current tuple, and execute returns a status string.

from dataclasses import dataclass, replace

import pytest


@dataclass(frozen=True)
class RefundAction:
    refund_id: str
    customer_id: str
    amount_cents: int
    currency: str


class PaymentSpy:
    def __init__(self):
        self.calls = []

    def refund(self, action):
        self.calls.append(action)
        return "mock-payment-1"


@pytest.mark.parametrize(
    "execution_amount, expected_status",
    [
        (15000, "approval_mismatch"),
        (10000, "executed"),
    ],
)
def test_approval_binds_to_reviewed_amount(
    make_flow, execution_amount, expected_status
):
    payments = PaymentSpy()
    flow = make_flow(payments=payments)
    reviewed = RefundAction(
        refund_id="refund-17",
        customer_id="customer-8",
        amount_cents=10000,
        currency="USD",
    )
    flow.save_request(reviewed)
    approval_id = flow.approve(reviewed.refund_id)

    current = replace(reviewed, amount_cents=execution_amount)
    flow.save_request(current)
    result = flow.execute(current.refund_id, approval_id)

    assert result == expected_status
    expected_calls = [] if execution_amount != 10000 else [reviewed]
    assert payments.calls == expected_calls

The changed-request case checks that execution refuses the mismatch and never invokes the payment dependency. Returning an error after issuing a payment would fail this test, even if the implementation later attempted compensation.

The unchanged-request case checks that the workflow can still execute the approved action exactly once in this scenario. It prevents a blanket refusal from looking like a correct implementation.

A rejection result alone cannot show that no payment was attempted.

The spy observes only calls routed through it. If the implementation can reach a second payment client or enqueue payment work elsewhere, those paths must also be isolated and observed.

Use the contract to review the agent’s diff

Give the coding agent the requirement, the invalid trace, and the acceptance cases before it edits code. Then review whether the diff preserves the same meaning:

• Approval captures a snapshot rather than a mutable object reference.
• Execution compares every declared action field, not just the refund ID or approval flag.
• A mismatch stops processing before payment dispatch or enqueueing.
• The payment payload comes from the same validated action that passed comparison.

The last point matters under concurrency. Comparing one version of the request and then rereading a newer version for payment leaves a gap. A production design needs an appropriate consistency mechanism, such as version-checked dispatch or a transactional handoff, based on its storage and payment architecture.

The tests above cover selected sequential transitions. They do not establish correct behavior for every field mutation, concurrent update, authorization defect, or downstream payment outcome. Passing them is evidence for a narrow contract, not proof that the implementation is safe.

Keep an approval checklist beside the requirement

Use this checklist when deciding which changes require reapproval:

For this example, changing any tuple field requires approval of the new tuple. A private display note can remain outside the tuple only if it has no effect on those decisions.

In five minutes, write down one workflow’s approved tuple and add one changed-field rejection case plus an unchanged control. That makes one approval boundary checkable; it does not replace testing the remaining fields or concurrent execution.

To use that declared intent when reviewing an AI-assisted change, start with ThunderLang’s getting-started guide.