Test AI-Generated Cancellation Code at the Acknowledgment Boundary

Summary slide. Pause the worker before output becomes visible, cancel successfully, then release it and check for output. Save the slide (PNG)

A worker can check a cancellation flag correctly and still publish output after cancellation succeeds. To test a “no output after acknowledgment” requirement, pause the worker before publication, acknowledge cancellation, then release it.

Consider this illustrative execution for one export job:

Step	Worker	Cancellation handler
1	Reads cancelled == False and starts preparing output.
2	Pauses just before publishing.
3		Sets cancelled = True and returns success.
4	Publishes the prepared output.

The flag worked. The contract failed.

Before choosing a cancellation API or asking a coding agent to implement the change, decide what successful acknowledgment promises.

Make acknowledgment a publication boundary

For this example, use a narrow requirement:

For one export job, after cancellation returns a successful acknowledgment, that job must not publish any further output.

Define the terms before writing code:

• Job: One export instance, not every export belonging to a user.
• Publication: The moment output becomes visible through the designated output sink. In the test below, this is an append to an in-memory list.
• Successful acknowledgment: The cancellation operation returns True. Receiving the request is not enough.
• Further output: New publication events after that acknowledgment. Output published earlier may remain visible.

This contract does not require computation to stop immediately. A worker may finish preparing bytes and then discard them. Nor does it require deleting an earlier export or undoing a notification already sent.

Acknowledgment constrains future publication, not necessarily ongoing computation.

If your API only promises “request received,” do not silently reinterpret it as this stronger guarantee. Change the requirement or introduce a separate acknowledgment that means publication has been blocked.

Save this four-part Boundary Card

Include this card in the task you give a coding agent, then retain it during review.

Part	Export cancellation contract
Requirement	After successful cancellation acknowledgment, this job publishes no further output.
Counterexample	The worker checks at startup, cancellation succeeds, then the worker publishes.
Acceptance test	Pause before publication, await successful cancellation, release the worker, await completion, and assert no publication. Also verify publication without cancellation.
Limit	This checks one controlled schedule and one modeled sink. It does not establish immediate computation shutdown or correctness across all schedules.

An implementation that merely “sets a cancellation flag” satisfies a mechanism description, not necessarily the observable requirement. The card keeps that distinction visible during review.

A startup check cannot govern a later publication race.

Control the race instead of waiting for it

A sleep-based test might start the job, wait briefly, and cancel. But that does not establish where the worker was when cancellation happened. The test could pass because cancellation arrived before work began, never exercising the vulnerable boundary.

Use two events instead:

1. A ready event tells the test that the worker has reached the publication boundary.
2. A release event keeps the worker there until cancellation has acknowledged success.

The ordering is explicit:

ready → successful cancellation acknowledgment → release → worker completion → assertion

The following Python example uses standard-library components. It includes a small reference implementation and two acceptance tests. This is illustrative code, not a reported test run.

import asyncio
import unittest


class ExportJob:
    def __init__(self, published, ready, release):
        self.published = published
        self.ready = ready
        self.release = release
        self.cancelled = False
        self.publication_lock = asyncio.Lock()

    async def run(self):
        # Stand-in for preparing this job's export output.
        payload = b"example export"

        # Test seam: immediately before the publication decision.
        self.ready.set()
        await self.release.wait()

        async with self.publication_lock:
            if self.cancelled:
                return
            # Mock publication: synchronous and visible immediately.
            self.published.append(payload)

    async def cancel(self):
        async with self.publication_lock:
            self.cancelled = True
        return True  # Successful acknowledgment.


class ExportCancellationTests(unittest.IsolatedAsyncioTestCase):
    async def test_no_publication_after_acknowledgment(self):
        published = []
        ready = asyncio.Event()
        release = asyncio.Event()
        job = ExportJob(published, ready, release)
        worker = asyncio.create_task(job.run())

        try:
            await ready.wait()
            acknowledged = await job.cancel()
            self.assertTrue(acknowledged)
            self.assertEqual(published, [])
        finally:
            release.set()
            await worker

        # Check after the worker has had its chance to publish.
        self.assertEqual(published, [])

    async def test_publication_without_cancellation(self):
        published = []
        ready = asyncio.Event()
        release = asyncio.Event()
        job = ExportJob(published, ready, release)
        worker = asyncio.create_task(job.run())

        try:
            await ready.wait()
        finally:
            release.set()
            await worker

        self.assertEqual(published, [b"example export"])

The non-cancelled control matters. An implementation that never publishes anything would otherwise satisfy the negative assertion.

When adapting this test, put the pause in the actual worker path immediately before its publication decision. Do not substitute a test-only cancellation implementation. Use a test-runner timeout to diagnose hangs, but not elapsed time to establish race ordering.

Coordinate the decision and the publication

The reference implementation uses the same lock for cancellation and publication. Checking the flag and appending output happen within one critical section.

There are two relevant orderings:

• Publication gets the lock first: The append happens before cancellation can acknowledge success. Earlier output is permitted by this contract.
• Cancellation gets the lock first: It sets the flag before returning success. A later worker entering the critical section sees the flag and skips publication.

Moving the flag check closer to publication is not enough if another operation can intervene between the check and the side effect. Both need coordination at the boundary the contract names.

For this in-memory sink, a synchronous append is that boundary. A real sink needs its own analysis: if a worker queues an upload while another component later exposes it, “queued” and “published” are different events. A local lock alone does not establish when remote output becomes visible.

For this isolated example, the small lock-based design fits the requirement. It is not a complete solution for a remote publisher without defining that publisher's completion and visibility semantics.

What passing does and does not establish

Passing this test supports a specific conclusion: for the controlled schedule, the worker does not publish through the modeled sink after cancellation acknowledges success.

It does not establish:

• Immediate shutdown: Computation may continue without publishing.
• Rollback: Output published before acknowledgment may remain.
• Coverage of every schedule: This test exercises one selected interleaving, not every possible execution.
• Coverage of every side effect: Notifications, temporary files, retries, and other workers need separate contracts if they matter.

For a chunked export, place the same boundary test before a later chunk as well. Let an earlier chunk publish, acknowledge cancellation while the next chunk is paused, then verify that no additional chunk appears. That distinguishes “no further output” from the stronger, different requirement “no output ever existed.”

A passing test supports its boundary, not every possible execution.

Define the contract before requesting the implementation

In five minutes, write a Boundary Card for one AI-assisted change: name the observable effect, define the acknowledgment, describe a violating execution, and specify where the test must pause. If you cannot identify the publication boundary, resolve that ambiguity before asking an agent to code.

The card is a review aid, not proof of correctness. It turns one vague promise into a behavior you can challenge deliberately.

Use your card as a starting point with the ThunderLang getting-started guide.

Save the field guide

A field guide distilled from this article. The text version follows below.

Save the infographic (PNG)

Text version

• The promise is unclear: Success might mean only that the cancellation request was received. Define successful acknowledgment as blocking further publication for this job, or keep the weaker promise explicit.
• The race is uncontrolled: A brief sleep does not tell you where the worker was when cancellation happened. Pause the actual worker before its publication decision using ready and release events so the ordering is explicit.
• The worker is paused: The ready event confirms that the worker has reached the publication boundary. Await successful cancellation, release the worker, await completion, then assert no publication to test the promised boundary.
• The negative check passes: A worker that never publishes could also pass the no-output assertion. Verify publication without cancellation so a broken output path cannot satisfy the test.

Illustrative failure despite a working flag: A worker reads cancelled as false and pauses before publishing. The cancellation handler sets the flag and returns success, but the worker then publishes its prepared output.

Passing covers one controlled schedule and one modeled sink, not every execution. A local lock alone does not establish when remote output becomes visible.