# Give coding agents checks they can learn from

> Clear boundaries and enforceable checks make an agent's mistakes easier to find. Keep their limits visible, and inspect changes to the checks themselves.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-10-28 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/agent-trust-checks/

I would rather make a rule checkable than keep asking a coding agent to remember it.

“Never create duplicate records” sounds clear. But every caller that writes records can interpret it differently. One checks first, another assumes someone else checked, and a third forgets.

A constraint at the place where the records are stored gives those paths a common answer. When the agent makes a mistake, it gets something concrete to investigate.

This is useful even when you work without AI. Coding agents make the benefit easier to notice because they can produce several plausible implementations before you finish reviewing the first one.

The question I would ask is where an incorrect implementation becomes visible. If the answer depends entirely on someone remembering a sentence in a project file, there is probably room for a better check.

## What does “learn from” mean here?

A failed test gives an agent information it can use during the current task. It can inspect the failing case, change its implementation, and run the check again.

That does not mean the model permanently learns your project's rule. In a later session, it may need the same instructions and feedback again. Keeping the check in the repository makes that feedback available whenever the relevant command runs.

The useful loop has a specific disagreement at its center. The implementation says an import succeeded. The check says two records now represent the same supplier product. The agent has to resolve that disagreement.

A vague instruction to “improve reliability” gives it much less to work with.

```mermaid
flowchart TD
  accTitle: How a check helps an agent correct a mistake
  accDescr: A written rule becomes an observable check. A failing check gives the agent evidence for a repair, followed by another run and human review.
  A[Write rule] --> B[Run check]
  B --> C{Matches?}
  C -->|No| D[Inspect]
  D --> E[Repair]
  E --> B
  C -->|Yes| F[Review both]
```

Human review still matters at the end. A check can pass because the implementation improved, because the check became weaker, or because the test never exercised the important path.

## Start with one precise rule

Imagine an application importing products from a supplier.

The supplier gives each product a stable code. Two records with the same supplier and product code would represent a duplicate. Another supplier can use the same code for a different product.

That last sentence changes the implementation. A constraint on the product code alone would reject valid products from other suppliers.

Before asking an agent to implement the import, I would write down the rule this way:

> Every product has a supplier identifier and a product code. The pair identifies one product. Importing that pair again must not create another product.

There are still decisions to make. Should a repeated import update a product's name? Should an older file be allowed to overwrite newer information? What should the user see when the same file is imported twice?

Those are product decisions. The uniqueness rule gives us a foundation, but it doesn't answer them.

For this example, let's keep the first stored product unchanged when the pair arrives again. We will report the repeated row as already present. A production importer may need an update policy, but we should choose it deliberately before adding an upsert that overwrites fields.

## Put the rule at the storage boundary

An application can ask whether the product exists before inserting it. That helps with a friendly response, but a separate check and write leaves room for another caller to insert the product between those operations.

The database should enforce uniqueness too.

Here is a small SQLite table. Run this in a disposable SQLite database; the example deliberately attempts invalid writes later.

```sql
CREATE TABLE products (
  id INTEGER PRIMARY KEY,
  supplier_id TEXT NOT NULL,
  product_code TEXT NOT NULL,
  name TEXT NOT NULL,
  UNIQUE (supplier_id, product_code)
);
```

The uniqueness constraint covers the pair. The two `NOT NULL` declarations make both parts required.

SQLite's [table documentation](https://www.sqlite.org/lang_createtable.html) explains that `UNIQUE` treats `NULL` values as distinct, including from other `NULL` values. A uniqueness constraint alone would therefore leave a hole in the intended rule if either identifier could be missing.

`NOT NULL` doesn't reject an empty string. Nor does this schema decide whether `pump-42` and `PUMP-42` identify the same product. For the example, we treat codes as case-sensitive and require the application to reject empty identifiers.

The [free SQLite course](https://flaviocopes.com/courses/sqlite/) covers constraints and the other database basics behind this example.

That is an important boundary to keep visible. “Required identifier” can mean more than “SQL value is not null.” If other writers can bypass application validation, decide whether those additional rules also belong in the schema.

We can now insert a product:

```sql
INSERT INTO products (supplier_id, product_code, name)
VALUES ('north-parts', 'PUMP-42', 'Water pump');
```

Running that statement again produces a uniqueness error. Inserting `PUMP-42` for a different supplier is allowed. Setting either identifier to `NULL` produces a different error.

The agent now has observable behavior to preserve. It can change the import code, but an ordinary insert cannot silently create the same pair twice while that constraint is present.

The illustration follows the same repeated product through the different checks. Each layer answers a different question. The final scene shows the response and the stored result, because both matter.

## Decide what a repeated import should do

A rejected database write is evidence that the rule worked. It is still the application's job to turn that result into the intended import behavior.

For our “keep the first record” policy, SQLite supports a targeted conflict handler:

```sql
INSERT INTO products (supplier_id, product_code, name)
VALUES ('north-parts', 'PUMP-42', 'Replacement water pump')
ON CONFLICT (supplier_id, product_code) DO NOTHING;
```

The original row stays unchanged. The [SQLite UPSERT documentation](https://www.sqlite.org/lang_upsert.html) explains how the conflict target selects the uniqueness rule. This handler doesn't turn a missing required value into a valid row.

The importer also needs to distinguish an inserted row from a repeated one using its database driver's result. It should report those outcomes honestly rather than treating every completed statement as a new product.

A response like “1 inserted, 1 already present” is more useful than “2 imported” when only one product was created.

Be careful about broad error handling. Catching every database error and calling it a duplicate can conceal a disconnected database or another constraint violation. Handle the expected conflict specifically, and let unexpected failures remain visible through the application's error path.

## Uniqueness doesn't make the whole operation safe to retry

Suppose two import workers receive the same product at about the same time.

Both can observe that it doesn't exist. The uniqueness constraint prevents both from successfully adding the same pair. Your application must still handle the resulting conflict or other database errors, including locking failures where relevant.

Now suppose importing a new product also sends a notification. Preventing a duplicate product doesn't automatically prevent a duplicate notification.

A worker could commit the product, send the notification, and fail before recording that it finished. Retrying the operation raises a separate question about what may happen again.

**Idempotency** means repeating an operation has the same intended effect as performing it once. You have to define the operation's boundaries before claiming that property.

For our small example, we are checking the stored product and the import response. If notifications, billing, or external API calls enter the workflow, they need their own retry and recovery design. A unique product key proves nothing about those effects.

An agent asked to “make imports idempotent” can otherwise stop after adding a database constraint and leave the rest of the workflow unchanged.

## Use types to make callers clearer

A narrow function interface helps the agent see what an operation needs. An import function can require a supplier identifier instead of accepting an arbitrary bag of properties.

Static checks then catch some incorrect calls while the agent works. They can also help you find the callers affected by a change.

But supplier data arrives at runtime. A TypeScript annotation doesn't inspect the JSON that arrived over the network.

The [TypeScript handbook](https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#type-assertions) explicitly explains that type assertions disappear at compilation and add no runtime checking. Writing `payload as Product` doesn't establish that the supplier sent a product.

I would follow the value from its source to the database write. Where do we check that the payload is an object? Where do we reject missing or empty codes? Does the supplier identifier come from trusted application context, or can the incoming file choose another supplier?

That last question goes beyond data shape. A valid-looking identifier doesn't establish that the current user is allowed to import for that supplier.

The layers complement each other:

| Layer | What it can establish | What still needs attention |
|---|---|---|
| Written instructions | The intended product behavior and its reason | Whether any execution path follows it |
| Static types | Whether checked code uses compatible interfaces | Untrusted runtime values and authorization |
| Runtime validation | Whether incoming values meet the chosen rules | Races with other writers |
| Database constraints | Whether a stored write violates an enforced invariant | The response and external side effects |
| Integration tests | Whether selected scenarios behave as expected | Cases and environments the tests don't cover |

You don't need a custom framework to connect these layers. Start with the tools the project already uses and make their responsibilities clear.

## Give the agent useful failure information

A check that only prints “import failed” leaves the agent guessing.

For a failing test, I want the name of the scenario, the expected result, and the actual result. For example:

```text
Scenario: repeat the same supplier product
Expected: 1 stored product; second result is already present
Actual: 2 stored products; second result is inserted
```

With that input, the agent can reproduce the extra row. After fixing it, there is still a response assertion to inspect: did the importer report the repeated row correctly?

Production diagnostics need a different level of detail. Record enough to locate the failing operation without dumping whole supplier files, credentials, or customer data into logs. A row number and an internal import identifier may be enough to find the relevant input through the application's normal access controls.

Project instructions should explain how to run the focused check and why its expectation exists. Avoid a rule that says only “tests must pass.” An agent can satisfy that sentence while misunderstanding which behavior you care about.

## Test the failure before accepting the fix

For a bug repair, a useful check should expose the original problem.

If the report says repeated imports create duplicates, first reproduce that behavior against the old implementation. Then apply the change and run the same scenario again.

This helps catch a misleading test. A mocked importer that always returns success won't reveal what happened in the database. A test that uses a new random product code on every run won't exercise a repeated import either.

Use a disposable database with the real schema and the actual import path where practical. Keep the data small enough to explain by inspection.

For the example, I would check these cases:

| Input | Expected stored result | Expected response |
|---|---|---|
| First `north-parts` / `PUMP-42` | One new product | Inserted |
| Same pair again | Still one product | Already present |
| `south-parts` / `PUMP-42` | A separate product | Inserted |
| Missing supplier or code | No new product | Invalid input |
| Repeated pair with a different name | Original name retained | Already present |

The last case protects the policy we chose earlier. Without it, an agent could replace “do nothing” with “update every field,” pass a row-count assertion, and still change the product's behavior.

Test a failed batch too. Decide whether the whole file must succeed together or whether valid rows can be imported while invalid rows are reported. Then inspect the stored rows after a failure. A message saying the import failed doesn't tell you whether some writes were committed.

## Review changes to the checks themselves

An agent working in a repository may be able to edit tests, remove a lint rule, or change a database migration.

A passing command needs to be read alongside that diff.

```mermaid
flowchart TD
  accTitle: Review a passing agent change
  accDescr: A passing check leads to inspection of the implementation and verification changes, followed by an independent reproduction before acceptance.
  A[Checks pass] --> B[Code diff]
  B --> C[Check diff]
  C --> D{Same rule?}
  D -->|No| E[Repair]
  E --> A
  D -->|Yes| F[Reproduce case]
  F --> G[Rows and reply]
```

A changed test can be correct. Perhaps the product requirement changed, or the old test asserted an internal implementation detail. The review question is whether the new expectation still matches the behavior you approved.

For our importer, dropping the uniqueness constraint would deserve an explanation. So would deleting the duplicate case, widening a type to `any`, or returning success from a catch block that previously reported errors.

Run a small reproduction yourself with input you understand. If you can inspect the resulting rows independently of the importer's own summary, you get another observation beyond the agent's report.

## Separate feedback from permission

A local check helps the agent correct its work. A required check in CI can also prevent a change from being accepted when that check fails, provided your repository's merge rules require it.

Neither arrangement should be confused with limiting what the agent can do.

If a coding process can edit the check and deploy directly, the check alone cannot prevent it from bypassing the workflow. Permissions determine which resources and actions are available. Review and deployment rules determine which changes reach users.

I would give a task access to the disposable data it needs for verification. Running a duplicate-import experiment shouldn't require production write credentials.

For a change that touches real data, ask what can be recovered before granting access. A successful test on a fresh database doesn't prove a migration will work on years of existing rows. Duplicates may already exist, and adding a constraint requires deciding which records to keep and how references should be handled.

That decision belongs in a reviewed migration plan. Silently deleting rows to make the new constraint pass would be a substantial change to the task.

## Keep the checks affordable to maintain

It is possible to build a project where every small edit requires navigating a framework of custom rules.

I would start with the mistake that repeatedly costs time. Use a database constraint, an existing compiler option, or a focused integration test where it fits. Put a short explanation beside the check when its purpose is easy to misunderstand.

[Fast tests for coding agents](https://flaviocopes.com/fast-tests-for-agents/) goes further into choosing feedback that an agent can use while it works.

Keep fast checks easy to run during the task. Run the broader checks before accepting the finished change. If a check fails intermittently, investigate it; teaching the agent to rerun until it gets green weakens the feedback you wanted.

Some important questions remain difficult to automate. A test can establish that the importer preserves a name. It can't decide whether customers would prefer the latest supplier name instead. A constraint can enforce a pair of identifiers. It can't tell you whether the supplier's codes stay stable across catalog revisions.

For your next agent task, pick one important rule and deliberately violate it in a safe environment. Follow the failure all the way to the response and the stored result. If the agent can reproduce that failure, explain its cause, and show the same check passing after a focused repair, you have something concrete to review.