Practicing engineering judgment with AI

A coding agent can give you a working implementation before you have decided whether it is the right change.

That makes it useful to pause before the first patch. Write down the problem in terms you can observe. Then decide what evidence would show that the change helped.

I think this is a good way to practice engineering judgment while using AI. You make a decision, let the agent investigate or implement it, and compare what happened with what you expected.

You can do this on ordinary work. A slow page, a broken import, or an awkward form gives you plenty to think about. You don’t need to design a distributed system to practice choosing between alternatives.

Let’s work through one example: a support application whose CSV export times out on large accounts. The application and all measurements below are hypothetical. The decisions are the kind you can practice in your own project.

Separate what happened from your explanation

“The export needs a background queue” already contains a solution.

“The export request returns an error for this account” gives us something to investigate.

I would start by recording the account’s approximate size, the selected filters, and the action that failed. Keep sensitive customer data out of the notes. A safe reproduction can preserve the relevant shape without copying private records.

Then establish what successful behavior means. Perhaps the customer needs every matching ticket, including old ones, in a file their spreadsheet can open. A fast download containing only the newest records wouldn’t solve that problem.

This gives us a small starting record:

The distinction between observation and explanation sounds obvious until you read a bug report that treats a suspected cause as established fact.

An agent can repeat the same mistake very convincingly. If the prompt says “fix the queue problem,” it has a reason to look for a queue problem. Give it the symptom and ask it to locate the failure before changing the architecture.

Ask for evidence that can change your mind

For this export, I would consider several possible causes.

The database query could be slow. The application could fetch extra information separately for each ticket. It could load the entire result into memory. The request could also be failing at an infrastructure boundary before the export finishes.

Each possibility suggests a different observation.

These are directions for investigation, not diagnoses. Finding a large allocation doesn’t establish that it caused the observed timeout.

I would give the agent a bounded task:

Reproduce the CSV export failure without changing its behavior. Identify where the request spends its time. Record the input size and the evidence supporting the likely cause. If the failure is outside the exporter, follow the request far enough to show that.

Notice that the task has a stopping point. We want enough evidence to choose a repair. We don’t need a performance study of every endpoint.

If the agent cannot reproduce the failure, that becomes the next problem. Compare the environments and input shapes. Don’t turn “I couldn’t reproduce it” into “the reported bug is fixed.”

Compare designs by the work they create

Suppose our hypothetical investigation finds that fetching rows is reasonably quick, but constructing the whole CSV in memory is the expensive stage.

Streaming the export is now worth investigating. It lets the application produce output progressively. But whether it reduces memory depends on the whole path: reading all rows into an array first can preserve much of the original problem.

A background job is another option. It can generate a complete file separately from the download request. That brings storage and job state into the design, plus a way for the customer to retrieve the result.

We could also restrict the date range, if that matches how customers use exports. That changes the product’s behavior, so it needs an explicit product decision.

A queue can be the right answer. I would want its obligations to be part of the choice, rather than discovering them after accepting the patch.

The illustration follows the streaming option. Its measurements are invented to show the reasoning, not results from a benchmark.

The decision remains open until the evidence supports it. If generating the file still exceeds the available request lifetime, reducing memory alone may leave the original failure unresolved.

Make a prediction before reading the patch

Once an implementation exists, it becomes easy to explain why its shape makes sense.

Try writing your concerns before seeing it. For our export, I would predict a few places where a streaming change could go wrong.

Rows could arrive in a different order. A pagination boundary could skip a ticket. The request could keep doing expensive work after the user cancels. An error halfway through could leave a file someone mistakes for a complete export.

Access checks still need to apply to every exported record. A performance repair that removes an account filter is a serious regression, even if the download completes quickly.

Those concerns help define the task:

Keep the existing export filters, columns, and permission rules. Address the observed buffering problem. Avoid unrelated changes. Show how the implementation handles a failure after output begins, and report any behavior the current download interface cannot communicate clearly.

You don’t need to guess every defect. The point is to give yourself an independent starting position. Later you can compare your concerns with the defects you found.

This also reveals questions you cannot yet answer. If you don’t know how the application signals an incomplete download, investigate it before deciding that streaming is acceptable.

Choose tests that could reject the implementation

A test that repeats the implementation’s assumptions gives weak evidence.

Suppose the exporter uses a helper that returns ticket rows. The agent mocks that helper, hands it five clean rows, and checks that five lines appear. That can help with formatting, but it leaves the real query and account filtering untouched.

I would use a small dataset whose expected result I can calculate without the exporter. Include records from another account and records outside the selected date range. Now an incorrect filter has something concrete to expose.

For CSV formatting, include a value with a comma, a quote, and a line break. Open the resulting file with a parser appropriate to the format. Counting newline characters is unreliable when a field can contain a line break.

Then test the larger input that triggered the original failure. A small correctness fixture cannot tell you whether memory growth has improved.

A complete verification report says what ran and what it showed. “All tests passed” leaves too much hidden when you don’t know what the tests exercise. The same distinction helps when choosing fast tests for coding agents.

My guide to debugging goes through the underlying habit of narrowing a failure. The same habit applies when an agent does much of the investigation.

Read the patch as a future maintainer

An implementation creates future work as well as current behavior.

Start with the request path. Find where the current user is established, where filters enter, and where the query applies them. Follow the data until it reaches the response.

Then look at the lifetime of resources. If the exporter opens a database cursor or a file, what closes it after success? What happens after an error or cancellation? A tidy happy path gives you only part of the answer.

For a retry, ask what can happen twice. Retrying a read is different from retrying a job that creates a stored file and sends a notification. Follow the side effects before accepting a broad retry wrapper.

For an added dependency, I want to understand why the existing platform couldn’t do the job. A dependency may be entirely reasonable. It still brings an interface someone must understand and maintain.

Read the tests and configuration changes too. Perhaps the implementation improved. Perhaps the timeout increased or the largest fixture disappeared. Both can produce a green result, with very different implications.

I would also ask the agent to explain one awkward path in plain language. For example: “The database fails after the first chunk has reached the customer. What does the customer receive, and which component records the failure?”

Then verify its explanation in the code. A confident description is useful as a map, but the behavior still needs checking.

Use disagreement to uncover assumptions

You don’t have to accept the agent’s first proposal to keep moving.

If it recommends a queue, ask what observation makes a queue necessary. If it recommends streaming, ask which part of the data path stays bounded. These questions give it something specific to investigate.

You can also ask for the strongest reason to reject the proposed approach. The answer may expose an assumption you hadn’t noticed, such as the export needing a stable view of changing records.

Be careful with asking another agent for approval without context. It may read the same clean patch and miss the same missing requirement. Give a reviewer the observed failure and the behaviors that must survive. Ask it to trace those through the change.

You still have to settle product tradeoffs. An agent can describe the implications of a download that occasionally produces partial output. It cannot decide on behalf of your customers whether that behavior is acceptable.

Decide how to ship and how to recover

A repair can pass local tests and still behave differently under real traffic.

For our export, I would decide what to observe after release before deploying it. Useful signals might include completed exports, failures, and the duration of large exports. Avoid putting exported ticket contents into diagnostic logs.

If the application already supports enabling the change for a small set of accounts, use that existing mechanism. Building an elaborate release system for one small patch can create more work than the repair needs.

Write down what would make you stop the rollout. “Watch it carefully” gives you little guidance. “Stop expanding access if exports lose rows or cross account boundaries” names behavior you can act on.

Recovery needs thought too. Reverting a code-only change can be straightforward. A change that introduces stored jobs, new database state, or a migration may require a different procedure. Check what the older version can still read before treating rollback as a button.

Your checks cover particular inputs and environments. Production observation can reveal a condition those checks missed, so the chart includes a recovery path after release.

Keep a short decision journal

After finishing, compare the result with the original plan.

Perhaps the implementation was quick, but you spent much longer discovering that the request never reached the export code. Perhaps you accepted a broad refactor and then had to untangle it to test one behavior.

Those observations suggest changes to your process. Next time you could verify the request path first, or ask for a minimal repair before discussing a redesign.

A useful note can fit in a few sentences:

I expected whole-file buffering to explain the failure. Measurements supported that for memory, but the download still hit the request deadline. Streaming improved one constraint and left another. For the next export issue, measure completion time against the deployed request path before choosing the design.

That is an invented journal entry for this example. Its value comes from recording what changed your mind. “Be more careful next time” would be difficult to act on.

Keep a successful prediction too. If you anticipated a missing account filter and the fixture caught it, that tells you which review habit helped. For a choice future maintainers need to understand, keep a decision record in the project as well.

Counting generated lines wouldn’t reveal either lesson. Neither would comparing how many minutes different people spent typing.

Practice on changes small enough to understand

For your next bug, write your expected cause before asking the agent to fix it. Include one observation that would make you reject that explanation.

After the investigation, compare the evidence with your prediction. If they differ, explain the difference before requesting an implementation. This keeps you involved in choosing the change.

For a second exercise, take a proposed dependency or service and list the responsibilities it adds. Who updates it? What happens when it fails? Which user-facing behavior depends on it? Then compare that with an approach using the project’s existing tools.

You can practice review on a patch you haven’t accepted yet. Choose one value entering from the outside, such as an account identifier or an import field. Trace where it is validated and every operation it influences. Ask the agent to do the same, then compare the paths you found.

When time is short, keep the exercise small. Spend a few minutes predicting the failure modes of one function, then inspect its error path. You can repeat that on real work without turning each task into a classroom assignment.

The next time you delegate a fix, keep one decision for yourself: write down what would convince you that the result is safe to accept. When the patch arrives, use that note before reading the agent’s summary.

Want me to talk about your product? You can sponsor this site.