Applied AI · Field guide

Build on learning.

Learn how work happens, guide an agent through it, and turn repeated lessons into tested rules and reusable code as the workflow evolves.

Start with the work you want to improve

A team does not need to apply AI to every stage of software development to get value from it. It needs to understand where effort goes, where quality suffers, and which constraints keep useful work from moving. Sometimes the next improvement is a clearer handoff, a simpler process, or a small script. Sometimes an agent can help with work that previously required too much interpretation to automate economically.

My starting point is a job the team already needs to do. Perform it manually, understand what makes it difficult, and improve it through repeated use. The first version can depend on considerable human attention. That attention is how we discover what the system eventually needs to handle.

Observe the manual process

Choose a specific job: investigate a test failure, prepare a change for review, reconstruct a customer issue, or assess a migration. Follow it from the original request through the accepted result. Capture decisions, source material, tool use, handoffs, corrections, and exceptions. Include the time spent checking and repairing the work.

Approved local tools or cloud services can record useful traces automatically, such as commands, test results, task history, and review comments. Be deliberate about what is recorded, who can inspect it, and how long it is kept. Exclude credentials and unrelated personal or confidential material. People should understand and control participation; recording work for improvement should not quietly become employee ranking.

Use the record to ask whether the job was done well and where effort was wasted. A slow step may require judgment, or it may be repeated context gathering that a tool could handle. A record shows what happened; it still needs interpretation from the people who understand the work.

Guide an agent through the same job

Give the agent the goal, relevant context, available tools, boundaries, and examples of an acceptable result. Start by directing the work closely. Correct misunderstandings, supply missing context, and explain why a proposed result does not meet the need.

Keep those corrections alongside the task and outcome. They reveal where instructions were ambiguous, evidence was missing, or the workflow itself was poorly understood. Avoid treating every failure as a prompt problem: the cause may be missing access, an unreliable tool, a bad process, or a task the agent cannot perform well enough.

For example, an agent investigating a recurring test failure might initially attribute it to a recent code change. An engineer checks the environment and discovers a fixture mismatch. The useful lesson is to distinguish code, fixture, and environment evidence before assigning a cause. Preserve that lesson with the example rather than merely telling the agent to be more careful.

Turn repeated corrections into dependable checks

Begin with guidance people can read and revise: instructions, examples, task-specific procedures, and clear exception handling. As the same task recurs, separate context-dependent judgment from rules that must always hold.

Where a rule is stable and precisely testable, encode it in a deterministic check: ordinary code that evaluates a defined condition. For a change-preparation workflow, that might verify that a required test command actually passed, that a referenced file exists, or that a patch stays within an allowed area. Enforce the check where it matters rather than relying on the agent to remember to run it.

Test a new check against examples that should pass and examples that should fail. A small check can enforce one important condition; passing it does not prove the whole task is correct. Keep human review for the parts that remain matters of judgment. Treat task-specific preferences as revisable guidance rather than making every correction a permanent rule.

Replace settled repetition with code

Repeated runs also reveal work that no longer needs an agent to reason through it. Extract stable steps into scripts or services: gather known inputs, normalize a report, run a prescribed comparison, or validate a result. Give the agent a reliable operation it can call instead of asking it to reconstruct the same sequence every time.

This can leave the agent doing less routine execution and more interpretation, coordination, and exception handling. Measure the whole job, including supervision, rework, runtime cost, maintenance, and downstream review. Faster generation is useful only if it improves the accepted result or the effort needed to reach it.

Build a learning loop around the workflow

With enough structure, parts of the improvement process can also be assisted. A system can group recurring corrections, find repeated tool sequences, propose instruction changes, generate candidate checks, and suggest reusable automation. Preserve the connection between each proposal and the work that motivated it.

Review those proposals, evaluate them against representative cases, and version the instructions and code before widening their use. Retain a way to reverse a poor change. An agent’s proposed improvement is a hypothesis to test; frequent repetition alone does not establish that the underlying practice is right.

Keep the pieces replaceable: task guidance, tools, evaluations, deterministic checks, and authority boundaries should be understandable independently. Humans need ways to intervene, explain an exception, revise the goal, and decide that a learned rule no longer applies.

Expect the bottleneck to move

A workflow that produces implementation drafts quickly may overwhelm review. Faster investigation may expose a slow product decision. Better automation may make a previously uneconomic customer service possible. Those changes can justify a different process or even a different goal.

Periodically step back from the individual agent. Is this still the right job? Does the team need the same handoffs? Which constraint now limits progress? Retire instructions and automation that preserve an obsolete workflow. More efficient execution can otherwise make the wrong process harder to question.

Keep development accountable

Apply the loop within the team’s software development lifecycle. Design decisions, code review, tests, acceptance, deployment, and operational ownership still need to work together. Assign authority deliberately and make exceptions visible. As the workflow earns greater autonomy, keep checks and human decision points where the consequences require them.

The same approach can apply to other business processes. AI expands the range of work we can attempt to improve because it can help interpret varied inputs and handle tasks that are difficult to specify fully in advance. The durable capability is learning how the work works, improving it with evidence, and changing the system when the team’s needs change.

Use the resource

AI-assisted engineering experiment canvas

Use this canvas to scope one iteration of the learning loop. Pair it with records of manual work, agent corrections, proposed guidance, and the checks that establish whether the changes help.

Download PDFDownload editable DOCX

PDF for printing or sharing. DOCX for adapting in your own tools.

Choose the next move that matches the situation

Work through a live decision

Bring the situation, evidence, and people needed to choose a useful next move.

Discuss a Working Session