How to Create an AI Agent: A 21-Case Local Release-Brief Worker

The first AI agent worth keeping is not the one that can call the most tools. It is the one whose objective, authority, budget, and output can be read before the first model request leaves the machine. That sounds less impressive than an autonomous demo. It is also the difference between a repeatable worker and an open-ended script with a chat model attached.

To make that claim testable, a small release-brief worker was specified around one fictional repository and one commit. Its job is narrow: read a manifest, a test summary, and a change summary, then write one review packet. A deterministic probe exercised 21 control cases covering admission, tool boundaries, loop budgets, duplicate calls, replay, and packet completeness. All 21 matched their expected outcomes.

This is a guide to how to create an AI agent by building the control shell first. The probe did not call a model, clone a repository, contact CI, publish a release, or deploy anything. It tests the part beginners are most likely to skip: the finite contract that remains when model behavior is uncertain.

Design decision: let a model propose the next step later. Keep admission, tool execution, approval, budgets, deduplication, and the final evidence packet in ordinary deterministic code from day one.

The control shell is the first working agent

An agent needs more than a prompt. It needs a job that can finish, a set of allowed observations, a bounded set of actions, and an owner who can tell whether the result is usable. For the local worker, those pieces fit on one page:

This shell is already useful without intelligence. It can reject a missing repository, a malformed commit, an unapproved destination, a duplicate write, or an incomplete packet. Adding a model later replaces the proposal mechanism inside the shell; it does not replace the shell itself.

That separation is the seam.

Evidence boundary: what 21 green cases do not prove

The retained fixture hashes to 62f1e43fb8eb090d5fcb56ff7a61f6d23b9622befc28f1f17e0004a4b8510dbe. The probe ran 21 cases and produced zero mismatches: nine run-admission cases, five tool-policy cases, three loop cases, two deduplication cases, and two packet cases.

Those numbers are control evidence, not an agent benchmark. No language model planned a step or wrote a summary. No repository, Git command, CI provider, network request, secret, deployment, or production account was involved. The run says nothing about reasoning quality, factual accuracy, latency, model cost, provider reliability, or whether a generated release recommendation would be correct.

That limitation is useful. A failed model evaluation and a failed authority check are different defects. By testing the envelope without a model, the team can identify whether a bad run came from planning, evidence, policy, or execution rather than filing everything under “the agent hallucinated.”

One job, one repository, one packet

A beginner project usually starts with a broad request: inspect my code and tell me whether it is ready. The hard part is not wording a clever system prompt. The hard part is turning “my code” and “ready” into inputs that cannot change under the worker.

The release-brief contract requires a repository identifier, a commit SHA, and three named evidence inputs. The output is one JSON packet with a stable path. If any of those values is absent or outside the assignment, the run is denied before a tool executes.

{
  "objective": "produce_release_review_packet",
  "repository": "demo/service-api",
  "commit": "4d7a8f1...",
  "inputs": ["manifest", "test_summary", "change_summary"],
  "output": "out/release-review.json",
  "limits": { "turns": 6, "toolCalls": 4 }
}

Commit binding matters because an agent can otherwise inspect one revision and report on another. The same rule applies outside software delivery: bind an invoice worker to an invoice ID and document version, a support worker to one ticket snapshot, or a research worker to a frozen source manifest. “Current” is convenient for a demo and miserable during an audit.

Scope is a data type.

The smallest useful loop has five states

The worker loop is intentionally plain: admit the run, propose a step, authorize the proposed tool, record the result, and decide whether to continue. A terminal packet ends the loop. A repeated state fingerprint or exhausted budget ends it without success.

state = admit(run)
while state is active:
    proposal = planner(state)          // model may live here later
    decision = authorize(proposal)
    result = execute(decision)
    state = record(state, proposal, result)
    if complete(state) or repeated(state) or exhausted(state):
        stop(state)

The model does not get to decide whether its own call is authorized. It may ask for read_test_summary; ordinary code checks the repository and commit binding before execution. It may ask to publish a release; ordinary code converts that request into APPROVAL_REQUIRED. A model can be persuasive without being permitted.

A turn budget alone is insufficient. A worker can alternate between two unproductive calls until the counter ends, consuming money without explaining why. The local shell fingerprints the meaningful state: objective, evidence already collected, pending tool, and output status. Seeing the same fingerprint again produces a named repetition stop. That is much easier to diagnose than a generic timeout.

Tools are a typed authority boundary

Function calling is often introduced as a convenience: describe a function and let the model invoke it. In an agent, the tool schema is also an authority boundary. Its types should make unsafe ambiguity difficult to express.

The three read tools require the admitted repository and commit. The packet writer accepts a validated packet and one exact destination. Publishing, merging, and deploying are not hidden aliases or clever prompt rules; they are separately named consequential capabilities that always require approval.

read_change_summary({ repository, commit })
write_review_packet({
  path: "out/release-review.json",
  packet: validatedPacket
})

// These never inherit permission from the review objective:
publish_release(...)  -> APPROVAL_REQUIRED
merge_branch(...)     -> APPROVAL_REQUIRED
deploy_production(...) -> APPROVAL_REQUIRED

Avoid one universal tool such as run_command(command) for the first version. It erases the distinction between reading a manifest and modifying a deployment. If a shell tool is unavoidable, place a deterministic command policy in front of it and test exact executable-and-argument shapes. A prompt that says “be careful” is not a policy engine.

Failure modes the harness keeps visible

The probe does not turn failures into prose. It gives each failure a stable decision that the caller can handle.

Named failures are a product feature. They let the UI offer a relevant next action, let metrics separate scope mistakes from planner loops, and let an operator see whether a run stopped safely. “Something went wrong” is not enough for a system that can act.

Testing twenty-one cases before adding a model

The test order matters. Begin with invariants that do not depend on natural-language quality:

1. Admit the exact repository, commit, inputs, and output; deny nearby shapes.
2. Allow the three reads and one packet write; deny unknown tools.
3. Pause consequential capabilities with the full proposed arguments.
4. Stop completed, repeated, and over-budget runs deterministically.
5. Write a complete packet once; make the same request idempotent.

Then add a model behind the planner interface and grade what changed: tool selection, claim support, summary quality, recovery from missing evidence, latency, token use, and variance across repeated runs. Keep the original 21 cases. A better planner must not loosen the authority contract that existed before it.

The release packet itself needs a schema. For this worker it must name the repository and commit, list evidence consumed, report test status, summarize changes, identify blockers, state a recommendation, and retain the run ID. A paragraph that “sounds complete” cannot substitute for those fields.

Troubleshooting loops, retries, and partial evidence

When the worker repeats a read, do not immediately increase the budget. Inspect the state fingerprint and the planner input. The result may not have been recorded, the evidence may use a different identifier, or the planner may not know that the goal is already satisfied. More turns amplify all three defects.

Retries need two identities: a run ID for the overall job and an operation ID for each side effect. Re-reading an immutable test summary is cheap and naturally repeatable. Writing the review packet should use the same operation ID on retry so a network interruption cannot create two competing artifacts. A future publish action would need its own idempotency contract and human approval; it must not borrow the packet writer's authority.

A retry is not permission.

Partial evidence should remain visible. If the manifest and change summary load but the test summary does not, the worker can produce an incomplete evidence record and a BLOCKED recommendation. It should not infer green tests from a familiar repository or rewrite “missing” as “probably passed.” An agent that can say exactly what it did not observe is more useful than one that always finishes.

Missing evidence is evidence.

A seven-day pilot should measure the review loop

The first pilot does not need broad autonomy. Give the worker read-only access to a small set of repositories and keep the packet write in a dedicated output store. Require human review for every recommendation. Track:

• admission denials by reason;
• tool calls per completed packet;
• repetition and budget stops;
• missing-evidence rate;
• packet schema failures;
• unsupported claims found by reviewers;
• human acceptance, correction, and rejection rates;
• latency and model cost after a planner is connected.

Sample failed and successful runs together. Looking only at accepted packets hides the cases where the worker was most dangerous or most confusing. If reviewers frequently repair the same field, improve the evidence contract or tool output before tuning the prompt.

For a managed runtime, create an AI agent on GolemWorkers only after the same objective, tool, approval, budget, and evidence boundaries are written down. Hosting can simplify process supervision and deployment. It cannot decide what the worker is allowed to do.

The decision record

The 21-case local result supports one narrow decision: the release-brief control shell is coherent enough to place in front of a planner experiment. It does not support unattended release approval, production access, or a claim that the worker understands a repository.

The tradeoff is deliberate. Building the shell first produces a less theatrical demo because most early work ends in denials, schemas, and stop reasons. In return, the first model run enters a system with a finite job, typed tools, visible failure states, and an auditable artifact. That is a better place to improve intelligence than an unrestricted loop.

Keep the fixture and its hash with the design record. When a tool, budget, packet field, or approval boundary changes, change the fixture and rerun the deterministic suite before touching model evaluations. The shell is the contract; the planner is one replaceable component inside it.

Where the implementation claims come from

• Anthropic: Building effective agents — the distinction between predefined workflows and model-directed agents, the value of simple composable patterns, and the need for environmental feedback and stopping conditions.
• OpenAI Agents SDK: Agents — instructions, models, tools, handoffs, and output types as agent configuration.
• OpenAI Agents SDK: Tools — function tools and typed schemas for exposing capabilities.
• OpenAI Agents SDK: Guardrails — checks around agent input, output, and tool calls.
• OWASP: Agentic AI threats and mitigations — layered controls for agentic systems and their tool-mediated attack surface.
• NIST AI RMF Playbook — Govern, Map, Measure, and Manage activities for documenting and operating AI risk controls.