A model, a tool, a skill, and a harness solve different problems. The distinction becomes useful when deciding who chooses the next action, what software executes it, and where a result is checked.

This glossary uses twelve terms to describe those responsibilities. Implementations and vendors draw their boundaries differently. The definitions below are working conventions rather than a universal stack. For a complete example of how the pieces interact, see AI agents and harnesses.

The twelve core building blocks

1. Large language models (LLMs)

What is an LLM?

A large language model (LLM) predicts a distribution over tokens from the supplied context. Generation selects tokens from that distribution according to the decoding strategy. It does not always choose the most probable token:

$P(w_t \mid w_1, w_2, \dots, w_{t-1})$

Architecturally, modern frontier LLMs are dense or mixture-of-experts (MoE) transformer decoders. They possess strong semantic understanding, pattern completion, code generation, and logical reasoning capabilities across natural language and structured formats.

Crucially, an LLM is passive, stateless, and computationally bounded:

• Passive: An LLM executes only when invoked. It does not initiate actions, maintain background processes, or wake itself up.
• Stateless: The model has no memory across API requests. The next inference needs relevant context supplied by the surrounding application. An API may retain conversation state or cached inputs, but those are service/runtime features rather than durable memory in the model weights.
• Computationally bounded: A model performs computation during inference, but it does not independently execute application code or retrieve external state. The host must supply tools and return their observations.

// Minimal LLM invocation: pure text in, text out
import { generateText } from 'ai'
import { openai } from '@ai-sdk/openai'

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Summarize the differences between optimistic and pessimistic locking in SQL databases.',
})

When to use raw LLMs

Use a raw LLM when the task is self-contained, stateless, and strictly bounded to language transformation:

• Text summarization, sentiment analysis, and style transfer
• Entity extraction from unstructured input into structured JSON schemas
• Translation across human languages or code dialects
• Single-turn reasoning or question answering over static, supplied text

2. Workflows

What is a workflow?

A workflow follows application-defined control flow, such as a pipeline or state machine. Its steps can call models, branch on results, or retry. The path is constrained by code, neither the outputs nor the latency are necessarily deterministic.

When to use workflows

Use workflows for high-volume, mission-critical business processes with strict compliance requirements:

• Invoice processing and KYC document verification pipelines
• ETL pipelines transforming unstructured text into tabular databases
• CI/CD build and automated deployment pipelines
• Regulated processes where auditability requires an invariant execution sequence

3. Autonomous agents

What is an agent?

An agent is an autonomous system that wraps an LLM in an iterative execution loop with access to external tools and memory to achieve a high-level goal.

$\text{Agent} = \text{LLM} + \text{Tools} + \text{Memory} + \text{Execution Loop}$

While a raw LLM answers a question, an agent pursues an objective. The user provides a target state (e.g., Fix the failing unit tests in the authentication service), and the agent autonomously reasons about the problem, chooses actions, inspects intermediate results, and self-corrects until the goal is achieved.

Core features of an agent

1. Autonomy: Operates independently without requiring human guidance at every micro-step. It determines the sequence and count of operations on the fly.
2. Goal-based: Driven by an explicit objective. Instead of blindly executing hardcoded procedural scripts, it evaluates whether each step brings it closer to the target state.
3. Adaptability: Dynamically recovers from failures. When a tool call returns an error, a file is missing, or a hypothesis is refuted, the agent inspects the feedback, alters its plan, and tries an alternative path.

The ReAct pattern: think, plan, act, observe

ReAct is one useful pattern for interleaving reasoning and actions. It is not a requirement for every agent architecture. In this conceptual loop, the model proposes the next action and the host executes it:

1. Think (Reasoning): The LLM analyzes the current goal, conversation history, and prior tool outputs to synthesize an updated internal state.
2. Plan (Action selection): The model decides which tool to invoke and formats the necessary arguments.
3. Act (Execution): The host runtime executes the tool (e.g., database query, shell command, API call).
4. Observe (Feedback ingestion): The execution output is appended back into the agent context window as a new message, triggering the next reasoning cycle.

// Conceptual implementation of an autonomous ReAct agent loop
interface Tool {
  name: string
  description: string
  execute: (args: Record<string, unknown>) => Promise<string>
}

async function runAgent(goal: string, tools: Tool[], maxIterations = 10) {
  const messages: Array<{ role: string; content: string }> = [
    {
      role: 'system',
      content: 'You are an autonomous agent. Reason, plan, and call tools to reach the goal.',
    },
    { role: 'user', content: goal },
  ]

  for (let i = 0; i < maxIterations; i++) {
    // 1. Think & Plan: Call LLM with tool schemas
    const response = await callLLMWithTools(messages, tools)

    if (response.isComplete) {
      return response.finalAnswer
    }

    // 2. Act: Execute selected tool
    const { toolName, args } = response.selectedToolCall
    const tool = tools.find((t) => t.name === toolName)
    if (!tool) throw new Error(`Tool ${toolName} not found`)

    // 3. Observe: Capture environmental feedback
    const observation = await tool.execute(args)

    // Feed back into loop
    messages.push({ role: 'assistant', content: `Tool call: ${toolName}(${JSON.stringify(args)})` })
    messages.push({ role: 'tool', content: observation })
  }

  throw new Error('Agent exceeded maximum reasoning iterations')
}

When to use agents

Use agents when the solution path cannot be statically predicted at compile-time:

• Interactive troubleshooting, debugging code, and bug fixing
• Exploratory data analysis and multi-step competitive research
• Automated incident remediation where diagnostic steps depend on telemetry
• Operating complex developer environments (e.g., executing shell commands, running test suites, writing patches)

4. Tools

What are tools?

Tools expose capabilities through interfaces the model can request, commonly a name, description, and input schema. A tool may perform deterministic arithmetic, query changing external state, or call another model. Its interface does not make its output deterministic.

An LLM cannot directly call a function. It generates a structured text payload indicating its intent to invoke a function along with the parsed parameters. The host runtime verifies the parameters, executes the actual code, and passes the output back to the model.

{
  "type": "function",
  "function": {
    "name": "query_database",
    "description": "Execute a read-only SQL query against the read-replica analytics cluster.",
    "parameters": {
      "type": "object",
      "properties": {
        "sql": {
          "type": "string",
          "description": "The SELECT SQL query to execute."
        }
      },
      "required": ["sql"]
    }
  }
}

Capabilities unlocked by tools

• Real-time information retrieval: Access live web data, private databases, or internal APIs.
• Deterministic computation: Delegate math, cryptographic hashing, or sorting to a native runtime instead of relying on token approximation.
• State mutation: Write files, deploy cloud infrastructure, issue GitHub pull requests, or send transactional alerts.

// Concrete tool declaration using Vercel AI SDK
import { tool } from 'ai'
import { z } from 'zod'

export const weatherTool = tool({
  description: 'Get current temperature and forecast for a geographic location.',
  parameters: z.object({
    city: z.string().describe('City name, e.g., San Francisco'),
    units: z.enum(['celsius', 'fahrenheit']).default('celsius'),
  }),
  execute: async ({ city, units }) => {
    const res = await fetch(`https://api.weather.internal/v1?city=${city}&units=${units}`)
    return res.json()
  },
})

When to use tools

Expose tools whenever an agent or LLM requires:

• Dynamic data that changes after the model's knowledge cutoff
• Exact arithmetic or domain logic that cannot tolerate hallucination
• Read/write access to external software ecosystems

5. Model Context Protocol (MCP)

What is MCP?

The Model Context Protocol (MCP) is an open standard designed to solve the $N \times M$ integration problem between AI clients (LLMs, IDEs, desktop agents) and external data sources or tools (databases, issue trackers, local filesystems).

Without MCP, every AI platform (ChatGPT, Claude, Cursor, custom company agents) must author proprietary integrations for every tool (GitHub, Postgres, Slack, Jira). MCP standardizes this interface via a universal client-host-server protocol built over JSON-RPC 2.0.

The relationship between MCP and tools: MCP as a standardized tool collection

A common point of confusion is the distinction between a tool and MCP.

An MCP server can expose tools, resources, and prompts through a negotiated protocol. A separate process is common, but process placement is an implementation choice:

• Tool (the atomic unit): A single function with a JSON Schema (e.g., git_commit or read_file). By itself, a tool is just an isolated function signature embedded in application code.
• MCP (the collection and protocol): An architectural boundary and packaging standard. An MCP server packages an entire suite of related tools (e.g., a GitHub MCP server exposes create_issue, merge_pr, fetch_diff, list_commits) and serves them over stdio or Streamable HTTP (or legacy HTTP/SSE transports) using standard JSON-RPC contracts.

┌────────────────────────────────────────────────────────┐
│ MCP Server: github-mcp-server                          │
│                                                        │
│  ├── Tools (Collection of callable operations)         │
│  │    ├── create_issue(title, body)                    │
│  │    ├── get_pull_request_diff(pr_number)             │
│  │    └── merge_branch(head, base)                     │
│  │                                                     │
│  ├── Resources (Read-only data endpoints)              │
│  │    ├── repo://tree/main/src                         │
│  │    └── repo://logs/ci-build-latest                  │
│  │                                                     │
│  └── Prompts (Predefined workflows & templates)        │
│       └── review_pr_template(pr_id)                    │
└────────────────────────────────────────────────────────┘

Instead of developers hardcoding 50 bespoke tool schemas inside their application codebase, an agent connects to an MCP server at startup, dynamically lists available tools via tools/list, and invokes them through tools/call.

Core primitives of MCP

MCP exposes three fundamental capabilities from servers to clients:

1. Tools: Executable capabilities requested by the model, with authorization and consent enforced by the host. That does not require a new approval prompt for every invocation.
2. Resources: Read-only contextual data, similar to REST GET endpoints or virtual filesystems (e.g., file contents, server logs, database tables).
3. Prompts: Pre-configured prompt templates authored by the tool provider for common workflows.

// Example: Creating a lightweight MCP Server exposing a collection of SQLite tools
import { Server } from '@modelcontextprotocol/sdk/server/index.js'
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js'

const server = new Server(
  { name: 'sqlite-mcp-server', version: '1.0.0' },
  { capabilities: { tools: {}, resources: {} } }
)

// Expose a collection of tools
server.setRequestHandler(ListToolsRequestSchema, async () => ({
  tools: [
    {
      name: 'execute_sql',
      description: 'Run SQL query on local SQLite database',
      inputSchema: {
        type: 'object',
        properties: { query: { type: 'string' } },
        required: ['query'],
      },
    },
    {
      name: 'list_tables',
      description: 'List all existing database tables and schema definitions',
      inputSchema: { type: 'object', properties: {} },
    },
  ],
}))

const transport = new StdioServerTransport()
await server.connect(transport)

When to use MCP

• Exposing enterprise data and tool suites once so all compliant AI tools across an organization can use them
• Decoupling agent application logic from third-party API integration code
• Running local, secure data connectors (e.g., inspecting local git repositories or SQLite stores) without passing API tokens to external SaaS vendors

6. Skills

What is a skill?

A skill is a packaged, reusable unit of procedural knowledge, domain conventions, and specialized instructions that teaches an agent how to accomplish a specific class of tasks effectively.

While a raw tool answers "what can I execute?" (e.g., git_commit, run_command), a skill answers "how should I behave to achieve this standard?" (e.g., how to write an idiomatic commit message, which architectural boundaries to respect, or which edge cases to verify).

Skills are typically defined as Markdown or declarative config files (such as SKILL.md in Claude or Codex, or semantic skills in Microsoft Semantic Kernel) containing:

• Domain prompt directives and reasoning steps
• Pre-execution validation checklists
• Required and prohibited tools
• Expected output schemas and formatting rules

Progressive disclosure: keeping context lean

A useful loading strategy for skills is progressive disclosure: load the description first, then the instructions and references when relevant. Specific loading behavior depends on the host.

In large codebases or enterprise environments, an agent may have access to 50+ specialized skills (e.g., security auditing, database migrations, React performance profiling, Kubernetes deployments). If an agent dumped every skill's full text into the system prompt on turn 1, it would consume substantial context before the user asks a question; the size depends on the actual files. This causes prompt bloating, context exhaustion, and severe attention dilution ("lost in the middle").

Progressive disclosure solves this by loading skills in three progressive tiers:

1. Tier 1: Catalog index (discovery metadata): Only a compact manifest is injected into the initial system prompt. Each skill contributes only its name, file path, and a short 1–2 sentence description of when it should be used (illustrative compact metadata).
2. Tier 2: On-demand activation (full instructions): When the agent receives a task that matches the skill's description (or when explicitly invoked by the user or an orchestrator), the host runtime reads the full SKILL.md body and appends it to the active context window.
3. Tier 3: Lazy sub-resource evaluation (deep references): Complex skills often link to supporting documents, validation scripts, or schemas. These remain on disk and are only read into context if the specific subtask demands them.

---
name: 'security-audit-skill'
description: 'Instructions and heuristics for auditing TypeScript codebases for OWASP Top 10 vulnerabilities. Use when reviewing code for security flaws.'
tools: ['read_file', 'rg_search', 'run_linter']
---

# Security Audit Playbook

When auditing code:

1. First, search for dangerous primitives: `eval()`, `dangerouslySetInnerHTML`, and raw SQL queries.
2. Verify all user inputs pass through a schema validator (Zod / TypeBox) before reaching database layers.
3. Check authentication headers and token expiration handling.
4. Categorize all findings by CVSS severity score (Critical, High, Medium, Low).

When to use skills

Use skills to institutionalize organizational playbooks and keep agent behavior consistent across projects:

• Codifying code-review standards or documentation templates
• Establishing runbooks for on-call triage and incident investigation
• Teaching agents domain-specific business rules (e.g., regulatory compliance checks)

7. Plugins

What is a plugin?

A plugin is a self-contained distribution and packaging mechanism that bundles tools, skills, assets, and authentication configurations into a single installable unit.

Plugins bridge software systems with agent runtimes. A GitHub plugin might package:

• 12 individual API tools (create_issue, merge_pr, fetch_diff, etc.)
• 2 specialized skills (PR review guidelines, branch naming conventions)
• OAuth credentials or API key configuration

{
  "name": "github-developer-pack",
  "version": "1.2.0",
  "description": "Comprehensive GitHub integration for coding agents",
  "tools": ["./tools/issues.js", "./tools/pull-requests.js"],
  "skills": ["./skills/pr-review.md", "./skills/conflict-resolution.md"],
  "auth": {
    "type": "oauth2",
    "scopes": ["repo", "workflow"]
  }
}

When to use plugins

Use plugins when packaging capabilities for distribution across developers, teams, or marketplace ecosystems:

• Extending agent IDEs (VS Code, Cursor, Codex) with 3rd-party services
• Enabling end users to toggle integrations (Slack, Jira, Google Drive) on and off
• Distributing internal tooling bundles across engineering teams

8. Multi-agent systems

What is a multi-agent system?

A multi-agent system (MAS) is a collaborative architecture where multiple specialized agents work together, each contributing unique skills, tools, and domain perspectives to solve complex problems that exceed the cognitive capacity of a single monolithic agent.

$\text{Multi-Agent System} = \sum (\text{Specialized Agent}_i) + \text{Communication Protocol}$

The primary driver: solving context explosion

Why not simply build one all-powerful monolithic agent?

The fundamental constraint is context growth. Irrelevant material can crowd out useful evidence. Context poisoning is a separate concern involving misleading or malicious content.

As an agent works through a long-horizon task (e.g., investigating a production bug across multiple microservices), it accumulates thousands of tokens of noisy shell outputs, stack traces, AST dumps, and exploratory thoughts. In a monolithic agent, this creates four severe points of failure:

1. Attention dilution ("lost in the middle"): Transformer attention degrades across bloated contexts. The model loses track of earlier system instructions, edge cases, and safety bounds.
2. Tool selection degradation: Presenting an LLM with 50+ tool schemas simultaneously causes tool hallucination, misformatted parameters, and invalid selections.
3. Growing input usage and latency: Long histories add input tokens to later calls. In an illustrative run with uncached history growing by 1,000 tokens per call, ten calls send 55,000 cumulative history tokens, while twenty send 210,000. That sum grows quadratically with call count under these assumptions, not exponentially. Actual billing and latency depend on caching, context selection, model, and provider.
4. Context pollution: Errors or speculative false hypotheses generated on turn 3 persist in context on turn 20, biasing the model into circular reasoning loops.

How separate agents can limit context growth:

• Context isolation: Each specialized sub-agent operates within its own small, pristine context window containing only the tools, skills, and system prompts relevant to its narrow mandate.
• Context distillation: When a sub-agent completes its subtask, it returns only a concise, high-level summary or final artifact (such as a unified diff or verified conclusion) back to the parent agent. The parent can avoid loading detailed intermediate output, while retaining references to evidence for later inspection.

Monolithic Agent Context (Bloated & Poisoned):
[System Prompt] + [50 Tools] + [Turn 1 Tool Output (4k tokens)] + [Turn 2 Log Dump (15k tokens)] ... -> 120k tokens

MAS Context Isolation:
Supervisor Context: [Goal] + [Task Decomposition] + [Worker 1 Distilled Summary (200 tokens)] -> 2k tokens
  ├── Worker 1 Context (Isolated): [Filesystem Tools] + [Local Search] -> Complete -> Returns 200 tokens
  └── Worker 2 Context (Isolated): [Testing Tools] + [Run Test Suite] -> Complete -> Returns 150 tokens

The engineering trade-offs of MAS

While separate contexts can reduce irrelevant material, it introduces significant engineering complexity:

• Coordination overhead: Building reliable inter-agent message formats, serialization schemas, and communication channels.
• Compounding latency: Sequential agent handoffs introduce multiple separate LLM round trips, multiplying wall-clock response times.
• Cascading errors & drift: If Sub-agent A generates a flawed distilled summary, Sub-agent B accepts it as ground truth and builds on bad assumptions without access to the discarded raw logs.
• System deadlocks & loops: Without strict termination conditions, two agents can debate endlessly or pass tasks back and forth in an infinite loop.

Sub-agent orchestration patterns

There are five major topological patterns for orchestrating sub-agents:

1. Hierarchical supervisor-worker (hub-and-spoke): A central supervisor agent decomposes the goal, spawns specialized child agents, passes targeted sub-prompts, and aggregates their distilled summaries. Workers never communicate directly with one another.
2. Sequential pipeline / chaining (handoff): Agents pass work as a structured baton along an assembly line (e.g., Architect $\rightarrow$ Developer $\rightarrow$ Tester $\rightarrow$ Technical Writer). Each agent refines the output of the predecessor.
3. Peer collaboration & debate (generator-critic): Two specialized agents work in an iterative loop. One agent generates candidate solutions (e.g., code patches), while an adversarial critic agent evaluates edge cases, security flaws, or test failures. They iterate until the critic approves or a turn threshold is reached.
4. Event-driven blackboard / shared memory (swarm): Agents do not invoke each other directly. Instead, they read from and publish to a centralized shared state board or message bus. Agents subscribe to task topics and claim subtasks matching their capabilities.
5. Dynamic fork-join (scatter-gather / parallel delegation): When an agent encounters independent sub-problems (e.g., searching three separate repositories or testing four hypotheses), it dynamically forks parallel worker agents, awaits their completion, and merges the disjoint write sets.

When to use multi-agent systems

• Complex software development spanning distinct stages (architecture, coding, test execution, documentation)
• Tasks with natural boundaries that benefit from parallel investigation across large codebases
• Systems where context explosion degrades single-agent reasoning over long execution horizons

9. Orchestrators

What is an orchestrator?

The orchestrator is the centralized coordination engine that governs multi-agent topologies, workflow execution, state persistence, and resource allocation.

While individual agents execute bounded loops, the orchestrator sits outside them:

• Routing: Inspects incoming tasks and assigns them to the appropriate agent or workflow.
• State & memory management: Maintains global execution state, tracks conversation history, and synchronizes shared memory across agents.
• Concurrency & scaling: Manages parallel execution, task timeouts, retries, and rate limits.
• Conflict resolution: Arbitrates disagreements when two agents produce contradictory conclusions or require the same exclusive lock.

// Architectural skeleton of a multi-agent orchestrator
class Orchestrator {
  private registry: Map<string, SpecializedAgent> = new Map()
  private sharedContext: Map<string, unknown> = new Map()

  registerAgent(role: string, agent: SpecializedAgent) {
    this.registry.set(role, agent)
  }

  async executeMission(goal: string): Promise<MissionResult> {
    const planner = this.registry.get('planner')!
    const plan = await planner.execute({ goal })

    for (const step of plan.steps) {
      const worker = this.registry.get(step.assignedRole)
      if (!worker) throw new Error(`Unassigned role: ${step.assignedRole}`)

      const stepResult = await worker.execute({
        task: step.description,
        context: this.sharedContext.get(step.dependsOn),
      })

      this.sharedContext.set(step.id, stepResult)
    }

    const synthesizer = this.registry.get('synthesizer')!
    return synthesizer.execute({ history: Array.from(this.sharedContext.entries()) })
  }
}

When to use an orchestrator

Implement an orchestrator whenever your architecture spans:

• More than two collaborating agents
• Asynchronous task queues with checkpointing, pause, or resume requirements
• Human-in-the-loop approvals interrupting long-running agent operations

10. Hooks

What are hooks?

Hooks are deterministic, lifecycle interception points that allow host applications to inject custom logic before, during, or after key agent events.

An LLM is inherently probabilistic, non-deterministic, and black-box. Hooks provide the deterministic software boundaries required to observe, inspect, mutate, or abort agent operations. Similar to Git hooks, Webhooks, or React lifecycle methods, agent hooks give software engineers application-defined interception points. Enforcement depends on which operations pass through those points.

Common agent lifecycle hooks

Modern agent frameworks (such as AI SDK, LangChain, or custom enterprise runtimes) expose hooks across the execution lifecycle:

Code implementation: a lifecycle hook registry

// Concrete implementation of an agent lifecycle hook registry
type HookFn<T> = (context: T) => Promise<T | void>

class AgentHookRegistry {
  private beforeToolCallHooks: HookFn<{ toolName: string; args: any }>[] = []
  private afterToolCallHooks: HookFn<{ toolName: string; result: any }>[] = []

  onBeforeToolCall(fn: HookFn<{ toolName: string; args: any }>) {
    this.beforeToolCallHooks.push(fn)
  }

  async triggerBeforeToolCall(toolCall: { toolName: string; args: any }) {
    for (const hook of this.beforeToolCallHooks) {
      await hook(toolCall)
    }
  }
}

// Registering hooks for security, governance, and logging
const hooks = new AgentHookRegistry()

// 1. Narrow capability authorization (illustrative policy).
// The dispatcher must also enforce this policy; a hook is not a process sandbox.
hooks.onBeforeToolCall(async ({ toolName }) => {
  const allowedTools = new Set(['read_parser_source', 'run_parser_checks'])
  if (!allowedTools.has(toolName)) {
    throw new Error('This task does not authorize that capability.')
  }
})

// 2. Budget and telemetry hook
hooks.onBeforeToolCall(async ({ toolName }) => {
  console.log(`[Telemetry] Agent invoking tool: ${toolName}`)
})

When to use hooks

Hooks are the foundational layer for:

• Implementing deterministic guardrails outside the model
• Emitting distributed tracing and OpenTelemetry metrics for every tool invocation
• Enforcing token, cost, and rate-limit budgets across long-running agents
• Injecting human-in-the-loop approvals before sensitive state mutations

11. Guardrails

What are guardrails?

Guardrails are programmable safety mechanisms, validation rules, and policy layers that evaluate inputs, intermediate actions, and outputs to prevent AI systems from taking inappropriate, insecure, or out-of-bounds actions.

Guardrails can reject specific invalid requests or reduce the chance of unsafe outputs. Their guarantees depend on their implementation and coverage; a model-based judgment or pattern filter is not a sandbox.

Why prompts are not guardrails

A pervasive architectural anti-pattern in AI engineering is relying on system prompts as security controls:

"You are a helpful assistant. You must NEVER run destructive shell commands, and you must NEVER reveal internal customer database records."

Prompt-based safety is fundamentally unreliable in production:

1. Vulnerability to prompt injection: Indirect prompt injections hidden inside external data (e.g., a malicious comment in a GitHub issue, text scraped from a webpage, or a row in a customer database) can instruct the model to ignore previous system instructions.
2. Probabilistic execution: LLMs are statistical token predictors, not deterministic logic engines. As a hypothetical calculation, a 1% violation probability per operation gives an expected 10 violations over 1,000 operations; the actual count is uncertain, and this is not a measured model rate. In enterprise software, a 1% failure rate for unauthorized data deletion is intolerable.
3. Context dilution: As conversation history grows into tens of thousands of tokens, system prompt instructions placed at the beginning suffer from attention degradation.
4. Zero physical enforcement: A prompt instruction cannot revoke a database user's DROP privileges, kill a runaway bash process, or isolate a socket connection.

Real safety requires deterministic code execution outside the model.

Implementing guardrails via hooks

Because prompts cannot enforce safety, guardrails are typically implemented directly inside lifecycle hooks:

1. Input guardrails (hooked at beforePrompt):
   • Flag suspected prompt injections before content enters context. Detection can miss attacks or reject benign content, so keep authorization independent of this filter.
   • Redact personally identifiable information (PII) before network transmission to external model providers.
2. Execution / tool guardrails (hooked at beforeToolCall):
   • Validate input structure with schemas, and enforce semantic restrictions separately. For database access, a fixed read-only query capability and appropriately restricted database credentials are stronger boundaries than a schema accepting arbitrary SQL text.
   • Enforce authorization policies (e.g., verifying the current user session has permissions to invoke refund_customer).
   • Sandbox filesystem paths and shell commands to prevent directory traversal or privilege escalation.
3. Output guardrails (hooked at afterModelCall):
   • Compare generated claims with retrieved evidence and report unsupported statements. This can reduce errors without guaranteeing correctness.
   • Scan responses for leaked API keys, tokens, or confidential data.
   • Validate structured output format compliance (e.g., Zod schema parsing) before passing payloads to downstream microservices.

The dispatcher should authorize a narrow capability, validate its arguments, and run it in an environment with the required filesystem and network restrictions. For example, run_parser_checks can map to a fixed command maintained by the application. A regular expression over arbitrary shell text cannot enforce the same boundary. Hook names in this article are illustrative, not a shared framework API.

MCP architecture and host responsibilities

When to use guardrails

Guardrails are non-negotiable in production:

• Any customer-facing application interacting with public users
• Any agent with write, execute, or financial mutation privileges
• Regulated industries (finance, healthcare, legal) with statutory privacy and audit obligations

12. Harnesses

What is a harness?

In classic software engineering, a test harness is a collection of software and test data configured to run a program unit under varying conditions while monitoring its behavior and outputs.

In this article, a runtime harness is the software around a model that supplies context, dispatches tools, records state, and enforces execution limits. An evaluation harness runs tasks and checks outcomes under controlled conditions. A system may share infrastructure between them, but the terms describe different responsibilities.

$\text{Harness} = \text{Environment Provisioning} + \text{Context Scaffolding} + \text{Execution Sandbox} + \text{Evaluation Engine}$

If the agent is the driver and tools are the steering wheel and pedals, the harness is the chassis, telemetry rig, and proving ground. The agent focuses solely on reasoning and choosing actions; the harness is responsible for setting up the external reality the agent perceives, enforcing operational limits, and verifying whether the agent's work actually solved the problem.

Core capabilities of an AI harness

An agent harness fulfills four core engineering functions:

1. Environment provisioning and sandboxing:

   • Spawns isolated, reproducible workspaces (e.g., ephemeral Docker containers, microVMs, git worktrees, or restricted local scratch directories).
   • Enforces filesystem read/write boundaries and network isolation policies to prevent runaway agents from corrupting host environments or exfiltrating data.

2. Context injection and prompt scaffolding:

   • Assembles and injects the baseline context before the agent's first turn: system prompts, workspace path definitions, environment variables, tool declarations, repo-level instructions (such as AGENTS.md), and available skills.
   • Shields the agent loop from manual setup boilerplate while ensuring predictable, reproducible initial state.

3. Runtime governance and telemetry:

   • Enforces execution budgets: maximum reasoning iterations (turns), token consumption limits, wall-clock timeouts, and financial cost caps.
   • Captures high-fidelity execution traces: recording observable model responses, proposed tool calls, tool results, and intermediate artifacts for replay, debugging, and post-mortem analysis.

4. Automated verification and evaluation (eval harness):

   • In benchmark and testing modes (such as SWE-bench, GAIA, or internal regression testbeds), the harness executes deterministic assertions after the agent concludes its run.
   • Runs test runners (pytest, jest, pnpm test), compiles code, checks git diffs, or runs an LLM-as-a-judge grader to produce a recorded result. A model-based grader remains an uncertain judgment; deterministic checks also establish only the conditions they actually test.

The evaluation and runtime loop

Below is a conceptual implementation of an evaluation and runtime harness running an agent against a sandboxed git repository and verifying the fix with automated tests:

import { execSync } from 'child_process'
import { runAgent } from './agent'

interface HarnessConfig {
  taskPrompt: string
  repoPath: string
  testCommand: string
  maxTurns: number
  tokenBudget: number
}

interface BenchmarkResult {
  passed: boolean
  totalTurns: number
  tokensUsed: number
  patch: string
  logs: string[]
}

async function runInHarness(config: HarnessConfig): Promise<BenchmarkResult> {
  // 1. Provision isolated git worktree
  const worktreePath = `/tmp/agent-worktrees/${Date.now()}`
  execSync(`git worktree add ${worktreePath} HEAD`, { cwd: config.repoPath })

  const telemetry = { tokens: 0, turns: 0, logs: [] as string[] }

  try {
    // 2. Inject context and run agent within execution sandbox
    const agentResult = await runAgent({
      goal: config.taskPrompt,
      cwd: worktreePath,
      maxTurns: config.maxTurns,
      tokenBudget: config.tokenBudget,
      onStep: (step) => {
        telemetry.turns++
        telemetry.tokens += step.tokenUsage
        telemetry.logs.push(`[Turn ${telemetry.turns}] ${step.summary}`)
      },
    })

    // 3. Extract generated patch
    const patch = execSync('git diff', { cwd: worktreePath }).toString()

    // 4. Run deterministic verification suite
    let passed = false
    try {
      execSync(config.testCommand, { cwd: worktreePath, stdio: 'pipe' })
      passed = true
    } catch {
      passed = false
    }

    return {
      passed,
      totalTurns: telemetry.turns,
      tokensUsed: telemetry.tokens,
      patch,
      logs: telemetry.logs,
    }
  } finally {
    // Teardown worktree and sandbox
    execSync(`git worktree remove --force ${worktreePath}`, { cwd: config.repoPath })
  }
}

When to use a harness

A harness is essential in four primary engineering scenarios:

• Benchmarking and evaluation: Measuring agent accuracy, tool-calling precision, and completion rates across standardized suites (e.g., SWE-bench, HumanEval, GAIA) or proprietary test sets.
• Safe agent execution: Running developer coding agents (such as Codex, SWE-agent, or Claude Code) in contained environments where untrusted tool execution (shell commands, scripts) cannot damage host machines.
• CI/CD regression testing: Testing prompt, model, or tool changes against a deterministic testbed before shipping agent updates to production.
• Trajectory collection for post-training: Capturing complete, reproducible action trajectories and feedback loops for reinforcement learning (RLHF/RLAIF) or supervised fine-tuning.

How the concepts connect: the unified architecture

Understanding modern AI requires seeing how these twelve pieces fit together in a compound system:

The diagram is one possible arrangement. A workflow can contain an agent step, a runtime can serve one agent without a separate orchestrator, and an evaluation harness can test either. Choose boundaries according to the task rather than implementing every box.

When to use what: practical decision framework

Choosing the right abstraction prevents over-engineering and eliminates runaway system complexity.

Architectural comparison matrix

Decision flowchart

When designing a feature, use this decision path to pick the simplest viable architecture:

1. Does the task require real-world actions or dynamic external data?
   • No: Use a raw LLM (prompt engineering + structured outputs).
   • Yes: Proceed to Step 2.
2. Is the execution path fixed and predictable?
   • Yes: Build a workflow (code pipeline calling LLMs at specific nodes with tools).
   • No: You need an agent (an LLM in an autonomous ReAct loop).
3. Does the task risk context explosion or require distinct domains of expertise?
   • No: Use a single agent equipped with relevant skills (via progressive disclosure) and tools.
   • Yes: First improve retrieval and context selection. Consider separate agents when the work has independent boundaries and the coordination cost is justified; verify the combined result.
4. How are tools and skills provided and distributed?
   • If exposing a reusable collection of tools and resources across different clients: serve them via an MCP server.
   • If distributing a pre-packaged bundle of tools, MCP configs, and skills: distribute as a plugin.
5. How do you intercept, monitor, and secure execution?
   • Attach lifecycle hooks (beforePrompt, beforeToolCall, afterModelCall) to enforce deterministic guardrails, logging, and budget limits outside the model. Never rely on prompts alone for safety.
6. How do you benchmark, isolate, and test the system?
   • Build an evaluation and runtime harness to sandbox execution, inject reproducible context, enforce resource limits, and run automated regression testbeds.

Start with one bounded task and an observable acceptance check. Add tools, context management, or coordination when a specific failure or requirement justifies the extra machinery. The execution walkthrough develops that approach with one parser example.