Brian Jagger
When OpenAI quietly replaced the Assistants API with the Agents SDK in early 2025, I will admit I was sceptical. I had spent eighteen months building around Assistants, threads and runs, and the prospect of yet another paradigm shift felt exhausting. Then I sat down with the SDK over a weekend, ported one of my production agents, and came away genuinely impressed. The Agents SDK is the first OpenAI primitive that treats agent-building as a serious engineering discipline rather than a hosted black box.
OpenAI Agents SDK: Complete Guide for Developers
In this guide I want to walk you through everything I wish I had known on day one. We will cover the core primitives - Agent, Tool, Handoff and Tracing - in enough depth that you can build something real by the end. I will share the production patterns I have settled on after migrating three systems off Assistants, and I will be honest about where the SDK is still rough.
This is not a documentation regurgitation. It is the guide I would hand to a colleague joining my team to build their first production agent on OpenAI’s stack.
Why the Agents SDK replaced the Assistants API
The Assistants API was a noble experiment. It gave us threads, runs and file search out of the box, but it pushed too much state into OpenAI’s infrastructure and made it impossibly hard to debug. I lost count of the times I stared at a requires_action run with no idea why the model had decided to call a tool.
OpenAI listened. The Agents SDK is, in their words, “an unopinionated, lightweight package with very few abstractions” that gives you agents you can run yourself, trace yourself and reason about. The headline differences I noticed within an hour:
- Local execution by default. You hold the loop. OpenAI just provides the model and tracing endpoint.
- No more threads and runs. State is yours to manage. The SDK provides Sessions if you want a quick win.
- Handoffs replace assistant chaining. A first-class primitive for one agent to delegate to another.
- Tracing is opt-out, not opt-in. Every run produces a trace visible in the OpenAI dashboard.
OpenAI deprecated the Assistants API for new development. Existing assistants still work, but the migration path is explicit and supported. If you are starting in 2026, the Agents SDK is the only sensible choice on the OpenAI stack.
Core primitives at a glance
The SDK is small enough that I can list its primitives on a single page. That economy is part of its charm.
| Primitive | What it does | Mental model |
|---|---|---|
| Agent | Wraps a model, instructions, tools and handoffs | A configured persona |
| Tool | A Python function the agent can call | A capability |
| Handoff | Delegation from one agent to another | A baton pass |
| Guardrail | Input or output validator | A bouncer |
| Session | Conversation state persistence | A thread |
| Runner | Executes the agent loop | The conductor |
| Trace | Structured run history | The flight recorder |
That is essentially it. Everything else - retrieval, evaluation, deployment - composes from these primitives plus your own Python.
For a comparison with the Anthropic equivalent, see my Claude Agent SDK vs OpenAI Agents SDK post.
Setting up your first agent
I find the best way to learn a framework is to build a trivially small thing and then scale it up. Here is the canonical hello-world I use in my workshops.
from agents import Agent, Runner
agent = Agent(
name="Triage",
instructions="You triage customer questions. Keep responses under 80 words.",
model="gpt-4o-mini",
)
result = Runner.run_sync(agent, "My order is late, what should I do?")
print(result.final_output)
Three lines of substance: define the agent, run it, read the output. No threads, no runs, no polling. The first time I ran this I laughed at how much complexity OpenAI had quietly removed.
A few subtleties worth flagging immediately:
- The instructions field can be a string or a callable, which is useful for injecting per-request context like the current user ID.
- Runner.run_s
- The default model is whatever you set. I usually start with gpt-4o-mini for cost and bump to gpt-4o or o4-mini when reasoning matters.
I will build on this example throughout the guide.
Tools: from function calls to MCP
Tools are how an agent does anything beyond chatting. The SDK takes a Pythonic approach: decorate a function, type-hint the inputs, write a docstring, done.
from agents import function_tool
@function_tooldef get_order_status(order_id: str) -> str:"""Return the current status of an order given its ID."""return lookup(order_id)The SDK extracts the schema from the type hints and docstring. You attach the tool to an agent via tools=[get_order_status] and it becomes callable.
Beyond plain Python functions, you get:
- Hosted tools: WebSearchTool, FileSearchTool, CodeInterpreterTool, ImageGenerationTool and the ComputerTool for browser automation. These run on OpenAI’s infrastructure.
- MCP servers: The SDK speaks the Model Context Protocol natively. Point it at an MCP server URL or local process and its tools become available.
- Agents as tools: Wrap another agent as a tool with agent.as_tool(...). This is subtly different from handoffs because the parent retains control.
A pattern I use constantly is to combine a hosted WebSearchTool with a few custom function tools and one MCP server. It gives me search, structured actions and integrations without leaving the SDK.
Handoffs: the multi-agent primitive
The handoff is, in my opinion, the single most original idea in the SDK. Instead of one agent calling another as a tool and waiting for a return value, a handoff transfers control entirely. The receiving agent inherits the conversation and continues.
from agents import Agent, handoff
billing = Agent(name="Billing", instructions="Resolve billing questions.")
shipping = Agent(name="Shipping", instructions="Resolve shipping questions.")
triage = Agent(
name="Triage",
instructions="Route the user to the right specialist.",
handoffs=[billing, shipping],
)
When triage decides to hand off to shipping, the user effectively starts talking to a new agent. The SDK exposes pre and post hooks so you can mutate state, redact PII or log the transition.
I have found handoffs work best when:
- The set of specialists is small and well-bounded (think under ten).
- Each specialist has a clear scope you can articulate in one sentence.
- The triage agent is given explicit examples of when to hand off versus answer.
For more on this pattern, see my multi-agent systems collaboration deep dive.
Sessions, memory and state
If you do not configure anything, every Runner.run starts fresh. For a chatbot that is wrong. The SDK ships Sessions for conversational memory.
from agents import Agent, Runner, SQLiteSession
session = SQLiteSession("user-123", "agent.db")
Runner.run_sync(agent, "Hi, my name is Alex.", session=session)
Runner.run_sync(agent, "What is my name?", session=session)
Behind the scenes the session persists the message list and replays it. SQLite is the default. There are connectors for Postgres and Redis, and you can implement your own by subclassing Session.
Three patterns I use in production:
- Per-user session: One session per authenticated user, persisted in Postgres.
- Per-conversation session: A new session per support ticket or call.
- Hybrid: A long-running session for preferences plus a short-lived session for the current task. The agent receives both via context.
For richer memory - summarisation, entity extraction, knowledge graphs - you compose on top of Sessions rather than expecting them to handle it.
Tracing and observability
This is where the SDK quietly outshines its predecessor. Every Runner.run automatically produces a trace, visible in the OpenAI dashboard or exportable via OpenTelemetry.
What I see in a typical trace:
| Span type | What it captures |
|---|---|
| agent.run | Inputs, outputs, model used, total tokens |
| tool.call | Tool name, arguments, return value, latency |
| handoff | Source agent, target agent, reason |
| guardrail | Input or output validator result |
| custom | Anything you log via tracing.custom_span |
The traces are sampled at 100% by default but you can downsample with the trace_sampler config in production.
I have integrated traces with LangSmith, Arize and Datadog via OTel exporters. The SDK does not lock you in. For teams that already invest in observability, the OTel hooks are the killer feature.
A warning from experience: do not log sensitive data into tool arguments. The trace captures everything. Use the redact hooks if you handle PII.
Guardrails and input validation
Guardrails are validators that run before or after the model call. They can fail fast, modify the input or output, or trigger a tripwire that halts the run.
from agents import input_guardrail, GuardrailFunctionOutput
@input_guardrailasync def block_prompt_injection(ctx, agent, user_input):score = injection_classifier(user_input)return GuardrailFunctionOutput(output_info={"score": score},tripwire_triggered=score > 0.9,)I attach guardrails for:
- Prompt injection detection on user input.
- Tool argument validation before risky tools execute.
- Output PII scrubbing before responses leave the agent.
- Policy compliance, for example refusing financial advice.
Guardrails are deliberately Pythonic. You can call OpenAI Moderation, a custom classifier, or even a small LLM judge. The tripwire pattern means a violation halts the run with a structured error you can handle in your application.
Streaming and real-time UX
Anyone who has built a chat UI knows that streaming is non-negotiable. The SDK supports it cleanly.
async for event in Runner.run_streamed(agent, "Plan a trip to Lisbon"):
if event.type == "raw_response_event":
print(event.data.delta, end="")
elif event.type == "run_item_stream_event":
handle_tool_or_handoff(event.item)
You get three event categories: raw token deltas, run items (tool calls, handoffs, messages) and lifecycle events. In production I pipe raw deltas to the UI for low-latency token rendering and use the run items to render structured UI elements like tool call cards.
For voice applications, the SDK pairs naturally with the Realtime API. The Agents SDK takes care of tool orchestration while Realtime handles speech-to-speech.
Evaluation patterns
I am almost evangelical about evals. The SDK does not bundle an eval harness, but it makes evaluation easy because runs are deterministic given fixed seeds and trace-rich.
My standard eval setup:
- A YAML file of test cases with inputs and expected outcomes.
- A runner that executes the agent on each case and stores the trace ID.
- An LLM-as-judge that scores correctness, faithfulness and tone.
- A regression check that fails CI if any score drops more than a threshold.
I usually wire this into the OpenAI Evals product when the test set is small and proprietary, and into LangSmith Evals or Braintrust when it scales.
“If you cannot answer the question ‘has this change made the agent better?’ with a number, you are not ready to ship.” This is the rule I drill into every team I work with.
For inspiration on what to test, see my AI agents portfolio projects and AI agents interview questions posts.
Production deployment architecture
The SDK is just a Python package. Deployment is therefore conventional. The architectures I have shipped:
- Single FastAPI service behind a load balancer, with Postgres for sessions and Redis for rate-limit state. Good for moderate traffic and easy to operate.
- Worker pool with a queue (RabbitMQ or SQS) for long-running agents that may take minutes. The HTTP layer enqueues, workers consume and stream results via WebSockets or Server-Sent Events.
- Edge-routed multi-tenant deployment with a thin gateway that picks the correct agent and session per tenant. This is what I tend to recommend for SaaS.
Operational essentials I never skip:
- Timeouts on every external call, including OpenAI.
- Idempotency keys at the request layer so retries do not duplicate side effects.
- Circuit breakers around tool calls that touch external systems.
- Per-tenant rate limiting in addition to OpenAI’s account-level limits.
For more on this discipline, see LLM routing and orchestration patterns.
Cost and rate limit management
Agents are token-hungry. A single multi-turn run can consume tens of thousands of input tokens. I have learned to manage this aggressively.
Tactics that work:
- Prompt caching. OpenAI’s automatic caching kicks in for prefixes you reuse. Structure your instructions so the first 1024 tokens are stable.
- Model routing. Use gpt-4o-mini for triage, gpt-4o for reasoning, o4-mini only when chain-of-thought is essential.
- Tool result pruning. Long tool outputs balloon context. Summarise before adding to the conversation.
- Session compaction. After N turns, summarise the older messages into a single system note.
- Rate-limit headers. Honour x-ratelimit-remaining and queue rather than hammer.
A useful baseline: for a customer support agent with three tools, expect 4-8k tokens per turn average. If you see 20k+ consistently, something is wrong.
Migrating from the Assistants API
If you are coming from Assistants, here is the mental mapping that helped me.
| Assistants API | Agents SDK |
|---|---|
| Assistant | Agent |
| Thread | Session |
| Run | Runner.run invocation |
| Tools (function, code, retrieval) | Function tools, hosted tools, FileSearchTool |
| requires_action | Tool calls handled inline by the loop |
| OpenAI-hosted state | Your own database |
The migration steps I followed for a production assistant:
- Re-author the assistant’s system prompt as Agent.instructions.
- Re-implement each function tool as a @function_tool decorated Python function.
- Replace client.beta.threads.create with a SQLiteSession keyed by user.
- Replace the polling loop on runs with a single Runner.run_sync call.
- Add tracing exporters and reconnect your monitoring.
Allow a week per agent including testing. Most of that week is QA, not code.
Limitations and roadmap
I would not be doing my job if I painted the SDK as perfect. The honest gaps:
- No built-in eval harness. You compose your own or use OpenAI Evals.
- Hosted tools have variable latency. WebSearchTool is great when fast and frustrating when slow.
- Tracing UX in the dashboard is improving but still less polished than LangSmith.
- Computer use is powerful but experimental. I have not yet trusted it in customer-facing flows.
- Voice integration requires bridging to the Realtime API by hand.
OpenAI has been shipping at a steady pace. Expect more first-party connectors, deeper MCP support and improved eval tooling through 2026.
