The eight-minute briefing.
The video is deliberately 40% architecture and 60% product judgment. Watch once end-to-end, then use the rest of this document to rehearse answers.
The model call is the small part.
An agent is not a chatbot with a tool. It is an application that plans, acts through tools, maintains enough state to finish multi-step work, and sometimes transfers control or pauses for a human.
Decision
The model proposes output items, including tool calls or a transfer.
Execution
The runtime validates, invokes, records, and feeds results back into the loop.
Operation
The product exposes state, cost, safety, traces, interruptions, and recovery.
Responses when you own the loop. Agents SDK when the SDK should run it.
The official guide draws a clean boundary. The Responses API is the model-response primitive. The Agents SDK provides an agent-run abstraction and manages recurring orchestration such as repeated tool calls, branching, handoffs, sessions, guardrails, tracing, and resumable approvals. The application’s server still owns deployment, tool implementations, storage choices, and approval policy.
| Dimension | Responses API | Agents SDK | PM implication |
|---|---|---|---|
| Core abstraction | A model response | An agent run | The SDK’s product surface is lifecycle semantics, not only request syntax. |
| Loop | Application implements it | Runner performs it | Value equals repeated orchestration removed without blocking escape hatches. |
| Multi-agent | Build routing/delegation | Agents-as-tools and handoffs | Make ownership explicit; do not reward topology complexity. |
| State | History, response chaining, Conversations | Same options plus SDK sessions and resumable state | State ownership must be understandable, portable, and debuggable. |
| Safety | Tool approvals; broader controls are custom | Input/output/tool guardrails and approval flows | Safety is a workflow UX, not a boolean filter. |
| Observability | Response objects and API logs | Traces across calls, tools, agents, guardrails, handoffs | Trace-to-fix is a first-class developer productivity outcome. |
Product surfaces you should be able to explain
Agent + Runner
Agent is the reusable configuration: model, instructions, tools, output contract, policies, and specialists. Runner owns the loop and stop conditions through async, sync, or streamed entry points.
Tools + orchestration
Hosted tools, function tools, MCP, and agents-as-tools give the model action. Manager-style composition retains central ownership; handoffs transfer active ownership.
State + humans
Sessions manage conversation memory. Run state can serialize interruptions, collect approval/rejection, and resume. Long-lived work can use durable integrations.
Operate + improve
Tracing records lifecycle spans; usage aggregates requests and tokens; hooks expose events; evals turn observed failures into repeatable tests.
from agents import Agent, Runner, function_tool
@function_tool
def lookup_order(order_id: str) -> dict:
return {"order_id": order_id, "status": "shipped"}
agent = Agent(
name="Support specialist",
instructions="Resolve order questions; ask approval before refunds.",
tools=[lookup_order],
)
result = await Runner.run(agent, "Where is order 1042?")
print(result.final_output)
The visible code is small because the runtime owns schema handling, repeated model turns, result injection, stop conditions, tracing, and usage. That is the activation wedge—not the whole long-term value.
Eve turns project structure into product opinion.
Vercel Eve calls itself a filesystem-first framework for durable agents. Conventional locations describe capabilities before the system runs.
Code-first runtime primitives
- Agent and Runner organize orchestration.
- Typed application code in Python or TypeScript.
- Integrates into storage, deployment, tools, and policy that the team owns.
- Durability is available through explicit integrations and resumable state.
Filesystem-first application harness
agent/ ├── instructions.md ├── tools/ ├── skills/ ├── channels/ ├── schedules/ ├── sandbox/ └── subagents/
- Discovery and compilation derive a runtime surface from files.
- Workflow-backed sessions checkpoint and resume by default.
- Channels, schedules, sandbox, skills, and evals share one project model.
| Question | Agents SDK thesis | Eve thesis | Interview insight |
|---|---|---|---|
| Where is authoring? | Objects and typed code | Conventional files and folders | Discoverability can be a product feature, not documentation work. |
| Where is durability? | Resumable state + integrations | Workflow checkpoints as a default | Defaults shift activation, operations, and lock-in simultaneously. |
| How broad is scope? | Runtime/orchestration SDK | Agent application framework | Category strategy is a choice of abstraction boundary. |
| What is the risk? | Users rebuild surrounding app scaffolding | Opinionated conventions constrain unusual systems | The roadmap should make the common path paved and the boundary permeable. |
Segment by maturity and responsibility.
Explorer / prototype builder
Job: prove an agent can complete a useful task today.
Needs: quickstart, sample tools, sensible defaults, immediate traces.
Failure: installation works, but the first real tool loop does not.
Application engineer — primary wedge
Job: ship a reliable workflow inside an existing product.
Needs: typed tools, stable state, streaming, approval UX, testability.
Failure: the demo cannot survive production data and failure modes.
AI platform team
Job: provide governed primitives to many internal teams.
Needs: policy, provider/model governance, reusable components, fleet metrics.
Failure: every app invents incompatible state, traces, and safety semantics.
Security + operations
Job: control risk and diagnose incidents without blocking innovation.
Needs: approvals, audit trails, redaction, budgets, retention controls.
Failure: observability itself leaks sensitive model or tool data.
A coherent progression is: fast first run → guided productionization → explicit control at scale. The platform should reveal complexity as the job requires it, not at installation time.
The boundary is the strategy.
Strong defaults with real escape hatches
Make the safe common path short. Let sophisticated teams replace models, storage, tracing processors, tool behavior, and infrastructure. An escape hatch that bypasses observability or changes semantics silently is not a good escape hatch.
Observable and consistent lifecycle contracts
A handoff, guardrail, approval, retry, and interruption should mean the same thing in code, traces, results, and evals. This reduces both cognitive load and support burden.
Multi-agent complexity must be earned
More agents increase calls, context, latency, cost, and evaluation surface. Encourage specialists when ownership or policy genuinely differs—not because a diagram looks sophisticated.
Make failure resumable, not merely visible
Observability answers “what happened?” A production runtime also helps answer “what can safely continue?” State snapshots, idempotency guidance, approval resumption, and durable execution belong to one recovery story.
Turn traces into evals
The highest-leverage flywheel is production failure → understandable trace → reproducible eval → verified fix → safer rollout. The product should shorten every edge in that loop.
Count outcomes, then guard the outcome.
Anti-metrics: raw run count, raw tool-call count, and number of agents per workflow. Each can rise while user value, reliability, or margins fall.
Prioritize the graduation bottleneck.
For most platform products, the existential gap is not “can a developer make one demo?” It is “can that team graduate to production and continue improving?” Diagnose the leaky stage before proposing the feature.
Activation gap
Improve quickstarts, errors, templates, and the first successful tool loop.
Production gap
Improve state, approvals, traces, evals, privacy, and cost controls.
Scale gap
Improve governance, reuse, fleet operations, migration, and compatibility.
A practical scoring frame
FREQUENCY
SEVERITY
FIT
LEVERAGE
RISK
RIGHT TO WIN
Validation plan: cohort the funnel from first run to production; interview recently activated and stalled teams; conduct concierge trace reviews; prototype failure clustering; measure reduction in time-to-root-cause and time-to-verified-fix; watch for privacy and false-confidence risks.
Approval is a product flow, not a modal.
Agent safety lives across the lifecycle: what instructions and tools are available, what input reaches which specialist, what tool arguments are allowed, which actions require approval, how a paused run survives, what the approver sees, what resumes after rejection, and what gets logged.
| Layer | Question | Failure mode | Product response |
|---|---|---|---|
| Input | Should this request enter the workflow? | Irrelevant or prohibited work consumes resources | Input guardrail with clear scope and user recovery |
| Tool | Is this action and argument safe now? | Model proposes a risky or malformed call | Schema validation, tool guardrail, approval, idempotency |
| Output | May this result be returned? | Final content violates policy or contract | Output validation and explicit failure behavior |
| Human | Can an accountable person make the decision? | Opaque approval, abandonment, or stale context | Summarize intent, impact, evidence, reversibility, expiration |
| Observability | Can we debug without leaking data? | Trace retention becomes a privacy incident | Redaction, sensitive-data controls, retention policy, export governance |
Use this answer operating system.
- Name the user and job. Avoid “developers” as the only segment.
- Identify the repeated failure or orchestration tax. Tie it to lifecycle evidence.
- State the principle and chosen abstraction boundary. Explain why the platform should own it.
- Propose the smallest leverage point and accepted tradeoff. Do not jump to a suite.
- Define success, guardrails, and learning plan. Include counter-metrics.
Case: “Should OpenAI build more multi-agent features?”
Case: “Should the SDK become more like Eve?”
Case: “How would you improve onboarding?”
Questions and model answers.
1. Why should the Agents SDK exist if the Responses API already exists?
The Responses API is the right primitive when an application wants to own model interactions and orchestration. The SDK exists because tool loops, specialist routing, state, guardrails, approvals, tracing, and result handling repeat across applications. Standardizing that lifecycle reduces orchestration tax and creates consistent observability and safety. The guardrail is that the SDK must preserve application ownership and escape hatches; otherwise convenience becomes lock-in.
2. Who is the primary user?
I would choose the application engineer graduating an agent from prototype to production. Explorers create adoption volume, but the application engineer experiences the full differentiated pain: tool semantics, state, approvals, latency, cost, debugging, and testing. Platform and security teams are critical secondary users whose requirements shape enterprise readiness.
3. What is the hardest product problem in an agents SDK?
Choosing stable lifecycle semantics while models, tools, and workflows evolve quickly. A runtime has to make common patterns easy without hiding enough detail that failures become mysterious. The PM challenge is to define contracts—run, state, handoff, approval, trace—that remain understandable across local development, production, and evaluation.
4. How do you decide between manager-as-tools and handoffs?
Start with ownership. Use a manager when one agent should synthesize results and enforce central policy. Use a handoff when a specialist should own the continuing conversation. Then compare context needs, latency, policy boundaries, and evaluation complexity. Multi-agent is justified by real specialization, not by the number of tasks.
5. What does Eve teach OpenAI?
Eve demonstrates a coherent demand for filesystem discoverability, stronger conventions, integrated channels and schedules, sandboxing, and durability by default. The lesson is that framework-level opinion can reduce production scaffolding. The strategic question is which of those insights should become optional SDK conventions versus a broader application platform, because scope expansion can weaken composability.
6. What would you prioritize next?
Absent data, I would prioritize the demo-to-production gap: trace clarity, eval integration, resumable state, approval ergonomics, and cost controls. My concrete bet is a production-readiness cockpit that turns observed trace failures into reusable eval cases. I would validate the assumption with lifecycle cohorts and trace interviews before committing.
7. What is the north-star metric?
Weekly successful production tasks completed through SDK-managed runs, qualified by product-specific correctness and safety gates. I would pair it with activation, reliability, efficiency, developer-productivity, and trust metrics. Raw runs are an input, not value.
8. How would you measure developer experience?
Use behavior, not only surveys: time to first successful tool run; time to add a second capability; trace-to-root-cause; trace-to-verified-fix; upgrade success; support incidents per production project; and retention of activated projects. Qualitative studies explain where cognitive load appears.
9. How should pricing influence product design?
The runtime should optimize cost per successful task, not minimize calls in isolation. Product surfaces should expose call count, token use, cache behavior, latency, and retries at the run and step level. Budgets and stop conditions should be composable. Pricing must not create incentives to hide observability or discourage safety checks.
10. How do you handle provider breadth versus consistent semantics?
Define a small portable core, publish capability differences clearly, and make provider-specific features explicit rather than silently degrading. Measure portability on real workflows. Consistency is valuable only when it does not erase differentiated model or platform capabilities.
11. What is a bad roadmap signal?
Requests for more orchestration primitives without evidence that current users cannot express high-value workflows. The underlying problem may be debugging, evaluation, state, latency, documentation, or organizational ownership. Diagnose the lifecycle bottleneck before adding topology.
12. How would you build trust in human approvals?
Give the approver enough context to judge: intended action, arguments, expected impact, evidence, reversibility, and expiration. Preserve the run durably while waiting. Make reject/edit paths clear. Measure abandonment, decision latency, reversals, and unsafe bypasses.
13. What is the moat?
Not the loop by itself—it is reproducible. The moat is an integrated learning system: best-in-class model/tool capabilities, stable lifecycle semantics, production traces, eval feedback, safety, ecosystem adoption, and a migration path that compounds developer trust.
14. How do you prevent abstraction leakage?
You do not eliminate it; you make it legible. Expose typed run items, raw responses where needed, trace spans, explicit state choices, provider capability differences, and well-documented escape hatches. A good abstraction compresses the common case and reveals the exceptional case.
15. How would you evaluate an agent runtime release?
Run contract tests for lifecycle semantics, representative workflow evals, reliability and latency benchmarks, upgrade/migration tests, safety red-team cases, and trace usability studies. Roll out by project cohort with rollback and compare task success, error classes, cost, and trace-to-fix time.
16. When should the product be opinionated?
Be opinionated where repeated failure has a clear safe default and where inconsistency creates ecosystem cost—tracing conventions, error taxonomy, approval state, schema validation. Be flexible where user systems genuinely differ—storage, deployment, domain policy, provider choice, and business logic.
17. How do you think about open source?
Open source can accelerate trust, debugging, contributions, and ecosystem reach for a developer runtime. The product strategy should clearly separate portable client/runtime value from hosted advantages such as trace operations, managed tools, evaluation, and model integration—without making the open layer artificially weak.
18. What could make the SDK fail as a product?
Unstable semantics, confusing state ownership, poor failure diagnosis, provider behavior that silently diverges, safety controls developers disable, or a gap between quickstart success and production readiness. A broad API with low trust is less valuable than a smaller one with predictable behavior.
19. What would you do in your first 90 days?
First 30: map lifecycle cohorts, top failure classes, support data, trace journeys, and stakeholder constraints. Next 30: select one graduation bottleneck and prototype with design partners. Final 30: ship an instrumented beta, define the metric tree, document the contract, and build the trace-to-eval feedback loop.
20. Give the 30-second product pitch.
Ask about boundaries and failure.
- Where do activated users most often fail between first successful run and production?
- Which lifecycle guarantees—state, handoff, approval, trace—are considered sacred contracts?
- How does the team decide what belongs in Responses, the Agents SDK, hosted platform surfaces, or ecosystem integrations?
- Which segment is currently the primary design center: application engineers, AI platform teams, or a broader developer audience?
- What do production traces reveal that support tickets and evals do not?
- Where does provider portability create value, and where does it constrain OpenAI-specific innovation?
- How does the team measure safety friction, especially guardrail false positives and approval abandonment?
- What would have to be true for the SDK to become more opinionated about durability, project structure, or deployment?
Primary sources.
- OpenAI API docs — Agents SDK overview and Responses comparison
- OpenAI Agents SDK — Agent definitions
- OpenAI Agents SDK — Runner lifecycle, state, approvals, durability integrations
- OpenAI Agents SDK — Handoffs
- OpenAI Agents SDK — Tracing
- OpenAI Agents SDK — Usage
- Vercel Eve repository and README
- Vercel Eve documentation — filesystem model, harness, durability, human input, sandbox, subagents, schedules, instrumentation, and evals.
Research note: Eve is labeled beta in its repository. Both products evolve quickly; verify renamed surfaces before quoting exact APIs in a live interview.