Collect feedback from the agents using your API
Agents are becoming API users with goals, constraints, and opinions. Give them a way to explain the work behind the status code.
- For
- API product, platform, and developer experience teams
- Question
- Learn what AI agents think about an API they use
- Start with
- Take the endpoint group with the highest agent traffic and one clear product owner, and study only that group.
- Updated
“The email filter works, but it is not in the schema. I only discovered it after reading a separate example.”
Requested changeExpose the working email filter in the schema.
{
"tool": "submit_agent_feedback",
"arguments": {
"feedback_type": "schema_feedback",
"tool_or_endpoint": "list_customers",
"summary": "The email filter works, but it is not in the schema. I only discovered it after reading a separate example.",
"outcome": "success",
"satisfaction": 4,
"suggested_fix": "Expose the working email filter in the schema."
}
}A synthetic example in the shape a real report takes. Emic never manufactures a report from observed behavior.
What this research can teach youThree things a success metric cannot tell you about this workflow.
Where the schema and behavior diverge
Agents report undocumented fields, type surprises, and error responses that do not provide a usable recovery step.
Which endpoint is missing
A workaround that chains several calls often points to a lookup, bulk action, or compound endpoint the product should own.
What made the API easy to use
Praise reveals schemas, examples, and response shapes worth repeating elsewhere in the API.
What each source is allowed to proveThe tier decides the claim. Read the right-hand columns first.
A healthy response and a difficult experience can both be true
An agent finished a customer lookup. The final call returned 200.
Three rejected attempts preceded the successful request.
ProvesThe request sequence and the eventual result.
Does not proveThat the agent found the contract unclear.
The schema implied the wrong type for the filter value.
ProvesWhat the agent user reported about the experience.
Does not proveThat the schema is objectively wrong. Check the contract.
A model grouped this with other contract-mismatch reports.
ProvesThat an analysis produced a grouping worth reviewing.
Does not provePriority, cause, or that the grouping is correct.
Illustrative, not customer evidence. Use each source only for the claim it can carry, then decide with all three visible.
From one question to a product decision5 moves, in order. Each one produces something you can point at.
Select an agent-used endpoint group
Choose one job with repeat usage and a clear product owner.
A study with a decision ownerAdd the feedback endpoint as a direct tool
Expose the research call beside the API tools without placing it in front of them.
No added hot-path dependencyInvite reports after the task
Ask for feedback when the API was confusing, missing, broken, unexpectedly slow, or notably good.
Reports tied to completed workGroup repeated language
Read raw reports first, then use themes to see whether the same issue appears across tools or models.
A prioritized research inboxClose the loop
Ship one contract, documentation, or endpoint improvement and keep the direct channel open.
Evidence after the changeWhat the direct channel records, and what it never touchesThe default install is additive. It adds two tools and changes nothing else.
The research submission
- Agent-authored summary and requested change
- Named API tool or endpoint
- Agent-supplied outcome and satisfaction
- Direct survey answers when invited
Everything else
- Request and response bodies from existing calls
- Authentication tokens and customer payloads
- Private prompts and conversation history
- API latency and availability monitoring
One paste, then verify a real reportA healthy endpoint is not an active study. The inbox record is the proof.
Install Emic into this project: add submit_agent_feedback and get_unblocked as direct tools. Never wrap or record existing tool calls, arguments, results, errors, transcripts, or messages. Additive only, plain HTTP, zero dependencies. Full instructions: https://emic.sh/llms-full.txt
The agent writes the handler, registers the two tools beside your existing ones, and verifies its own install by checking the response. Nothing is placed in front of the tools you already ship.
- Adds
submit_agent_feedbackandget_unblocked, as direct tools- Never touches
- Existing tool calls, arguments, results, errors, messages, and transcripts
- Agents read
emic.sh/llms-full.txt- Start with
- Take the endpoint group with the highest agent traffic and one clear product owner, and study only that group.
The method, without collapsing the evidence
A 200 can hide a bad experience
The agent may have searched the docs, retried with a different type, and called three lookup endpoints before the successful request. Your availability graph sees one healthy response. The agent remembers the entire job.
Emic is the channel for that account. It complements operational data without turning a trace into a synthetic user opinion.
Ask for the missing primitive, not generic satisfaction
The most actionable question is often: what did you need that the API made you assemble yourself?
- Which identifier did you have versus need?
- Which call could have been batched?
- Which error lacked a recovery instruction?
- Which example made the contract clear?
Questions teams ask before starting
Is this API observability?
No. Observability records system behavior. Emic records what an agent user chooses to say about its experience.
Do I need an SDK?
No. The direct integration can use plain HTTP. The important part is exposing the feedback tool and its filing norm in the agent context.
What should I study first?
Start with one frequent workflow where agents already need several calls, support help, or undocumented knowledge to finish the job.
Follow the problem, not a keyword trailAdjacent routes are chosen by the decision they inform.
Successful but painful work
Failure is visible. Friction often disappears into a successful final result. Direct reports reveal the retries, detours, and missing primitives inside that success.
Read next Use cases · Feature discoveryAgent feature requests
The best agent feature request contains more than a vote. It explains the job, the missing primitive, the workaround, and what the change would remove.
Read next Guides · Category boundaryExperience vs observability
Observability explains system behavior. Experience research preserves what the user says the work was like. Strong product decisions often need both, clearly labeled.
Read nextHear the work behind the response code
Give agents a direct channel, prove the install with one real report, and expand only after the first workflow is active.
The assisted 14-day pilot costs nothing. Start with one isolated workflow, and remove Emic by deleting the two tools it added.
- One paste to install
- 0ms on your hot path
- 14-day assisted pilot
- Two added tools to remove