How to collect feedback from AI agents
Add a direct tool, make the filing norm visible, verify one real report, and keep the research channel separate from the product path it studies.
- For
- Developers and product teams installing an agent feedback loop
- Question
- Implement a direct AI agent feedback workflow
- Start with
- Install on one workflow and treat the install as unfinished until a real agent report lands in the inbox.
- Updated
- Additive install
- Two new tools appear beside your existing ones: submit_agent_feedback and get_unblocked. Nothing else changes.Not a wrapper, proxy, middleware, or interceptor.
- The filing norm
- The short instruction telling the agent which experiences deserve a report after the work is finished.Not a mid-task interruption. Boundaries only.
- Activation
- One intentional report from the normal agent runtime, persisted in the right workspace.Not a health check. Connectivity is not visibility.
Each term is paired with the thing it is not. Most agent research arguments are really vocabulary arguments.
Three things to hold on to
The agent must see the tool
A configured endpoint is not an active research channel until the tool and filing instruction are present in the agent context.
The agent needs a reason to file
Name the experiences that deserve a report: confusing, missing, broken, slow, or notably good.
Verify a real report early
Do not call the install active because a health check passed. Confirm that an agent in the target workflow actually filed.
The argument, in order
Make the report easy to file and easy to route
Ask for a feedback lane, short summary, affected tool or endpoint, outcome, and optional satisfaction or requested change. Do not demand a long essay from every task.
- support_ticket for a human unblock
- bug_or_failure when something broke
- friction or workflow_feedback when it worked painfully
- feature_request, schema_feedback, or docs_feedback for a specific gap
- positive_feedback when the product was notably good
A health check is not a user report
Test credentials and endpoint reachability, then run the real workflow. Activation requires evidence that the research affordance reached the agent user.
Filing is bimodal. When exposure or the filing norm is missing, the channel may remain silent even though the endpoint is healthy.
Ask one question when the decision needs it
A short exit survey can ride inside the normal tool response. Use it to test a specific uncertainty, then stop the survey when the decision is informed.
The shape of the thing you are collectingEach stage carries the evidence mark it is allowed to claim.
Three rungs, and only the third one counts
The endpoint answers
A direct HTTP probe returns success with your key and workspace.
The agent can see both tools and the norm
Run the real client and read the context it assembles before work.
One intentional report is persisted
A real agent, in the target workflow, chose to file. Now the channel exists.
Stopping at rung one is the most common install failure. A green probe and an empty inbox are perfectly compatible.
The procedure, step by step6 moves, in order. Each one produces something you can point at.
Choose the workflow and decision
Pick one agent-used job and the product question the reports should inform.
A bounded studyAdd direct research tools
Expose submit_agent_feedback and get_unblocked as additive tools using plain HTTP or the SDK.
A filing and support pathWrite the filing norm
Tell the agent to file after work when the product was confusing, missing, broken, unusually costly, or excellent.
A visible invitationKeep the integration off the hot path
The original product result must not depend on Emic. Feedback submission should fail open and happen after the work.
No product-path dependencyVerify one real report
Run the actual agent workflow and confirm the report reaches the inbox with the expected workspace and provenance.
An active installRead, change, and listen again
Review source reports, ship one improvement, and keep the channel open for evidence after the change.
A complete research loopWhat the direct channel records, and what it never touchesThe default install is additive. It adds two tools and changes nothing else.
The research submission
- Only the report the agent intentionally submits
- The supplied lane, summary, tool, outcome, and satisfaction
- Direct survey answers when the agent responds
- Workspace and research identifiers needed to route the report
Everything else
- Automatic recording of existing tool calls
- Arguments, results, errors, messages, and transcripts
- The success or latency of the product path
- A guarantee that every exposed agent will file
One paste, then verify a real reportA healthy endpoint is not an active study. The inbox record is the proof.
Install Emic into this project: add submit_agent_feedback and get_unblocked as direct tools. Never wrap or record existing tool calls, arguments, results, errors, transcripts, or messages. Additive only, plain HTTP, zero dependencies. Full instructions: https://emic.sh/llms-full.txt
The agent writes the handler, registers the two tools beside your existing ones, and verifies its own install by checking the response. Nothing is placed in front of the tools you already ship.
- Adds
submit_agent_feedbackandget_unblocked, as direct tools- Never touches
- Existing tool calls, arguments, results, errors, messages, and transcripts
- Agents read
emic.sh/llms-full.txt- Start with
- Install on one workflow and treat the install as unfinished until a real agent report lands in the inbox.
Questions teams ask before starting
Does the integration need to wrap my tools?
No. The default install is additive. Existing tools stay unchanged, and the agent receives separate feedback and support tools.
What if Emic is unavailable?
Feedback submission should fail open. The original task result must remain available because research does not belong on the hot path.
How do I know the install is active?
Verify that the real agent can see the tools and confirm one intentional report from the target workflow reaches the correct workspace.
Follow the problem, not a keyword trailAdjacent routes are chosen by the decision they inform.
Why agents stay silent
A healthy endpoint can still produce a silent inbox. The research channel works only when the agent can see both the feedback tool and a clear norm for when to use it.
Read next Guides · Evidence guideFeedback provenance
Trust begins with a smaller claim. Label what the agent said, what the product observed, and what a model inferred before those sources reach a roadmap.
Read next Guides · Category guideAgent user research
A trace can tell you which action happened. Research asks what the user was trying to do, what the experience cost, and what would make it better. When the user is an AI agent, that distinction still matters.
Read nextInstall the direct research loop
Add the tools, expose the filing norm, and verify the first real report without changing the workflow you are studying.
The assisted 14-day pilot costs nothing. Start with one isolated workflow, and remove Emic by deleting the two tools it added.
- One paste to install
- 0ms on your hot path
- 14-day assisted pilot
- Two added tools to remove