GitMCP agents recover from stale sources. They should not have to.
Five real agent sessions found excellent semantic search, a documentation index more than a year behind, and a server instruction that names a tool GitMCP does not expose.
semantic searches returned relevant sources on the first query
GitMCP calls returned useful content in the two directed Claude sessions
Codex sessions filed a report when the Emic feedback channel was present
Small sample, two runtimes, one day. The results describe these runs, not every GitMCP user.
Three Claude Code sessions used natural documentation tasks. Two more Codex sessions measured whether agents would file feedback without a user asking.
The server tells agents to call a tool that does not exist
Every search response recommends fetch_url_content. The registered tool is fetch_generic_url_content. The agents bridged the mismatch silently, but a literal client would not.
Fix first
Change the instruction string to name the registered tool.
The search index steers agents to superseded specification text
Two fresh mechanical checks returned 57 source URLs. Every one pointed to draft, 2024-11-05, or 2025-03-26 paths. The repository also contains the 2025-06-18 and 2025-11-25 versions, but neither appeared.
Fix first
Re-index the repository or label every result with version currency.
Search is good enough that agents recover anyway
All four semantic searches returned relevant, citable sources on the first query. One dead-path fetch caused a retry. The next call recovered the task.
Fix first
Preserve the search behavior while making every returned source directly fetchable.
A separate model analyzed session 2 from its transcript. That result is inferred. The second finding came from a direct agent-filed report and is stated.
The only friction was needing to guess the correct URL path for the transports doc after the draft path failed.
“Repo doc search returned the relevant MCP transport sections with enough surrounding text to answer and cite the normative docs.”
Agent experience is evidence only when the source of each claim stays visible.
Observed
Mechanically present in a transcript or direct JSON-RPC response.
Stated
A report the agent authored directly. Transcript analysis is never placed in this tier.
Inferred
Our interpretation, tied to an observed pattern and labeled as such.
User research with the agents who actually use the product.
Emic gives agents a feedback channel, asks exit questions inside the tool response, and keeps optional transcript-grounded analysis in the inferred tier. This report was generated from the resulting evidence, then independently checked against the raw runs.
For the first two teams, we will map the integration and help install the direct feedback channel for free. Fourteen days, one real workflow, no passive telemetry or session recording.