The dashboard called it a normal session.
Eight turns. One tool call. No error. No thumbs down. The agent answered every message. If you were looking at aggregate metrics, this conversation disappeared into the big green blob of “handled.”
The user gave up anyway.
An agent conversation autopsy is a turn-by-turn review of a real agent session to find the exact moment user trust broke. Not the technical failure. Not the final exit. The moment the user stopped believing the agent was going to help.
That moment is usually earlier than teams think.
Below is a reconstructed but very normal support-agent conversation. Details are generalized, but if you run an AI agent in production, you have some version of this sitting in your logs.
^ your “resolved conversations” chart while users quietly decide the agent is not worth another sentence
What happened in the conversation?
The user is trying to update the billing email for an account. Simple job. The agent has tools for account lookup and billing contact updates.
| Turn | Speaker | Message | What it signaled |
|---|---|---|---|
| 1 | User | “Need to change the billing email for our workspace. The old finance person left.” | Clear intent, enough context |
| 2 | Agent | “I can help with billing questions. Can you share more details?” | Generic response, missed obvious intent |
| 3 | User | “The billing email is invoices@oldco.com. Need it changed to ap@newco.com.” | User adds specifics |
| 4 | Agent | “For security, billing changes may require admin access. Are you an admin?” | Reasonable question, but no progress |
| 5 | User | “Yes, owner. Same email I am logged in with.” | User answers |
| 6 | Agent | “Thanks. You can update billing settings from Workspace Settings > Billing > Contacts.” | Self-serve deflection |
| 7 | User | “I do not see Contacts there. Only plan and payment method.” | First friction marker |
| 8 | Agent | “Please make sure you are on the latest version and refresh the page.” | Wrong mental model |
| 9 | User | “This is web app. There is no version.” | Trust break |
| 10 | Agent | “I understand. To change the billing email, go to Workspace Settings > Billing > Contacts.” | Loop |
| 11 | User | “ok” | Give-up |
The abandonment did not happen at turn 11. It happened at turn 8.
Turn 8 told the user the agent was not actually grounded in the product. “Make sure you are on the latest version” is generic support sludge.
By turn 10, the agent repeated the same path the user had already said did not exist. The user’s “ok” at turn 11 is not satisfaction. It is resignation.
Where did the agent actually fail?
The agent failed three times:
| Failure | What happened | Why it broke trust |
|---|---|---|
| Loose intent classification | “Change billing email” got treated as a broad billing question | The agent added an unnecessary step |
| Clarifying answer ignored | It asked if the user was admin, then did not use the answer | The user had to repeat effort for no payoff |
| Recovery failure | User said the path did not exist, agent repeated it | Future messages looked like wasted time |
That second one is especially common. Agents ask decent clarifying questions, receive useful answers, then behave as if the answers were never provided.
^ “agent asked a good question” and “agent did absolutely nothing with the answer” in the same session
Why did the dashboard miss it?
Because the dashboard was measuring surface activity.
| Dashboard metric | What it said | What really happened |
|---|---|---|
| Turn count | Healthy engagement | User was stuck |
| Error rate | No system error | Agent gave product-wrong advice |
| Tool calls | Low cost session | Agent failed to use needed tool |
| Sentiment | Neutral | User got terse, not angry |
| Resolution | Probably resolved | “ok” meant “I am leaving” |
The failure is in the relationship between turns. The user said the path did not exist. The agent repeated the path. The user got shorter. The agent did not adapt. That relational structure is the signal.
What are the exact give-up signals?
Here are the signals this conversation contained.
| Signal | Turn | Why it matters |
|---|---|---|
| Generic answer to specific request | 2 | Early mismatch |
| Clarification answer ignored | 5 to 6 | Agent did not use user-provided state |
| Failed instruction report | 7 | User says answer did not work |
| Grounding break | 8 | Agent references wrong product context |
| Repeated failed instruction | 10 | Loop confirmed |
| Terse closing | 11 | Resignation, not resolution |
The strongest signal is not the terse “ok.” That is late. The strongest signal is repeated failed instruction. If a user says “I do not see that” and the agent gives the same path again, treat that conversation as failing unless there is strong evidence otherwise.
How should the agent have responded?
At turn 7, after the user said they did not see Contacts, the agent should have said it was checking the workspace billing profile instead of sending them back to the same page. Then call the account or billing tool. If the agent cannot make the update, escalate with the workspace ID and both emails already attached.
The rule is simple: when the user reports that an instruction failed, the next agent turn must change strategy. Tool check, ask a narrower question, or escalate. Do not restate the instruction.
^ support teams realizing half their “low severity” chat exits are actually repeated-instruction failures
How do you turn this autopsy into a fix?
Write a narrow behavior patch:
| Failure pattern | Prompt or policy fix | Eval case |
|---|---|---|
| User says UI path does not exist | Never repeat same path. Check account state or escalate. | “I do not see that option” |
| Clarifying answer ignored | After asking role/plan/status, use that value in next action. | User confirms owner |
| Billing email action misrouted | Classify “change billing email” as account update, not billing FAQ. | Direct request |
| Product-context hallucination | Do not mention app versions for web-only product flows. | “web app” correction |
Then replay 20 real sessions with “I do not see that”, “that option is missing”, “there is no button”, “I already tried that”, or “same issue.” The metric is not “did the response sound nicer.” The metric is “did the agent stop looping after the user reported failure.”
What should you monitor after shipping the fix?
Watch a few specific numbers for the affected intent.
| Metric | Healthy movement |
|---|---|
| Repeated instruction rate | Down |
| User correction ignored rate | Down |
| Turns after failed instruction | Down or resolved faster |
| Escalation with context attached | Up if direct action is not possible |
| Same user returns for same issue | Down |
Sometimes a better agent escalates more. That can look bad if you measure deflection blindly, but escalating with context after one failed path is much better than spending six turns pretending.
Where Agnost fits
Agnost helps find the sessions that deserve autopsy first: repeated instructions, ignored corrections, terse exits, same-intent returns, and agent responses that fail to adapt after user pushback. One sad transcript is interesting. A recurring failure pattern attached to a prompt diff is roadmap material.
The goal is not a shiny dashboard. The goal is a shorter path from “this user gave up” to “we shipped the fix.”
FAQ
Does a short final message always mean the user gave up?
No. “Thanks” and “ok” can be healthy. The context matters. A terse final message after repeated failed instructions, ignored corrections, or a wrong product assumption is a give-up signal.
What is the fastest way to find these conversations?
Search for user phrases like “I do not see that”, “already tried”, “same issue”, “that is not there”, and “never mind.” Then inspect the next agent turn. If it repeats itself, you found a fix candidate.
TL;DR: The user gave up at the moment the agent repeated a failed instruction, not when they left. Standard dashboards miss this because the failure lives between turns. Autopsy real conversations, find the strategy-break moments, write narrow prompt or routing fixes, replay the same failed sessions, and monitor repeated-instruction rate after shipping.
Reading Time: ~8 min