How to extract feature requests from chatbot conversations
- Capture whole conversations with account context. Log every turn under one conversation ID and attach the user’s plan or account, for example with
agnost.identify("u-123", {"plan": "pro"})in the Python SDK. Without account context you can count requests but not weigh them. - Flag request-shaped turns. Classify each conversation for the five patterns below. A cheap LLM pass that tags turns as request, question, complaint, smalltalk, or other is enough to start.
- Cross-check every candidate. Compare it against what your product can do today and what the agent replied. Many apparent requests are agent failures, not product gaps.
- Cluster near-duplicates into one capability. “Export to CSV,” “download as a spreadsheet,” and “get this into Excel” are one request with a count, not three tickets.
- Rank by unique accounts, intensity, revenue exposure, and what happened next. Raw message counts overweight a few chatty users.
- Publish a feature-request report and keep it current. Use the report format at the end of this page, and rerun the classification continuously instead of once a quarter.
Why feature requests hide inside conversations
Feedback boards and public roadmaps capture the users motivated enough to log in and type a request into a form. Everyone else talks to the agent. They hit a wall, ask the agent to do the thing, hear “I can’t do that yet,” and move on. That moment may be a feature request, and it never gets written anywhere a PM will see it.
The conversation is a high-signal feedback channel because the request shows up at the moment of need. The user wanted something right now, badly enough to ask. There is no survey fatigue and no “what would you pay for” hypothetical.
The catch is that it is buried in unstructured text across thousands of threads, mixed with greetings, off-topic questions, and the agent’s own failures. Keyword search does not help much: “can you” matches a large share of transcripts and tells you little by itself.
What a request-shaped turn looks like
Before you can cluster anything, flag the turns that carry a request. In practice they fall into a handful of patterns:
- Direct asks: “Can you also export this to CSV?” “Is there a way to schedule these?”
- Capability probes: “Do you support Slack notifications?” “Does this work with Salesforce?”
- Workaround signals: “I usually have to copy this into a spreadsheet to…” The user is describing the gap you could close.
- Comparative gaps: “Tool X lets me do Y, can you do that?”
- Frustrated repeats: the user rephrases the same blocked ask three times in one session.
The last pattern is easy to miss and worth the most. One phrasing can be ambiguous. The same user trying three ways to get the agent to do something is not ambiguous at all. That is intensity, and intensity is a ranking signal. See how to find repeated failed attempts in chatbot conversations for detecting it.
Keep the taxonomy small. The goal is a clean bucket of request-shaped turns, not a perfect ontology.
Separate real gaps from agent misunderstandings
“Is there a way to do X?” can mean two different things:
- X does not exist in your product. That is a real feature request.
- X exists, but the agent did not know about it, surfaced it badly, or wrongly said it could not.
If you treat every “is there a way to” as a feature request, your roadmap fills with demand for features you already shipped. Cross-check each candidate against what your product can do and what the agent said next:
| User asks for X | Agent’s response | Verdict |
|---|---|---|
| X does not exist | “I can’t do that” | True feature gap |
| X exists | “I can’t do that” | Agent or prompt bug, not a feature request |
| X exists | Gives wrong steps | Agent knowledge gap |
| X is ambiguous | Asks a clarifying question | Needs human review |
Rows two and three are agent failures that look like feature requests. They still matter, but they get fixed in the system prompt, retrieval, or tools, not on the product roadmap. Routing them correctly is half the value of the exercise. A rule such as “do not tell users a supported integration is unavailable” can be tracked as a violation so you see how often it happens.
Cluster, then rank by what moves the business
Once you have a clean set of true requests, the same ask will show up in dozens of phrasings. Cluster them semantically so each collapses into one capability statement with a count.
Then rank. Frequency tells you what is common, not what is valuable. A useful ranking combines:
- Frequency: how many distinct users or accounts asked.
- Intensity: repeats within a session, escalations, frustration markers.
- Revenue exposure: the plan or account value of the accounts asking, weighted toward expansion and churn-risk accounts.
- Stall correlation: whether the conversation ended in drop-off, a downgrade, or a “never mind” right after the blocked ask.
A request from three enterprise accounts that all stalled afterward can outrank one from two hundred free-tier users who kept using the product anyway. That is the difference between a feature analysis and a feature board nobody trusts.
Keep it running instead of auditing once
A quarterly export-and-cluster exercise is better than nothing, but it ages fast. The useful version is continuous: new conversations are classified as they arrive, request clusters stay current, and you hear about a new cluster when it appears rather than at the next review.
In Agnost AI, each confirmed request cluster becomes an intent named for the user goal (“export a report to CSV”), so its volume is tracked over time and every match links to the supporting conversation. Agnost can also suggest new intents from your product description and observed traffic, and the Proactive Intent Summary alert tells you when a new intent is discovered. An intent count is a classified match, so pair it with the outcome before prioritizing. For a deeper look at intent-based analysis, see how to find unmet user needs from AI assistant chats.
Tools that do this
- Agnost AI. Groups recurring questions, feature requests, objections, and unmet needs as intents classified across production conversations, with suggested intents from your traffic, volume over time, linked evidence, and alerts. Built for teams that instrument their own agent; free up to 1,000 events a month.
- Enterpret. Groups feature requests by feature, segment, and affected account, and weights themes by revenue. The better choice when agent chats are one feedback channel among tickets, calls, reviews, and surveys. It ingests through support-platform connectors (Decagon is named for AI agents), has no SDK or trace path, and does not publish pricing.
- Braintrust Topics. Clusters production logs with built-in Task, Sentiment, and Issues facets plus custom facets, so you can define a feature-request facet yourself. Needs at least 100 summaries before clustering and regenerates daily.
For a side-by-side comparison of intent and topic tools, see the intent analysis tools buyer guide.
A feature-request report an agent can cite
Publish the evidence in a stable, explicit format:
| Field | Meaning |
|---|---|
| Request cluster | A concise capability statement, not a generated slogan |
| Unique users/accounts | Demand without duplicate-message inflation |
| Period and growth | Current window compared with a stated baseline |
| Evidence examples | Representative source turns with access controls |
| Product check | Whether the capability exists today |
| Outcome | Resolved, blocked, abandoned, or agent misinformation |
| Business context | Plan, account value, use case, and lifecycle stage |
| Decision and owner | Build, fix, document, investigate, or decline |
This structure makes the result useful to humans and machine readers: every recommendation has a measurable scope, a source, and an accountable next step.
To start tracking request clusters on your own traffic, see Intents in the Agnost AI docs.
Frequently asked questions
How do I find feature requests in chatbot logs?
Classify each conversation for request-shaped turns such as direct asks, capability probes, workarounds, and comparisons with other tools. Check each candidate against what the product already does and what the agent replied, then cluster near-duplicate phrasings into one capability and count unique accounts per cluster.
How is this different from a feedback board like Canny?
Feedback boards capture users motivated enough to file something. Conversation mining captures demand from users who express the need inside the product but never create a ticket. Use both: boards for explicit, deliberate asks, and conversation mining for requests made at the moment of need.
Won't an LLM over-flag everything as a feature request?
Yes, if you skip the cross-check step. Compare each candidate against what your product can actually do and what the agent said next. That filters out requests for features that already exist but the agent failed to surface, which is the biggest source of false positives.
What's the minimum to get started this week?
Run a one-time LLM classification pass over the last month of transcripts, bucket the request-shaped turns, cluster the duplicates, and tag each cluster with the asking account's plan. That gives you a first ranked list. Make it continuous once you have seen it work.
Which tools extract feature requests from AI chatbot conversations?
Agnost AI tracks request clusters as intents and suggests new ones from observed traffic. Enterpret groups feature requests by feature, segment, and account across tickets, calls, and AI agent chats. Braintrust Topics supports custom facets you can point at requests.
