What is the AI Insight Score?
Overview
The AI Insight Score is an automatically calculated score that helps you identify and prioritize the most significant insights across your journeys. It combines two factors — impact and reliability — to surface which insights are both impactful and well-supported by data.
Unlike the experience impact score you set manually, the AI Insight Score is calculated automatically and updates whenever new evidence is added or existing evidence changes.
How it's calculated
The AI Insight Score is the result of multiplying two factors:
Impact factor Measures the overall experience impact based on collected evidence. It aggregates sentiment scores from individual pieces of evidence and uses absolute values — so it captures impact whether positive or negative.
Reliability factor Measures the credibility of an insight based on the diversity and volume of its data sources. It increases with the number of unique data points and reaches its highest value when evidence comes from multiple source types. TheyDo considers five distinct source types: interviews, support logs, feedback, survey replies, and manually created notes.
Combining multiple source types increases reliability more than adding more data from a single source. An insight backed by one interview and one survey will score higher than one backed by ten interviews alone.
How Journey AI updates the score
When you use Journey AI to generate journeys or upload research files, TheyDo automatically extracts insights, assigns scores to new insights, and recalculates scores when new evidence enriches an existing insight.
Each time you run Mine Insights, Journey AI processes your selected files, extracts relevant insights, matches them to existing insights where applicable, and recalculates Insight Scores accordingly. Scores are continuously updated as insights are enriched — not just assigned once at creation.
Manually influencing the score
You can influence the AI Insight Score through manual actions without waiting for Journey AI to run.
Linking or unlinking evidence Adding or removing evidence from the Evidence tab automatically triggers a score recalculation.
Adjusting experience impact on quotes Select the impact icon next to a quote inside the insight to set its experience impact directly. Changes recalculate the score immediately.
Adding manual notes Notes you add manually contribute to the score in the same way as quotes extracted from files. Open an insight, select Add new note, enter your content, and set the experience impact.
Understanding the evidence structure
Each insight has two tabs that serve different purposes:
- Evidence tab — contains quotes that directly contribute to the AI Insight Score
- Insight tab — shows linked insights that provide context but do not affect the score
This separation gives you precise control over what influences a score. If a quote doesn't belong in the score calculation, keep it in the Insight tab rather than the Evidence tab.
Summary insights
Summary insights — created via the summarization feature — work slightly differently. Instead of Evidence and Insight tabs, they have a Summary Evidence tab. This tab aggregates insights from nested journeys, and the Summary Insight Score is the sum of all included Insight Scores. Any updates to nested insight scores automatically update the summary score.
Legacy quotes
All quotes created before the AI Insight Score was introduced have been automatically migrated. Old quotes remain visible in the Insights tab, and the same content now appears in the Evidence tab as a new quote entity.
For insights from early 2024 and before: you can still access old quotes, set experience impact values manually, and view verbatim content on hover.
Cleaning up legacy quotes: If you'd like to remove outdated quotes without affecting new insights, go to the Insights Library, apply a filter for quotes, select them in table view, and select Delete. This won't impact the new quotes in the Evidence tab.
Tips
- Use the AI Insight Score to prioritize which insights to review first — it's decision support, not a final verdict
- Treat a score that looks wrong as a signal to check the evidence: unlink irrelevant quotes, add missing ones, or adjust experience impact values
- Aim for evidence from 2–3 different source types per insight to maximize reliability
- Avoid overloading a single source type — too much data from one source reduces the reliability factor