Analyzing Your Results
How to turn recorded sessions into prioritized action items.
The Review Process
Watch First, Analyze Second
Resist the urge to take notes on the first watch. Just observe.
First pass (observation):
- • Watch the full session without pausing
- • Notice your emotional reactions (frustration, surprise, delight)
- • Get a holistic sense of the experience
Second pass (documentation):
- • Pause and note specific moments
- • Capture timestamps for key events
- • Write down exact quotes
What to Look For
Behavior Signals
- • Where did they click first?
- • Did they scroll past important content?
- • How long did they hesitate?
- • Did they try something multiple times?
Verbal Signals
- “I'm not sure what this does”
- “Oh, I expected that to...”
- “Where would I go to...”
- “This is nice” / “This is annoying”
Completion Signals
- • Did they finish the task?
- • How long did it take?
- • Did they take the expected path?
The Four-Category Framework
Sort every observation into one of four buckets:
Category 1: Critical Failures
What it looks like: User couldn't complete the task at all. Gave up.
Action: Fix immediately. These block core value.
Examples: “I have no idea how to do this” (then stops), clicking wrong buttons repeatedly with no progress, expressed confusion and abandoned task.
Category 2: Major Friction
What it looks like: User completed the task but struggled significantly.
Action: Prioritize for next development cycle.
Examples: Took 3x longer than expected, required multiple attempts, expressed frustration but pushed through, said “that was harder than it should be”.
Category 3: Minor Friction
What it looks like: Small hesitations or confusion, but minimal impact.
Action: Log for future improvement. Don't rush.
Examples: Brief pause before clicking correct button, “Hmm” followed by correct action, asked question but figured it out quickly.
Category 4: Working Well
What it looks like: Smooth, successful, no friction.
Action: Document and protect. Don't “fix” what works.
Examples: Immediate understanding, positive comments (“oh, that's intuitive”), completed faster than expected.
Pattern Recognition
Individual Findings vs. Patterns
- One tester said it: Maybe their specific context or expertise.
- Two testers said it: Possible pattern emerging.
- Three+ testers said it: Confirmed pattern. Prioritize.
Building a Pattern Map
Create a simple table to track issues across testers:
| Issue | T1 | T2 | T3 | T4 | T5 |
|---|---|---|---|---|---|
| Didn't notice CTA | X | X | X | ||
| Confused by pricing | X | X | X | ||
| Navigation worked | ✓ | ✓ | ✓ | ✓ | ✓ |
Patterns with 3+ marks are your priorities.
Edge Cases vs. User Types
Sometimes “edge cases” are actually user types. If a confused tester matches your target audience → important. If they're outside your target → note but deprioritize. If two distinct user types have opposite reactions → you may need to choose your audience.
The Verbatim Quotes Library
Why Exact Quotes Matter
- • Designers trust user words more than your interpretation
- • Executives respond to customer voice
- • Prevents “telephone game” distortion
- • Useful for presentations and documentation
Capturing Good Quotes
Include context:
“I thought this button would save, but it published. That's scary.” - While trying to save a draft
Tag by emotion:
- [Frustration] “Why is this so complicated?”
- [Delight] “Oh wow, that just worked!”
- [Confusion] “Wait, what does this mean?”
Prioritization Frameworks
Impact vs. Effort Matrix
Plot each issue on a 2x2:
Quick Wins
High impact, low effort → Do first
Major Projects
High impact, high effort → Plan carefully
Fill-ins
Low impact, low effort → If time
Deprioritize
Low impact, high effort → Maybe never
Severity × Frequency
Severity score (1-3):
1 = Minor annoyance, 2 = Significant friction, 3 = Complete blocker
Frequency score (1-3):
1 = One tester, 2 = Two testers, 3 = Three+ testers
Priority = Severity × Frequency
| Issue | Sev | Freq | Score |
|---|---|---|---|
| Can't find signup | 3 | 3 | 9 (Critical) |
| Confusing icon | 1 | 2 | 2 (Low) |
| Wrong button color | 2 | 3 | 6 (Medium) |
Communicating Findings
The 5-Minute Summary
For stakeholders who won't read a full report:
- 1. Who we tested (1 sentence): “5 potential customers matching our ICP”
- 2. What we learned (3 bullets): Biggest issue, second biggest, what worked well
- 3. What we're doing (2-3 actions): Immediate fix, planned improvement, further testing needed
The Detailed Report Structure
- 1. Executive Summary - Key findings in 2-3 paragraphs
- 2. Methodology - Who tested, what tasks, when
- 3. Key Findings - Organized by severity or feature area
- 4. Evidence - Quotes, timestamps, video clips
- 5. Recommendations - Prioritized action items
- 6. Appendix - Raw notes, full transcripts
Show, Don't Tell
Include video clips of key moments: 15-second clips of confusion, before/after comparisons, side-by-side of different testers hitting the same problem.
Building a Testing Rhythm
After Each Test Round
Iteration Cadence
- Early stage (pre-launch): Test → Fix critical issues → Test again. Weekly cycles.
- Growth stage: Test after each major feature. Every 2-4 weeks.
- Mature product: Quarterly deep tests + continuous lightweight feedback.
Analysis Checklist
After reviewing all sessions:
- ☐ Categorized each finding (Critical/Major/Minor/Working)
- ☐ Identified patterns across testers
- ☐ Collected verbatim quotes with context
- ☐ Prioritized using impact/effort or severity/frequency
- ☐ Created summary for stakeholders
- ☐ Defined next actions with owners
- ☐ Scheduled follow-up test