Testing with Users

Why This Matters

You’ve built something. It works, at least on your machine. But here’s an uncomfortable truth: you have no idea if it actually solves your users’ problem until you watch real people try to use it.

This chapter teaches you to run effective usability tests, synthesize feedback without overreacting, and iterate systematically. You’ll also learn how to integrate external services through APIs, because most products need to connect to something outside themselves.

Part 1: The Strategy

The Curse of Knowledge

You built this thing. You know exactly where every button is, what every label means, why the workflow goes left to right. Your users don’t. They’re seeing it for the first time, bringing different mental models, and expecting it to work like other things they’ve used.

This is the curse of knowledge, once you know something, it’s nearly impossible to imagine not knowing it. The only cure is watching people who don’t know.

What is Usability Testing?

Usability testing is observing real users attempt to accomplish real tasks with your product. It’s not a demo. It’s not a survey. It’s watching someone struggle (or succeed) while you stay quiet and take notes.

What usability testing reveals: - Where users get stuck or confused - What they expect vs. what they get - Whether your core value proposition actually lands - What terminology confuses them - Which features they ignore completely

What usability testing doesn’t reveal: - Whether they’ll pay for it (that’s validation) - Whether the market is big enough (that’s research) - What features to build next (that’s strategy)

Running a Usability Test

Before the Session

Define your goals: What questions are you trying to answer?
Prepare 3-5 tasks: Specific things you want them to try
Recruit the right people: They should match your target user
Set up recording: Screen + audio (get permission)
Prepare your script: Consistent intro and task instructions

The Test Script Structure

INTRO (2 minutes)
"Thanks for helping me test this. I'm trying to improve [product],
and watching you use it helps me find problems. I'm testing the
product, not you, there are no wrong answers. Please think out loud
as you go. I'll mostly stay quiet and take notes."

WARM-UP (2 minutes)
"Tell me a little about yourself. Have you ever [relevant experience]?"

TASKS (15-20 minutes)
"Imagine you want to [scenario]. Go ahead and try to do that."
[Stay quiet. Observe. Take notes.]

FOLLOW-UP (5 minutes)
"What was that experience like for you?"
"What was most confusing?"
"What would you change?"

WRAP-UP (1 minute)
"Thank you so much. This was really helpful."

During the Session

Do: - Stay silent while they work - Let them struggle (it’s painful, but necessary) - Take detailed notes on behavior, not just outcomes - Note where they pause, click wrong things, or seem uncertain - Ask “What are you thinking?” if they go quiet

Don’t: - Help them unless they’re completely stuck - Explain why something works that way - Defend your design decisions - Lead with questions like “Did you see the button in the corner?” - Ask “Do you like it?” (opinions aren’t data)

What to Observe

Observe	What It Tells You
Task completion	Can they do the core thing?
Time on task	Is it faster or slower than expected?
Error rate	Where do mistakes happen?
Recovery	Can they fix their mistakes?
Verbal comments	What frustrates or delights them?
Facial expressions	Where do they show confusion?
Click patterns	What do they try first?

Synthesizing Feedback

After 3-5 tests, you’ll have a lot of notes. Time to find patterns.

The Affinity Mapping Process

Write each observation on a sticky note (or digital equivalent)
Group similar observations together
Name each group (these become your themes)
Prioritize by frequency and severity

Severity Scoring

Severity	Description	Example
Critical	Users cannot complete core task	Can’t sign up
Major	Task is difficult but possible	Takes 5 minutes to find main feature
Minor	Annoying but doesn’t block	Label is confusing
Cosmetic	Polish issue	Icon looks dated

Fix critical and major issues first. Don’t let minor issues distract you from what matters.

Iteration Frameworks

The Build-Measure-Learn Loop

BUILD → MEASURE → LEARN → BUILD → ...

Each cycle should be as fast as possible: - Build: Make a small, testable change - Measure: Observe users with the change - Learn: Decide what to do next

Common mistake: Building too much before measuring. Ship smaller, learn faster.

When to Pivot vs. Persevere

Consider pivoting when: - Users consistently don’t understand the value - The problem you’re solving isn’t painful enough - Multiple users suggest a different core use case - You’ve iterated 3+ times with no improvement

Consider persevering when: - Users struggle but eventually succeed - Issues are about UX, not core value - Fixes are straightforward (labeling, placement) - Users express desire for the product

Responding to Feedback Without Overreacting

Not all feedback is equal. One user’s opinion isn’t a mandate.

Process feedback through these filters:

Frequency: Did multiple people say this?
Severity: Does it block the core task?
Source: Is this from your target user?
Alignment: Does it fit your product vision?

Red flags that feedback is misleading: - “I would want…” (hypothetical) - “My friend would love…” (secondhand) - Feature requests without a problem statement - Requests that conflict with your core value prop

Useful responses to feedback: - “That’s interesting, tell me more about when that would happen.” - “What are you trying to accomplish when you need that?” - “How do you handle that today?”

Testing at Scale

Once your product has users, you can test at scale:

Method	Best For	Sample Size
Usability testing	Deep understanding	5-10 users
A/B testing	Optimizing metrics	1000+ users
Surveys	Broad sentiment	100+ users
Analytics	Behavior patterns	All users
Session replay	Specific issues	As needed

Start with usability testing. You need qualitative insights before you can optimize quantitatively.

Part 2: Building It

Supabase Storage

Your product probably needs to handle files, profile pictures, documents, uploads. Supabase Storage provides an S3-compatible file system that integrates with your database and auth.

Setting Up Storage

Go to your Supabase dashboard
Navigate to Storage
Click New bucket
Choose a name and privacy setting

Bucket types: - Public: Anyone can read (good for profile pictures) - Private: Auth required to access (good for user documents)

Storage Policies

Like RLS for your database, storage policies control who can upload and download files:

-- Allow users to upload to their own folder
CREATE POLICY "Users can upload to own folder"
ON storage.objects FOR INSERT
WITH CHECK (
  bucket_id = 'user-uploads' AND
  (storage.foldername(name))[1] = auth.uid()::text
);

-- Allow users to read their own files
CREATE POLICY "Users can read own files"
ON storage.objects FOR SELECT
USING (
  bucket_id = 'user-uploads' AND
  (storage.foldername(name))[1] = auth.uid()::text
);

Uploading Files

// Upload a file
const { data, error } = await supabase.storage
  .from('user-uploads')
  .upload(`${user.id}/profile.jpg`, file, {
    cacheControl: '3600',
    upsert: true
  });

// Get a public URL
const { data: urlData } = supabase.storage
  .from('user-uploads')
  .getPublicUrl(`${user.id}/profile.jpg`);

File Upload Component

function ProfileUpload({ userId }) {
  const [uploading, setUploading] = useState(false);

  const handleUpload = async (e) => {
    const file = e.target.files[0];
    if (!file) return;

    setUploading(true);

    const { error } = await supabase.storage
      .from('avatars')
      .upload(`${userId}/avatar.jpg`, file, {
        upsert: true
      });

    if (error) {
      alert('Upload failed: ' + error.message);
    } else {
      alert('Upload successful!');
    }

    setUploading(false);
  };

  return (
    <input
      type="file"
      accept="image/*"
      onChange={handleUpload}
      disabled={uploading}
    />
  );
}

APIs: Connecting Your Product to the World

Most products don’t exist in isolation. They send emails, process payments, use AI, integrate with other tools. This happens through APIs (Application Programming Interfaces).

What is an API?

An API is a set of rules for how software talks to other software. Think of it like a waiter at a restaurant:

Your app (customer) makes a request
The API (waiter) delivers it to the kitchen
The server (kitchen) prepares the response
The API brings back what you ordered

Instead of building everything yourself, you use APIs: - Stripe API for payments - OpenAI API for AI features - SendGrid API for emails - Twilio API for SMS

HTTP: The Language of APIs

APIs communicate using HTTP, the same protocol your browser uses. Key concepts:

HTTP Methods:

Method	Purpose	Example
`GET`	Retrieve data	Get user profile
`POST`	Create new data	Create new order
`PUT`	Update existing data	Update user settings
`DELETE`	Remove data	Delete a post

Status Codes:

Code	Meaning	What to Do
`200`	Success	Process the response
`201`	Created	Resource was created
`400`	Bad Request	Fix your request
`401`	Unauthorized	Check authentication
`404`	Not Found	Resource doesn’t exist
`500`	Server Error	Try again later

Making API Requests

// Using fetch (built into browsers)
const response = await fetch('https://api.example.com/users', {
  method: 'GET',
  headers: {
    'Authorization': 'Bearer your-api-key',
    'Content-Type': 'application/json'
  }
});

const data = await response.json();

API Authentication

APIs need to know who’s calling them. Common methods:

API Keys:

headers: {
  'Authorization': 'Bearer sk-abc123xyz'
}

OAuth: For accessing user data from other services (Google, GitHub).

JWT (JSON Web Tokens): For your own APIs, stateless authentication.

Third-Party API Integration Example

Let’s integrate with OpenAI to add AI summarization:

// pages/api/summarize.js (Next.js API route)
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

export default async function handler(req, res) {
  if (req.method !== 'POST') {
    return res.status(405).json({ error: 'Method not allowed' });
  }

  try {
    const { text } = req.body;

    const completion = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [
        { role: 'user', content: `Summarize this: ${text}` }
      ]
    });

    res.json({
      summary: completion.choices[0].message.content
    });
  } catch (error) {
    res.status(500).json({ error: 'Failed to summarize' });
  }
}

API Security Essentials

Never expose API keys in frontend code. Browser DevTools shows everything.

Do this:

// Call your own backend, which has the secret
const response = await fetch('/api/summarize', {
  method: 'POST',
  body: JSON.stringify({ text })
});

Not this:

// DANGER: API key visible to anyone
const openai = new OpenAI({
  apiKey: 'sk-abc123' // ← This is now public!
});

Environment variables:

# .env.local (never commit this file)
OPENAI_API_KEY=sk-abc123
STRIPE_SECRET_KEY=sk_live_abc

// Access in server-side code only
const key = process.env.OPENAI_API_KEY;

Claude Code: Skills

Skills are Claude Code’s most powerful feature for reusable workflows. A skill combines instructions and code into a portable, shareable package.

What Are Skills?

Skills let you teach Claude Code how to do specific tasks, once, then invoke them anytime with a slash command:

/summarize-feedback   # Run your custom skill
/generate-test-data   # Another custom skill
/review-component     # And another

Creating a Custom Skill

Skills live in .claude/skills/ in your project:

mkdir -p .claude/skills

Create a skill file .claude/skills/test-synthesis.md:

# Test Synthesis Skill

When invoked, analyze the provided usability test notes and generate:

## Output Format

### Summary
One paragraph overview of the testing session.

### Key Findings
Bullet list of the most important observations, ordered by severity.

### Recommendations
Prioritized list of changes to make, with effort estimates.

### Quotes
Notable verbatim quotes from users that illustrate key points.

## Rules
- Focus on behavior, not opinions
- Prioritize by frequency (multiple users) and severity (blocks core task)
- Be specific about what to change, not just what's wrong
- Include positive findings too, what worked well

## Input
Paste your usability test notes below, then I'll synthesize them.

$ARGUMENTS

Now use it:

/test-synthesis [paste your notes here]

Skills vs. Slash Commands

Feature	Slash Commands	Skills
Location	`.claude/commands/`	`.claude/skills/`
Purpose	Quick prompts	Complex workflows
Can include code	No	Yes
Can span multiple files	No	Yes
Shareable	Copy the file	Package and distribute

Useful Skills to Create

feedback-synthesis.md: Synthesize multiple user interviews or test sessions

component-review.md: Review a React component for best practices

api-design.md: Design an API endpoint following REST conventions

test-generator.md: Generate test cases for a component

This Week’s Sprint Work

Sprint 3 requires:

PRD document: AI-assisted, covering your core feature
Core feature functional: With data persistence
3 user tests: Watch people use it
User feedback synthesis: What worked? What didn’t?
Platform strategy decision: Web-only, PWA, or Capacitor?

Running Your User Tests:

This week, find 3 people who fit your target user profile and run usability tests:

Prepare your script (use the template above)
Set up screen recording (Loom, QuickTime, or built-in tools)
Run each test (20-30 minutes)
Take notes during and immediately after
Synthesize findings using affinity mapping

Use Claude Code to help:

claude

I just ran 3 usability tests. Here are my notes: [paste notes]

Help me synthesize these into: 1. Key findings ordered by severity 2. Specific changes to make 3. What to keep because it worked well

Key Concepts

Curse of Knowledge: You can’t see your product like new users do
Usability Testing: Observing users attempt tasks, not asking opinions
Task Completion: The core metric, can they do the thing?
Severity Scoring: Critical > Major > Minor > Cosmetic
Build-Measure-Learn: Fast iteration cycles
Pivot vs. Persevere: When to change direction vs. keep improving
Supabase Storage: File uploads with bucket policies
HTTP Methods: GET, POST, PUT, DELETE
Status Codes: 200 (success), 400 (client error), 500 (server error)
API Keys: Never expose in frontend code
Environment Variables: Secure storage for secrets
Skills: Reusable Claude Code workflows combining instructions and code