Generative AI
AI Ethics
UX Research
UX Design
Designing Granular, Low-Friction
Feedback for AI Systems
Users can highlight and report issues naturally, without leaving the flow of conversation
PROJECT OVERVIEW
While investigating how people give feedback on AI tools like ChatGPT, I noticed that users often highlight parts of a response when something feels off. I designed a lightweight reporting tool that builds on this behavior, making it easier to flag issues like bias without disrupting the conversation flow.
This academic project leveraged ChatGPT as a case study but was independently designed and executed without any affiliation to OpenAI.
My Role
UX Designer
UX Researcher
Team
1 Project Manager
1 UX Researcher
1 UX Designer
Duration
3 months (2024)
Skills
User Interviews
Affinity Mapping
Data Analysis
Rapid Prototyping
01 / PROJECT HIGHLIGHT – INTERACTION PATTERN
Turning behavior into design direction
80% of participants highlighted text
Revealing a common reading behavior we hadn’t prompted.
"I didn’t even realize I was highlighting the text..it just helps me stay focused and follow along as I read."
02 / PROJECT HIGHLIGHT - SYSTEM GAP
Bias reporting is hidden behind basic feedback
“I just rewrite the prompt when the answer’s wrong. I don’t think about giving feedback.”
"I thought the thumbs-down just meant I didn’t like the response. I didn’t know
it was actually for feedback."
50% of users misunderstood the thumbs icons
Users unsure if they meant preference or feedback.
Users avoided disruption
They rewrote prompts instead of reporting issues.
03 / PROJECT HIGHLIGHT - gray areas
Users didn’t view the entire response as problematic
All users notice issues in just part of the response, not the entire answer
Feedback tools should allow for nuanced, context-aware reporting instead of treating every flagged response as entirely wrong.
“I know it’s biased, but sometimes it feels more like personalization than a problem.”
“I just saw one part as the issue. The rest of the answer was fine.”
04 / PROJECT HIGHLIGHT - Solution
Users can effortlessly highlight and report issues directly within the flow of the conversation
We uncovered more than what’s shown above.
Let’s break down the thinking and research that informed this solution.
01 / project background
As concern around AI bias increases, current reporting tools still aren’t built for everyday users
While GenAI systems continue to evolve, most rely on expert audits to assess fairness and safety. But expert reviews alone often miss real usability issues and subtle harms that emerge in everyday use.
A quick look at how ChatGPT, Claude, and Perplexity handle AI response reporting tools
Goal
How might we create a seamless reporting experience that empowers everyday users to flag algorithmic harms, while equipping AI teams with clear, actionable insights to improve fairness and trust?
02 / USER INSIGHTS
Users feel lost, ignored, and interrupted when trying to give feedback
Why
To understand how real users interact with ChatGPT’s current feedback tools
What
To evaluate how discoverable and clear the existing reporting features are.
Who
Tested with 8 frequent ChatGPT users (ages 22–45) from diverse backgrounds and levels of tech familiarity.
How
Participants interacted with ChatGPT using a think-aloud protocol, while a notetaker observed remotely via Zoom.
What we heard
What it means
Where we noticed it
"I’m not sure what the thumbs up or down even means. Am I reporting a problem, or just saying I liked it? And if I want to explain why, there’s no way to do that."
Feedback tools feel unclear and lack direction
Users hesitated or ignored feedback buttons when unsure of their purpose
"I know it's biased, but sometimes it feels more like personalization than a problem it really depends on the context."
Not all bias is inherently negative; nuance matters
Users discussed mixed feelings about bias, depending on context and intent
"If I have to stop what I’m doing just to report something, it completely takes me out of the moment. It’s easier to just ignore it and move on."
Reporting disrupts user flow
Several users paused mid-task, gave up on reporting, or simply retyped their prompt to avoid breaking their flow.
"I never really know what happens after I report something. Does anyone actually read it, or is my input just lost in a pile of data?"
Reporting feels like a black hole
Users asked what happens after submitting and expressed doubt it made a difference
03 / Design Opportunities
Users Need Clarity, Control, and Confidence to Give Feedback
These four breakdowns helped shape our design priorities.
Core Challenge
Design Opportunity
Feedback tools were unclear and lacked direction
Clarify the purpose of feedback buttons and separate rating from reporting
Reporting interrupted users’ flow
Let users report issues within the chat interface without losing focus
Bias felt personal or contextual, not always harmful
Allow users to flag specific parts of a response and choose the type of bias
Users received no confirmation or follow-up after reporting
Show confirmation and provide visibility into what happens after submitting a report
04 / Concept Exploration
We explored dozens of concepts before landing on the right one
Before committing to a solution, we explored a wide range of ideas through Crazy 8s, storyboarding, and speed dating. We tested safe, risky, and “out there” approaches to understand how users reacted to tone, effort, and emotional impact. This helped us validate core needs and identify design principles that shaped our final concept.
Diverging quickly with Crazy 8s
We began with four “How Might We” questions based on key user pain points. Each teammate rapidly sketched 8 ideas for each prompt, helping us cover dozens of possibilities around motivation, workflow, transparency, and system trust.
HMW make reporting effortless?
HMW reduce reporting friction?
HMW help users understand bias?
HMW create trust through follow-up?
Visualizing core needs through Storyboarding
From our sketches, we created storyboards to visualize key moments and emotional responses. Each storyboard mapped to a user need and tested ideas at three levels:
Safe
Low-friction, familiar UX
Risky
Pushes for stronger feedback or engagement
Out There
Extreme designs that test limits of trust, emotion, or control
This storyboard represents one of our “out there” concepts, designed to test how users might react to public accountability and social pressure as a motivator. While most users found the alarm element too extreme, the concept sparked valuable conversation about trust, tone, and emotional safety in reporting workflows.
Testing reactions and refining direction with Speed Dating
Speed dating is a rapid feedback method where users react to a series of short, scripted storyboards. Instead of evaluating final designs, they respond to early ideas helping us understand emotional responses, expectations, and deal-breakers before committing to a direction.
What worked
In-flow interaction
Users preferred feedback options that stayed within the chat interface.
Subtle confirmation
Lightweight signals (like “report received”) built trust without disruption.
What Didn’t Work
Social pressure designs
Public alerts or shaming mechanisms felt invasive and broke trust.
Overly complex bias framing
Asking users to analyze training data or “guess” bias felt overwhelming.
"If I wanted homework, I’d go back to school. Just let me flag it and go."
“It’s weird that I need to be emotionally available just to report something.”
This activity gave us clear signals on what not to build, and strengthened our confidence in low-friction, optional, and user-controlled reporting experiences.
05 / VALIDATION
Validating core interactions with paper prototypes
With limited time in our class project, we chose a low-fidelity paper prototype to quickly test the core reporting flow: highlight → report → confirm. This method let us validate interactions, surface user expectations, and gather feedback fast—before investing in high-fidelity design.
Highlight
Categorize
After clicking “Report Bias,” users choose from a list of predefined bias categories within the same dialog.
Expand
Track
05 / FINAL PROTOTYPE
Designing the end-to-end feedback experience
To bring our solution to life, we designed a seamless reporting flow that lives inside the chat experience. It reduces disruption while improving clarity and user control. The final prototype integrates four key steps: highlight, categorize, expand, and confirm. Each interaction was shaped by user feedback, making the tool feel lightweight and optional, while still offering depth when needed.
Other Projects
Other experiences I’ve designed.
