We tried using ChatGPT for user-testing. Here’s what happened

Can ChatGPT replace real user testing? We put it to the test at Heatmap.com, our analytics tool for e-commerce revenue, by comparing ChatGPT against a remote user testing tool.

The results: AI can’t fully replace real users, but it’s a powerful way to speed up the design process, get instant usability critiques, and refine prototypes before testing with real people.

In this video we show exactly how we used ChatGPT to simulate novice and expert users, run A/B tests on prototype PDFs, and turn AI into a critique partner for faster product design.

00:00 Can AI replace your user-tests?

01:09 The limits of remote testing

02:15 How we put GPT to work

03:41 What GPT got right (and wrong)

04:43 GPT vs remote testing tools

05:23 Where AI really fits

05:59 Faster design with real users at the end