45 min
071 - The ROI of UX Research and How It Applies to Data Products with Bill Albert Experiencing Data w/ Brian T. O’Neill - Data Products, Product Management, & UX Design
-
- Technology
There are many benefits in talking with end users and stakeholders about their needs and pain points before designing a data product.
Just take it from Bill Albert, executive director of the Bentley University User Experience Center, author of Measuring the User Experience, and my guest for this week’s episode of Experiencing Data. With a career spanning more than 20 years in user experience research, design, and strategy, Bill has some great insights on how UX research is pivotal to designing a useful data product, the different types of customer research, and how many users you need to talk to to get useful info.
In our chat, we covered:
How UX research techniques can help increase adoption of data products. (1:12)
Conducting 'upfront research': Why talking to end users and stakeholders early on is crucial to designing a more valuable data product. (8:17)
'A participatory design process': How data scientists should conduct research with stakeholders before and during the designing of a data product. (14:57)
How to determine sample sizes in user experience research -- and when to use qualitative vs. quantitative techniques. (17:52)
How end user research and design improvements helped Boston Children's Hospital drastically increase the number of recurring donations. (24:38)
How a person's worldview and experiences can shape how they interpret data. (32:38)
The value of collecting metrics that reflect the success and usage of a data product. (38:11)
Quotes from Today’s Episode
“Teams are constantly putting out dashboards and analytics applications — and now it’s machine learning and AI— and a whole lot of it never gets used because it hits all kinds of human walls in the deployment part.” - Brian (3:39)
“Dare to be simple. It’s important to understand giving [people exactly what they] want, and nothing more. That’s largely a reflection of organizational maturity; making those tough decisions and not throwing out every single possible feature [and] function that somebody might want at some point.” - Bill (7:50)
“As researchers, we need to more deeply understand the user needs and see what we’re not observing in the lab [and what] we can’t see through our analytics. There’s so much more out there that we can be doing to help move the experience forward and improve that in a substantial way.” - Bill (10:15)
“You need to do the upfront research; you need to talk to stakeholders and the end users as early as possible. And we’ve known about this for decades, that you will get way more value and come up with a better design, better product, the earlier you talk to people.” - Bill (13:25)
“Our research methods don’t change because what we’re trying to understand is technology-agnostic. It doesn’t matter whether it’s a toaster or a mobile phone — the questions that we’re trying to understand of how people are using this, how can we make this a better experience, those are constant.” - Bill (30:11)
“I think, what’s called model interpretability sometimes or explainable AI, I am seeing a change in the market in terms of more focus on explainability, less on model accuracy at all costs, which often likes to use advanced techniques like deep learning, which are essentially black box techniques right now. And the cost associated with black box is, ‘I don’t know how you came up with this and I’m really leery to trust it.’” - Brian (31:56)
Resources and Links:
Bentley University User Experience Center: https://www.bentley.edu/centers/user-experience-center
Measuring the User Experience: https://www.amazon.com/Measuring-User-Experience-Interactive-Technologies/dp/0124157815
www.bentley.edu/uxc: https://www.bentley.edu/uxc
LinkedIn: https://www.linkedin.com/in/walbert/
There are many benefits in talking with end users and stakeholders about their needs and pain points before designing a data product.
Just take it from Bill Albert, executive director of the Bentley University User Experience Center, author of Measuring the User Experience, and my guest for this week’s episode of Experiencing Data. With a career spanning more than 20 years in user experience research, design, and strategy, Bill has some great insights on how UX research is pivotal to designing a useful data product, the different types of customer research, and how many users you need to talk to to get useful info.
In our chat, we covered:
How UX research techniques can help increase adoption of data products. (1:12)
Conducting 'upfront research': Why talking to end users and stakeholders early on is crucial to designing a more valuable data product. (8:17)
'A participatory design process': How data scientists should conduct research with stakeholders before and during the designing of a data product. (14:57)
How to determine sample sizes in user experience research -- and when to use qualitative vs. quantitative techniques. (17:52)
How end user research and design improvements helped Boston Children's Hospital drastically increase the number of recurring donations. (24:38)
How a person's worldview and experiences can shape how they interpret data. (32:38)
The value of collecting metrics that reflect the success and usage of a data product. (38:11)
Quotes from Today’s Episode
“Teams are constantly putting out dashboards and analytics applications — and now it’s machine learning and AI— and a whole lot of it never gets used because it hits all kinds of human walls in the deployment part.” - Brian (3:39)
“Dare to be simple. It’s important to understand giving [people exactly what they] want, and nothing more. That’s largely a reflection of organizational maturity; making those tough decisions and not throwing out every single possible feature [and] function that somebody might want at some point.” - Bill (7:50)
“As researchers, we need to more deeply understand the user needs and see what we’re not observing in the lab [and what] we can’t see through our analytics. There’s so much more out there that we can be doing to help move the experience forward and improve that in a substantial way.” - Bill (10:15)
“You need to do the upfront research; you need to talk to stakeholders and the end users as early as possible. And we’ve known about this for decades, that you will get way more value and come up with a better design, better product, the earlier you talk to people.” - Bill (13:25)
“Our research methods don’t change because what we’re trying to understand is technology-agnostic. It doesn’t matter whether it’s a toaster or a mobile phone — the questions that we’re trying to understand of how people are using this, how can we make this a better experience, those are constant.” - Bill (30:11)
“I think, what’s called model interpretability sometimes or explainable AI, I am seeing a change in the market in terms of more focus on explainability, less on model accuracy at all costs, which often likes to use advanced techniques like deep learning, which are essentially black box techniques right now. And the cost associated with black box is, ‘I don’t know how you came up with this and I’m really leery to trust it.’” - Brian (31:56)
Resources and Links:
Bentley University User Experience Center: https://www.bentley.edu/centers/user-experience-center
Measuring the User Experience: https://www.amazon.com/Measuring-User-Experience-Interactive-Technologies/dp/0124157815
www.bentley.edu/uxc: https://www.bentley.edu/uxc
LinkedIn: https://www.linkedin.com/in/walbert/
45 min