Inside DuckDuckGo

DuckDuckGo
  • 5.0 (13)
  • Technology

Behind the scenes with the DuckDuckGo team — sharing insights on product, engineering, leadership, and AI. insideduckduckgo.substack.com

  1. 1d ago

    Duck Tales: The performance review process at DuckDuckGo (Ep.47)

    In this episode, Jeffrey (Operations) and Anne (People Ops) discuss how performance reviews work in a distributed leadership model, and how the process has scaled as we’ve grown. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are only lightly edited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Jeffrey: Hello everyone, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo. We discuss the stories, technology, and the people that help build privacy tools for everyone. In each of these episodes you’ll hear from the employees about our vision, product updates, engineering, or things like approaches to AI. Anne: There’s a lot of Jeffrey: My name is Jeffrey and I’m on the internal efficiency team at DuckDuckGo. Anne: Yeah, Jeffrey: And today I’m joined with my colleague Anne. Anne, do you want to introduce yourself? Anne: Hi, I’m Anne. I’m on the People Ops team at DuckDuckGo. It’s been about five years now, which I can’t believe, but lots of exciting stuff over that time. Jeffrey: Great. Today we’re gonna be talking about our internal performance reviews and how we run them. And how would you describe our performance review process to someone who’s never heard of DuckDuckGo? Anne: Well, I think the important thing to start with is sort of describing DuckDuckGo’s distributed leadership model, because with distributed leadership, that’s going to feed into performance reviews. And so the short answer I would give with that is that there’s no one decider of performance. There’s multiple different ways, multiple different people that give input, as well as a really strict requirement that it be evidence-based. And we’ll get into this a bit later, I’m sure. But we’ve done a lot of work on the evidence-based piece and being an asynchronous organization really helps with that because mostly everything is written down. So that’s that’s the high level that I would give. Jeffrey: Who do we how do we decide what collaborators get involved with each person’s reviews? Anne: Yeah, this has been a really interesting question over the years. And I would I would just love to brag a bit about how we used to do it when it was, you know, we were a smaller organization and then talk about how we’ve evolved that because well, I think way long ago it was the organization was small enough to just kind of know. You knew who should be giving input for who. And then the organization grew to the point where we needed help figuring out, right? You can’t one person couldn’t hold it all in their head. And so you very generously helped us create an algorithm that put out a spreadsheet that basically it ingested information about people’s leadership relationships to one another, their work overlap with one another, and their experience giving feedback, among other things, calculated all of those relationships, also taking into account load. Because you know, maybe one person should review 50 people, but that’s just too many, right? So balancing those trade-offs and then making a suggestion, which was really cool. And then those suggestions obviously had to be reviewed by humans and still, you know, load adjusted and everything. And there were several rounds of human review. So that was great. And then that also came to the point where we outgrew that because even those suggestions were gonna start fraying at a point, and the human review was cumbersome. And so we’ve moved, I think quite naturally toward leaning on functional calibrators who are closer to the functional work and then pulling in the objective side of the house, which is sort of like where work is getting executed and having those be the default inputs for people. But yeah, that’s kind of how that’s changed over time. And it’s really, it’s been a really exciting problem, fun problem to be a part of. Jeffrey: For the most part it’s scaled with as the company has grown and we’ve just had to make small changes here and there. Anne: Yeah, I it’s as a people person, I also say, you know, people say how do we keep our culture as we grow? And this is similar to that where you say we don’t keep something, right? We s how do we scale something in accordance with our values as we grow? And then this I think would be a really good example of that in a really unique one too. Jeffrey: Cool. What makes evaluating performance here different from a typical company? Anne: Well, similar to what I said before, right? Distributed leadership means you have to figure out who it’s not like a managerial chain where the manager writes a performance review with peer feedback and then that gets approved by their manager and that gets approved. You know, it’s not a chain of approvals, it’s a collaborative process, which is also why that was one of the main reasons. There were several, but one of the bigger but bigger blockers for us purchasing performance review software from a company was that none of them had the functionality to have the level of collaboration that we have, like back and forth collaboration. And so there’s that. And then the real just like I said earlier, reliance on data. We are an asynchronous organization that really leverages processes in our project management tool, Asana, to fill the gaps of what a lot of live meetings would be. And I would imagine other Duck Tales have talked about no meeting Wednesdays, which is like for real. At other companies, it’s kind of in air quotes, but I would say here, not all of their companies to be clear, but oftentimes that can slip. But here it’s pretty, pretty legit. And so yeah, we’re generally meeting light, and so everything is in Asana, and you can pull that in to work summaries. And that’s also something you’ve been involved with, is really having a set of validated summary of work over the period of the year with links, right? So you can actually go and dig into the work over the course of the year as needed. And yeah, that’s the other thing I would really be I would say is is unique that the reliance on evidence and links and things you can actually see and look at and review is pretty high. Jeffrey: Right. You touched on this a little bit, but can you go into how the process has scaled as we’ve grown? Or evolved. Anne: Yeah, so yeah, that was one of the things is obviously sw figuring out how to fill those collaborative evaluator roles. Another is automation. We’ve really, you know, built our own system that integrates with where work gets done for us, which is Asana. And as a people operator, I feel blessed that I’m just given IT resources to be like, hey, you need this system to work. Here’s some really awesome skilled IT folks to help you make it a reality. And we have some really cool people on the team working to build our own performance review system that integrates exactly with where work happens for us. And so I’d say that is one that we’ve built that system and it’s evolved over time alongside other systems, right? So we have things like our daily asynchronous personal stand-up. Right. So everybody does a stand-up for themselves every day and that gets recorded. And so then that can filter back into this whole performance review process because you have a record of what you were focused on every day. You have a record of the project you were focused on every week. You we have a whole summary of the relationships of your project advisory, career advisory, everything. You know, we’ve basically been automating more of these things to be centralized in a way that’s digestible for users. And those two things have really evolved together in in a way that’s pretty cool. Jeffrey: I’m really proud of the our daily stand up we call our daily one must, but it’s been with us for a while now and it’s all asynchronous where people can fill it in in the morning and then everybody who needs to know can see what their stand up is. Cool. Anne: Yeah, that’s the transparency part too, right? You can kinda go see what anyone’s working on at any time, which is pretty cool. Jeffrey: How is this process suited for our remote asynchronous first culture? Anne: Yeah, I mean I think similar to the things I’ve said before too, it’s it’s it fits the culture because for one, we have all of this substrate to work with, right? We have all of the the asynchronous culture is really reliant upon things being written down. And when things are written down, you can really shortcut a lot of the you know, forgetting what happened or whatever, or even just misremembering. And so you know, we have like bias checks that can happen on the data because the data are actually there, obviously all reviewed by people. But yeah, so it’s fit for us for that reason because of what you’re actually working with. And then the collaboration, too, is I think this is just part of asynchronous work in general. If you’re in a live meeting discussing something, it really just kind of depends on your processing speed in the moment for what you’re thinking and deciding on. And we don’t make decisions as a general rule live. We can have discussions about them. Or maybe if it’s a really gnarly one, but generally decisions are happening in a really thought out way. And that’s the same thing with performance conversations, right? So it’s not just having a live debate about something in a calibration, right? You’re actually looking at, you’re having time to consider things and look at the actual evidence as opposed to just having a live conversation with all of its associated biases. So that’s that’s one thing I would I would say. Jeffrey: Default to having things written down helps with the evidence. Yeah. What are the challenges with the p

    Duck Tales: The performance review process at DuckDuckGo (Ep.47)

  2. Sep 30

    Duck Tales: The team and process behind DuckDuckGo’s customer support (Ep.46)

    In this episode, Beah (Chief Product Officer) and John (Support) discuss our customer support process, why we read all tickets (good or bad), and how we actually use them to improve all aspects of the user experience. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are only lightly edited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Beah: Hello, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. Today you’re gonna hear about how we do support at DuckDuckGo, and my guest is John. John, do you wanna introduce yourself briefly? John: Yeah, for sure. Yep, I’m John. I work within our marketing and communications team. And for the past few years, my main goal has been managing and improving how our support operates for the DuckDuckGo subscription and beyond. Beah: Sweet. So if you’ve used DuckDuckGo support, perhaps you’ve gotten a message from John or one of the other teammates who we’ll learn more about in this call or in this pod. So I’m Beah. If you haven’t met me, I’m on the product team at DuckDuckGo. All right, let’s jump in. I’m gonna ask John some questions and we’ll go from there. So tell me John, just to kind of set the scene, what sort of things do we provide customer support for? John: A lot. The bulk of the support that we offer is through the DuckDuckGo subscription. So for us who handle support, it means handling things like billing questions, assisting with activation across multiple devices, questions about the plans that we offer, and more technical troubleshooting related things. The subscription can be bought a few different ways, each with its own different path to activation. So a good bit of the support that we offer is helping our users navigate these paths to get more comfortable with our subscription and its features. So some of our great front-end team members built a support form where most of our tickets run through. It was built to be kind of as self-service as possible, but a category is picked and depending on what the selection is, a potential solution gets surfaced. This way, plenty of users can get an answer without really needing to submit anything or wait for us to respond to them. But basically, our help pages kind of sit behind this and there’s links to that support form on every one of our subscription help pages. It’s the first stop for most of these tickets. We’ve been piloting an AI agent on a few help pages and I’m sure we’ll get into that as well. Beah: So like if we don’t, if we can answer, if we can get people an article right then and there that answers their questions, that’s the best case scenario for everybody because they get an immediate answer, but if not, then we have a process to kind of hand it off to a human or an in-between, which is like AI assisted answers. John: Exactly. That’s right. Beah: And maybe I should say like before we had the subscription, which is the first paid product that DuckDuckGo offered, and the only one we offer, we didn’t really have a support team or support process. I mean, we had emails for different things, but like effectively we didn’t have to worry about customer support. And it was the introduction of the paid subscription that sort of like brought this problem to our doorstep and to you. Is that fair to say? Yeah, yeah. John: Yes. Absolutely. Yeah. Yeah. Perfect. Beah: Okay, cool. So tell me a little bit more about like how this all works like team-wise, who’s working on this, how have we structured that? John: Absolutely. We have a small but mighty team. So when that subscription first launched, there were a few of us that kind of rotated handling these support tickets. After a few months, we streamlined it significantly to where I was the only one who was handling things, which probably seemed like it was unfeasible and it really was. Early last year, we brought on an awesome specialist as... Beah: It, I mean it, it worked, you just didn’t sleep great. John: Yes, yeah, it worked. It worked. But there was a lot that was dependent on me and it caused a lot of lead time for some people to get responses. It was a lot to manage basically. So we brought on an awesome specialist as we continued to grow. She’s been awesome in the organization and the optimization of our entire end-to-end process. She’s got more support experience than I do by far and is a great Swiss Army knife of skill sets. So with her help, we, along with dozens of other teammates throughout the organization, we kind of overhauled the whole support structure at DuckDuckGo. And then recently, earlier this year, we brought on two more support rockstars to kind of help shoulder the ticket load to ensure that our users aren’t waiting too long before getting a real human response. So they’ve been a great addition. They’ve been working fantastically. And they’ve added an additional support layer to the mix. But yeah. Beah: Got it. And you, you, you mentioned like lots of other people. I mean, so there’s like the people who are actually writing the emails back. And then I think part of what you’re getting at, tell me if this is right, is like, you know, they are likely in many, not in all cases, but in some cases, like need to go talk to an engineer to understand what’s going on, or like a product manager or somebody else who’s been working on the product directly day in and day out and like they’re just kind of interfacing between like this huge, you know, pool of resources who know different things about DuckDuckGo. John: Yes, yeah, and that, the whole concept of working out in the open really helps make that a lot easier. So we have engineers and developers that really plug in whenever a support signal points to the need to escalate for a better response that we don’t know or we can’t handle. Beah: It has been for you. John: So that direct line to the actual builders that are behind the scenes ensures that we go straight to the source and we don’t have to wade through multiple layers of people to get an answer that’s accurate. Beah: Gotcha. Cool. So tell me, like, kind of riffing off of that, how does this fit into like our broader approach to customer feedback loops? Like we have this email support system, but we also like interface with users or get feedback from users like on social media or, you know, all these various ways, which I would say for context for listeners, like we really, we like really read that stuff and really pay attention to it. Like we’re pretty, I’d say we’re pretty obsessed in fact with understanding. Like if you’re sending us, if you’re a user and you’re sending us signals about your experience, like we are pretty hungry for that. So yeah, tell me about like the boundary, the interface boundary between like that kind of feed, those customer feedback loops and the support that you’re talking about. John: Yeah, so it’s a big circle. So essentially, the support piece to me, from what I’ve seen, I feel like I’ve been in every aspect of support, internal and external, that we offer. And this support for the subscription that we do is truly one of the most honest and raw feedback channels that we have. Most of the time, people write in when there’s a problem. Like, you’re not going to be contacting support to be like, hey, great job. Each ticket gets categorized in those categories. Beah: Though feel free to. John: Yeah, you’re more than welcome to, yes. Beah: Yeah. John: But we don’t even, we don’t handle things that are just a subscription, we handle any and all questions that come in, which I might get, we might get a deluge of tickets with me saying that. But, that said, we get a really wide range of feedback. It’s truthful. It’s unsanitized. It’s anonymous. So one thing that I’ve insisted upon since we launched this is really reading that rougher stuff on purpose. So I thrive on reading through our negative feedback. A good bit of it relates not just to the poor experience with the support team that happened from time to time, but also their experiences with the products that we offer. So both halves of those go to those who can action them. When we see things, multiple complaints about a specific piece of the product, we’ll let the people know who could actually work to fix it. So actions like these help drive rewrites to our help pages that might seem confusing. Clear responses to our users in the future. Updates on our products, bug fixes. Beah: That is a problem that would not be using my C. John: And this was also the catalyst to including suggested solutions on our help, our subscription support form. So sometimes when you select something, you’ll see a suggested solution that bumps you to a help page where you can clear things faster, like we mentioned prior. But yeah, things like social media posts. If you’re on Reddit and you see a response from someone that’s like, hey, it’s John at DuckDuckGo. That’s me. And app store reviews, those get watched by our team and others. And then they’re routed to us when they’re support related. So it ensures that those feedback loops are all connecting together. Beah: Nice. All right. So tell me a little bit, like, as our support function has grown and needed to scale as the subscription has grown, like what changes have you had to make? John: Yeah, absolutely. So for a little bit of inside baseball, in late 2022, DuckDuckGo acquired the company that I co-founded, bringing my friend and I on board to help build out the personal information removal piece o

    Duck Tales: The team and process behind DuckDuckGo’s customer support (Ep.46)

  3. Sep 23

    Duck Tales: How we use AI at DuckDuckGo while staying true to our principles (Ep.45)

    In this episode, Marc (Engineering) and Zach (Data Science) discuss how we use AI internally: our most common uses, how we use it privately, and why our principles matter more when using AI, not less. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are only lightly edited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Marc: Hi, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. My name is Marc and I’m on the engineering team. I have with me Zach. Zach, you want to introduce yourself? Zach: hi, yeah. I’m Zach Deane-Mayer on our data science team here at DuckDuckGo. I am still a Duckling, so I’ve been here for about ten months. and I work on building our internal AI systems. Marc: Great. yeah, so today we’re gonna talk even more about AI, which is Zach’s specialty. I think today we’re gonna talk specifically about how we use Duck or sorry, how we use AI internally at DuckDuckGo. so yeah, I guess to start out, as a company, you know, we’ve adopted AI pretty pretty heavily internally as a tool to help us move faster. can you talk a little bit, Zach, about the the approach that we’ve taken here? Given that I think this you were in on the ground floor with this, right? Zach: Yeah, yeah. So DuckDuckGo as a company takes I I think a somewhat unique approach to AI as we take a unique approach to everything we do. We very much have a culture of of things like first principles thinking and and you know our own spin on things. Marc: Right. Zach: And a very important part of what we do are our privacy and security. so before we rolled out many of these tools, we spent quite a bit of time internally getting comfortable with their failure modes and the blast radius potential for for those failures. One of the first places we started using AI a lot was with coding tools. So right off the bat with Cursor and then Claude Code. And you know, one, the models are very, very good at writing code. But then two, if you are effectively using tools like git and GitHub, an agent that’s limited to writing code can only do so much damage if your code base is backed up in GitHub and you don’t let, for example, people force push to main or whatever like that. Like if your GitHub is properly configured. And so we had a lot of questions like that early on. And then, you know, once once we were confident, we really were able to kind of let open the floodgates and let people really start start using both Cursor, Claude Code and now to a lesser degree Codex to to write code and automate as much of engineering as they can. Marc: Yeah, yeah, that’s great. you mentioned privacy, right? Like that is obviously it’s a core tenet of our products. It’s also a core tenet of how we operate here. and yeah, to your point, we do tend to do things a little bit differently. so I’m curious how the privacy kind of shaped our internal workflows with respect to AI, given that, you know, w we even though we don’t collect user data, we do still have things that we obviously don’t want to to share more broadly. Zach: Yeah. Yeah, absolutely. so our privacy engineers very much think of like a an information hierarchy of stuff that is you know very I don’t know what the word is I want to use for the scale, but there’s a there’s a high to low scale. and so there’s certain information that we don’t even save to disk in our production servers. There’s certain information that is stored in memory and is never retained anywhere in our systems. Marc: Right. Zach: and that is fundamentally the safest form of data. If you don’t if you don’t store a piece of information, it’s it’s very hard to accidentally store it in the wrong place and in a way that you don’t you don’t want to do. and and that kind of principle thinking applies to AI too. Like if if an AI system doesn’t have access to certain data, it It can’t do the wrong thing with that data. and one of those ways to prevent access is just by by fundamentally not not storing the data in the first place. As you go through that hierarchy, you know, there there’s information that we’re more and more comfortable with working on internally, like for example, you know, time series traffic of you know how many users are on our various websites, like that sort of aggregated data that can’t be tied to an individual person. That’s a lot safer to start saying, hey, I’m gonna, you know, I’m on the data science team, I’m gonna do some data science work and look at, you know, whatever, whatever it is that the set of questions I have. and then I think the kind of the bottom piece that is incredibly important is we pay a lot of attention to our data processing agreements that we have with all of our vendors, but especially our AI vendors. and the contractual agreements you have with people about what they are and aren’t allowed to do with your data are are also very important. so so yeah, there’s there’s a spectrum of things and you go talk to any privacy engineer, I mean you can very quickly learn how just how deep the problem goes. Marc: Yeah, yeah, it’s interesting you mentioned principled thinking, right? Like I I think that’s something that is a core part of the culture here. And I I think it’s fair to say that we have, you know, spent a lot of time talking about how to adapt AI to our principles as opposed to adapting our principles to AI, right? which I feel like you see a lot in the industry right now. Zach: Yeah, yeah. Yeah. Yeah. And and and and that’s I mean that’s a that’s a great point. Like two of our principles that matter a lot to me personally are questioning assumptions and validating direction. Marc: Absolutely, yeah. Zach: and you know, I’m I’m like the the AI, you know, waving, you know, the flag and the the pom-poms and saying, let’s go AI internally. Marc: Right. Zach: but like I I find I have to like, you know, remind people, just because we’re using AI doesn’t mean we throw our principles out the window. Like, Marc: Absolutely. Zach: I am you know, again again as a leader here, I am more interested in like like I want to validate direction too. And so I am very interested to hear from people places where AI isn’t working. Like, like I want to see, you know, the PR that’s a 7,000 line of code wall of text where a team’s going, whoa, like we can’t we can’t review this, let alone ship it. Marc: Right. Zach: And those those kinds of failure modes, like AI is rife with them. And we don’t, you know, we don’t just look at an AI’s output and say, well, an AI made it, so let’s ship it. Like you you should use AI as as you know, a lever against, you know, with your principles, not not as a as a reason to throw your your principles out the window. But like again, like just questioning assumptions and validating direction, like you know, AIs are notoriously bad at questioning assumptions. Marc: Right. Zach: Like this will happen all the time, even with the latest models, where you’ll be like, hang on a sec. Like you said X and then you said Y, which implies not X, like, but you never connected the pieces together in in in in the chain of logic. Marc: Sure, sure. Zach: and so I I think I think your principles matter more when you’re using AI. Marc: that’s interesting. Yeah. Yeah, that I think that’s really well put. actually leads us into our next question to a degree. there’s yeah there’s a lot of hype right now, obviously with AI. I mean to put it mildly, right? the hype cycle is has been super interesting. there’s a lot of concern about AI in the workforce today, right? And we’re certainly not immune to that, although we don’t really see it that way. as a company. So maybe can you talk a little bit about how we’re how we’re getting so many people to use it effectively and and you know where we’re trying to get to. Zach: Yeah. Yeah, so AI is a very powerful tool. and it’s also in some ways a a slippery tool. Like it’s it’s changing so fast. It there’s not like a long history of like I don’t know, I I can’t I can’t think of an equivalent technology, but there’s there’s not like The field has changed so fast, everybody’s brand new to it. Marc: Yeah. Zach: And there’s a lot of surface area of of unknown unknowns, of things you don’t know that that can go go wrong. Marc: Well. Zach: I I heard a joke I really like that like working with AI, it’s like a cross between Amelia Bedelia and the guy from Memento. Marc: That’s good. Zach: and and it’s a great joke, but it’s like, you know, one, interprets everything you said too literally. and then two, it forgets tomorrow everything that you said to it today. And like, like those two problems, like, you know. It’s you know, I I’ve been playing around with Fable a lot recently. It’s an amazing model for one-shotting really difficult programming problems. But you still get that sometimes over literality, like like what you say wasn’t quite interpreted the way you meant it, and you have to be careful of that. And then the other side of like, you know, you come in tomorrow and you have to curate context that you take over between sessions. And I I think honestly that that latter part has been the part for me that has been the most difficult to adapt to is just what do you remember and how do you remember and how do you persist that memory? Marc: Sure, sure. Zach: And that’s very much something you have you have to build for yourself. and and you know from the individual level to the team level,

    Duck Tales: How we use AI at DuckDuckGo while staying true to our principles (Ep.45)

  4. Sep 9

    Duck Tales: DuckDuckGo will find and remove your personal information — like your name and email — from sites that store and sell it (Ep.44)

    In this episode, Konrad (Engineering) and Shilpa (Engineering) discuss Personal Information Removal: what is a data broker, how we remove your records without ever sending your data to our servers, and the ‘whack-a-mole’ of constantly changing data broker sites. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Learn more about the DuckDuckGo Subscription here. Disclaimers: (1) The audio, video (above), and transcript (below) are only lightly edited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Konrad: Hello everyone and welcome to Duck Tales. We will look under the hood of DuckDuckGo, talk about stories, technology, and people who are building the privacy tools that you all use. And in each of the episodes, you’ll hear from different employees about our vision, updates to our products, how we do engineering, or how we approach AI. So today I’m your host. My name is Konrad Zwinell. And I’m working at DuckDuckGo for seven years now. I work on the front end team at the very beginning. Then I run privacy engineering team. And now I’m back on the front end team working with Shilpa, who is my guest today. And Shilpa is running a personal information removal tool team. And she will tell us more about it today. Shilpa, why don’t you introduce yourself? Shilpa: Yeah, hi. I’m Shilpa. I have been in DuckDuckGo for, I think, a year and a half now. I joined in on the Apple team working on iOS and macOS. And for the last few months, I’ve been driving the team that does works on the personal information and mobile product. Konrad: Nice. OK, so first question that people may have after hearing this intro is, what exactly is personal information removal? Could you tell us, high level Shilpa, what’s the product and what it does? What’s the idea behind it? Shilpa: Yeah, so in the US, this is predominantly a US-based product. In the US, there’s entities called data brokers that basically harvest your personal information from various different sources, your online activity, social media, what have you. And they collect that information, and they basically buy and sell it as a commodity for advertisers and different entities. And our product basically goes and looks at all of these data brokers, looks for your personal information. If it’s there, then it submits a request to remove it. And if the data broker is complying, they will go ahead and remove it and we will be able to verify the removals for you as well. Konrad: Nice. So basically reducing your footprint on the web, reducing availability to your personal information, such as address, phone number in some cases, relatives, if I remember correctly. Shilpa: Yeah. Konrad: So they collect quite a bit of information about you and make it really easy for advertisers, but also other people to look it up. For FV, Shilpa: Yeah. Konrad: yeah, so that’s a bit scary. How does it work from the user perspective? Let’s say you want to use PIR. You want to try it out. How is the experience looking? Could you share with folks how it looks UI-wise and what you have to do to try it out? Shilpa: Yeah, so today, actually in some platforms, you can even go try it out for free, part of the experience on macOS or Windows, where you can go ahead and put in your personal information, some basic stuff, like your name and date of birth and city and you will be able to go see, we will go ahead and do a search on the various different data brokers that we support. And it should be able to show you all of the different brokers that have your information, the information that they do have. And the first time, if you see this actually, it’s a little bit, it’s actually quite a bit scary to see how much of your information is out there. And literally anyone can do a search and pull up a lot of your information about you, your family and stuff. So yeah, once we do the search and then we’ll go ahead and go to those, once we know that your records are there on those specific brokers, we go ahead and submit requests to opt you out. Konrad: Thank Shilpa: And then there may or may not be some steps in between all of which we will take care of for you. And then usually there’s a delay from a couple of days to a few weeks maybe. Then if the record is removed, we will go ahead and do a scan again and make sure that your record is removed. And then we’ll report to you that that record’s been flagged as removed. And not just that, we also do periodic scans to make sure we do like maintenance scans to make sure that your record is removed and it continues to be removed. Because the way that data brokers work is once they remove it, they’re not very intentional or deliberate about these records. They’re just collecting and gathering from whatever sources they’re getting. So if it’s removed, doesn’t mean it stays removed. Sometimes it goes, sometimes it comes, sometimes it’s merged from other sources. So it’s important to continue to keep, keep removing it. Konrad: Yeah. And we keep on adding new brokers, right? Meaning, you keep the PIR running, we will add new brokers and we will opt you out out of those brokers. So it’s not only like insurance that your records don’t pop up on the same brokers, but as we find new brokers starting business and appearing on the web, we will try to opt you out from those too, right? Shilpa: Yeah, yeah. It’s the whole world of data brokers is very fluid. There’s new ones coming up. There’s mergers happening. There’s some of them are going defunct. Some of them are temporarily defunct. They come back. So we’re all constantly keeping an eye out for how things are changing, fixing it and adjusting for the changes they’re making, as well as constantly looking out for new ones that are coming up, seeing how we can support removals on those sites. Konrad: That’s right. And I guess that’s our biggest challenge, right? Like fighting with this ever-changing websites and brokers kind of even fighting back on purpose, trying to kind of avoid our product, like make it not work. Would you say it’s the main challenge? Do we have like some other challenges? Shilpa: I would say that is pretty much the main challenge. They are deliberately or not constantly changing things on their side. And since they don’t have a very clean, clear request that we can make to them, just to say, here’s my record, go remove it. But it involves usually going through and scraping the website. It’s not a very clean process, which means that we have any minor changes on their end leads to us breaking things on our side. So we have to constantly keep an eye out and monitor, make changes, fix. So this constant whack-a-mole, I think, is one of the biggest challenges. It’s also a good engineering challenge. So we are always thinking of various different ways to innovate and try to be as flexible as possible and see how we can actually get ahead of them. That actually is like the main engineering challenge related to this product for us. Konrad: Yeah, we both like a challenge. That’s why we are on this team, right? OK, let’s see. So we are both engineers. Shilpa: Absolutely. Konrad: So maybe let’s do some shop talk and talk how this work under the hood. So yeah, can you tell us a bit about that? Shilpa: Yeah, so doing a little bit of engineering talk, normally you would imagine that, you know, like we take the personal information from the users and then we go and make a nice clean API call directly to data brokers and they give us a response and we can say either success or failure. But that is actually not the case. Data brokers do not make our lives easy. What we actually have to do is basically what you would do if you were to manually go ahead and remove it. You would go to their website, type in your information, do a search, see if the record is found, and then go to another form. Again, type in an opt out request, and in between there may be various different stages. Depending on the data brokers, you may have to solve some captchas. You may have to get a confirmation code from the email or click on a link in the email. So we do all of that in the entire lifecycle of search and remove a record. So yeah, and I think one unique thing to the way we do versus our competitors is that when we take your personal information, it stays on your device. We do not send it off to our servers where it stays forever and potentially vulnerable to various other attacks. So we’re very particular about keeping all of your personal information within your device. So we just make the request from your browser within your device and keep it in your device so that it’s a lot less liable to be breached in any other way from server side. Konrad: Right, because that’s our approach. Because the competition existed when we first started building this product, but we didn’t just copy the existing model. We thought it through and decided that what they are doing, it’s not going to work for us because we don’t want to receive personal information from our users. We don’t want to process that. We don’t want to deal with it. And so we came up with this model where everything happens on device, which has some pros. It has some cons. But yeah, we are making it work. And yeah, as you said, it’s challenging. But it’s the right thing to do to keep your data private. OK, so we talked a bit about what’s under the hood. We talked about the challenges, how those brokers are fighting back and by also going offline and changing their websites all the time and changing those forms. And yeah, so that has been our focus for a bit. You started talking, you mentioned like a bit about like flexibility, but we also do monitoring. We do other things like how we are adapting to this like well, complex environment. Shilpa: Yeah, so con

    Duck Tales: DuckDuckGo will find and remove your personal information — like your name and email — from sites that store and sell it (Ep.44)

  5. Sep 2

    Duck Tales: The 3Ds of DuckDuckGo. Delight, discovery, and dependability (Ep.43)

    In this episode, Gabriel (Founder) and Beah (Chief Product Officer) discuss our focus on delight, discovery, and dependability (the 3Ds), what they are, and how they shape our day-to-day work. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Show notes: Listen to our episode on Hack Days here. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Beah: Hello and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In this episode, we are going to be talking about what we internally call the 3Ds: discoverability, dependability, and delight. And if we If you haven’t met me, I’m Beah. I’m on the product team and I’m here today with Gabriel, founder and CEO of DuckDuckGo. Anything anything you want to add to your introduction, Gabriel? Gabriel: No, I do not. If you’ve watched this at all, you’ve already seen me a bunch. So and they’ve seen you as well, so Beah: Yes. Cool. Okay. So yeah, let’s just jump in. 3Ds. What what’s the deal? What I mean, I said what each D stood for, but like contextualize this whole thing. Gabriel: Yeah, so part of the way we run DuckDuckGo as a company, just like internally, like how we operate, is we have a what we call our top priority, which is like the main thing that we think we need to focus on as a company. And that’s changed every few years. But like right now we’re in the middle of and we’ve been doing it for I think around 18 months, something like that, this 3Ds focus on discovery, dependability, and delight as you pointed out. And those are product things. So it’s really a focus on making our product better in those kind of three areas. And just to walk through them briefly, discovery is we actually have a lot of stuff in our product that people do not know about. So there’s the basic thing that a lot of people still don’t even know DuckDuckGo exists and we have a product and we’re a privacy company. But when you get a little beyond that, you know, we’re a search engine, we’re a browser, we have a subscription now. In our browser, you can sync your passwords and bookmarks. There’s all sorts of features, and like the holistic product is a lot bigger than even most of our users realize. And so it’s kind of discovery of those features. Dependability is a little more straightforward. That’s like, okay, you’re a user, the product should be very dependable. But I mean that means like reliable. You can switch to it easily. All the workflows work well. There’s no rough edges in it. Obviously, that’s a tall order when you run a search engine and a browser and a VPN and all these other things we do. But we’ve been nail we’ve been working down the list of all the little issues that people you know complain about, rightly so, that might have been issues with their bugs in our product and we’re trying to fix them. And then the third one is delight, which is, you know. I think most intuitive, I guess, but even harder to actually articulate how we get there. Beah: Yeah. Gabriel: But it’s like you have a delightful product experience. It’s those products and experiences that you are unexpected, really bring you joy, like you really want to share with your friends and family. Like we want to bring that delight into using DuckDuckGo. Beah: Yeah. Gabriel: So the focus has been on all three of those, because we think we need to do all three to make a great product. And so we have projects and people working on, you know, basically all three at all at all times at the moment under our top priority. Beah: Yeah, just to add to the discovery point, I mean we get feature requests and feedback. Feature requests for features we have. Like people just straight up asking for features that exist. Yeah. Private browsing. Yeah. Yeah. Gabriel: Yes. I would argue that might be our top request. Things like, yeah. Yeah. This is a core discovery issue. Yeah, and even more fundamental than that is like a lot of a lot of search engine users, because that was our first product, and we have the most search engine users, don’t even know we have a browser. And our most browser users on mobile don’t realize we have a desktop browser. Beah: Yeah. Gabriel: So regardless of features within all those things, like just at the very basic level, people don’t even know we offer a browser. And they should. Yeah. Beah: Yeah. And like sometimes it’s as simple as like changing like making something more visible and then other times I think it’s about the language we use or like the way we characterize something so that people know that it’s what they’re looking for. So yeah, there’s like that one’s yeah. Gabriel: Yeah, one more point on that too. Like that one’s subtly hard as well, because like you don’t discovery means that you have to get people to know about it to your point. And you maybe think of something, which is that you have to get in front of people then with a message, but that can be annoying. Like it look can look like an ad, right? Beah: Mm. Gabriel: And so that’s a delicate balance. It’s like, yeah, you’re a cer you’re a private search user, you probably want our private browser because it’ll actually make the benefit of privacy much more you know, better in your life if you have them paired together. However, you don’t want a thousand messages of, hey, we have a browser downloaded now. So there’s there’s like this delicate balance of like trying to get awareness to get the people who will who want it who want it to get it without annoying people who don’t want it. Beah: Mm. Totally. Yeah. And there’s just like it’s the competition of many things. Like you probably want our browser. When in our browser, you probably want your passwords and your bookmarks. And then you probably want those synced across devices. And you probably want this thing that’s opt in turned on. And like, yeah, you can just we have to I think we spend a lot of time trying to make sure that we’re not like bombarding people with too much messaging or promos and it’s but then but then again we get feedback requests for things that are sitting there somewhere behind the scenes. So yeah. Gabriel: So just before we move on to that one practically, then what that means is we’re constantly running experiments basically of like how best to do these in product kind of promos. And and and I think to your point, like it’s better to do them. The naive approach is to shove them all in onboarding, which people then whiz by. But the better approach is like very contextual, like maybe like when you do a bookmark for the first time, ask people if they want to sync. And then the other part of discovery, just to end that part, is is just our external marketing. So like not in talking a lot of in-product stuff, but we also spend a lot of time just doing general marketing, especially in the US, about educating people about our different experiences. Beah: Yeah. Yep. So we ended up talking a lot about discovery, but just another like zooming out question. How how did we land on these besides the fact that they all share a D, why did we pick why did we pick these three things? Gabriel: That’s that’s clearly part of it. Yeah. Yeah. Yeah, I mean if you think about the whole product kind of marketing funnel, which is you need awareness in a product, then you need to know a little bit more about its benefits and value proposition to consider using it. They call that consideration. Then if you kind of are interested in that, like in our case, like privacy benefits, you may trial and then that initial trial point on like is kind of an activation in the funnel. And then you want to retain users and then ultimately a referral. You want users to be happy enough that they’re referring you. So that’s the traditional like marketing funnel. And different companies have issues you know, at different parts of the marketing funnel where they would want to focus. I think it was pretty clear that we had some gaps in in in kind of all parts of the funnel almost equally. And the 3Ds from a product perspective map onto that pretty well. So, like the discovery part is just we keep measuring these surveys, but most people still don’t know about us, and most people don’t know about our different products. So there’s that’s an obvious gap. The dependability one, we’re constantly running user surveys. Beah: Mm. Gabriel: And so once people are activated, you know, our retention has gotten a lot better in the last three years because we’ve really improved the dependability of all our products. But we could tell it was a little bit of a leaky bucket, like people had come in and really wanting to love us and try us, but having an issue switching, or you know, we didn’t have, you know, sync or, you know, with their bookmarks. And so we felt that that was also deserved a lot of a focus because we could see that people weren’t always staying when they were activating. And then the third part is word of mouth. So we had grown mainly through word of mouth. The most most of our users have come through word of mouth. But a lot of that was on the back of privacy being one of the biggest stories in tech. And I I hope that’s coming back, and it should, because AI has a lot of privacy issues. But you know, since the pandemic that has been less so. And so to get word of mouth without a big news cycle all the time, you need delight. Beah: Mm-hmm. Gabriel: And so it’s really mapping these to a funnel and making it a product focus of that funnel. Beah: Yeah. Gabriel: And then something that has either if you’re gonna pick names, you either have to rhyme or they have

    Duck Tales: The 3Ds of DuckDuckGo. Delight, discovery, and dependability (Ep.43)

  6. Aug 26

    Duck Tales: 32% of AI users tell AI things they've kept from other people. Our new study on AI's privacy problem (Ep.42)

    In this episode, Zac (SVP, User Insights) and Mario (Marketing) discuss our study on the AI privacy problem: from stats and surprises, to AI hot takes, and how DuckDuckGo can help. Show notes: Learn more about the study and our findings here If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Mario: Okay, there we go. Hey everyone. Welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, the technology, and the people that help build the privacy tools for everyone. In each episode, you will hear from employees about our vision, product updates, engineering, marketing, but also our approach to AI. Today you’re going to hear about some brand new research, very fresh, that we ran about what Americans actually think about privacy and AI. And I’m here today with someone who happens to have the exact same haircut as me. And that means luscious locks, of course. Zac, do you want to introduce yourself? Zac: Yeah, hey everybody, I’m Zac Pappas, SVP of User Insights. I’ve been at DuckDuckGo for 14 wonderful years, and I’ll be talking to you about this study today. Mario: Wow, fourteen years. Yeah, we’re we’re very lucky to to have you here. you’ve been, you know, working in Insights for a long time. You’re one of the first, very first DuckDuckGo employees, is this correct? Yeah. Zac: Yeah, I think it was technically the second employee back when we were just a few people around the Paoli office here in Pennsylvania. So it’s yeah, it’s been a long fourteen years. but I think it’s been really exciting and things changed so so frequently that it doesn’t really feel like we’re doing the, you know, same insights or kind of same research over and over again. Mario: Yeah. Well that’s that that’s pretty good. But it’s also useful because it means you also have about as long as a view as anyone on how people feel and think about privacy, you know, what they want protected, what they say versus what they do. So I think this is gonna be an interesting conversation. And yeah, by the way, you’ve introduced yourself. I’ve been talking, I have been rude, I haven’t introduced myself. My name is Mario, I’m in the on the marketing team. I’ve been here for almost four years now. a fun fact about myself is that I like to tell people I once came in fourth in the Eurovision Song Contest. For those who don’t know, it’s like it’s big music kind of song festival and competition in Europe. So yeah, I like to say I once came in fourth, but kind of. So not exactly. And I’ll just leave it at that. what about you, Zac? Do you have any fun facts you want to share? Or obnoxious facts? Also loud. Zac: nothing that interesting or stand out. I guess y like most people, when I have the time I will go out to my garage, fire up the forge, and do a bit of blacksmithing. so pretty standard stuff. Mario: Wow, okay, that’s that’s quite cool. I mean, this is this is Duck Tales, not Zac Tales. If it was Zac Tales, I would ask you a lot more questions, maybe put in a request. So Zac: Next episode, yeah. Mario: yeah, next time. Okay, let’s let’s jump in. So we as a company and you particularly, since you’ve been here for so long, we’ve been studying how people think about privacy for close to, I know, in this case like 15 years or more. You know, what’s what’s new is doing that specifically for AI. in June, our user insights team surveyed 1944 US adults, all quota balanced to census on age, gender, region, et cetera, about their AI privacy concerns, what they know, what they don’t know, what they actually do about it. you’ll find the methodology on the show notes if you’re interested. And this is this is what we want to talk about today. So, Zac, let’s let’s start with the juicy stuff. To you, what was the most surprising thing in this study? Zac: I think given the fact that AI has been so pervasive and around for so long, most surprising thing was that I think forty-three percent of everyone knew none of six basic facts about how how AI chats are handled. So that includes a third of active AI users who can’t, you know, recall or kind of signal what basic data privacy and data handling practices that the AI services that they’re using are actually adherent to. Mario: Well— well, can you tell us a little more like what facts you’re talking about? Zac: Yeah, I actually wrote them down so I can get them correct for you. So the facts we asked people about were AI chats can be subpoenaed by courts and law enforcement. AI chats can be made public in a lawsuit. Employers can access work or enterprise AI accounts. Settings to delete or stop storage vary by provider. Free accounts often have weaker privacy than paid accounts, and humans can review your chats. Some of these maybe you would expect most people who haven’t dug too far into AI or privacy policies, they maybe wouldn’t think of those things first, but certainly human employees. Employees can review your chats. We’ve seen instances of this in the past with other companies in data collection when there’s a breach or when there’s a mishandling of data. Those are some pretty basic things that I would have assumed people at least had some suspicions about, but it’s surprising that they didn’t. Mario: Yeah, this is this is a little yeah, it was a little unexpected the the level of of at least to me, you know, listening to this. I I was looking like looking a little bit here at the at the details. Like I saw, for example, on the that the AI chats can be subpoenaed by the government, for example, which means they could then made accessible in public because there were a lot of news about this, about some cases about AI companies. It was about only twenty five percent of people knew this. So why do you think people don’t know these facts given it has been you know, so much on the news. I know this isn’t in the study, but just curious about what you would you Zac: Yeah, I I mean i it it’s definitely a different technology. I think it’s been a lot more opaque, the way that it’s been integrated and with the speed that it’s been rolled out, and the kind of the breadth that it’s been rolled out to most products that people are using. I think they get a different flavor and a different version of AI everywhere that they look with, you know, incomplete data practices, meaning they haven’t, you know, execute at the executive level, most companies haven’t really figured out how to deploy AI safely. So what they’re getting is say a rushed experience, but something that probably wasn’t built in the best interests of data privacy or kind of long-term sustainability in that regard. so here, yeah, it’s a it’s a maybe a combination of it being kind of a pervasive, like broad applic broad applied technology as well as how fast it has risen. and maybe my own personal experiences, there isn’t that that much useful information for a mainstream person, maybe who’s not a developer or a coder who’s you know gonna be really in the weeds of the technology. It’s kind of hard to find some of the better, you know, more mainstream or general approaches than how to use it safely. Mario: Yeah, yeah, I can s I can see that. Thanks for that. So yeah, we know what’s the most surprising thing about the study for you, but like asking the bigger question, like, are people worried about privacy in AI in the AI context? Zac: Yeah, absolutely. Seventy-five percent of people are worried about privacy when using AI. And that has been consistent for years as we’ve been running these studies. And we’ve seen this with external research as well. invasion of privacy, just the discomfort and uneasiness of using it and not knowing how the data’s being handled, and surveillance are usually around like 50% of what worries people the most when it comes to AI, and usually something privacy specific is in like the top two factors among those that are concerned about or like withholding use and not using it at all as a c it’s a contributing factor to why they’re not. Mario: Yeah, and and yeah, and I I remember some of these studies that are published, like outside of our own, and it’s always like main concerns about AI and there’s usually one that’s like, you know, economic impact or job losses and then privacy is usually hanging there on the top. So I I definitely review myself. Zac: Yeah, absolutely. And you know, I I think that gets lumped in with a lot of things. particularly the thing that I think most people do end up worrying about is that it’s going to have some impact on their financial situation down the line. That’s your personal details being involved in a breach, credit card details, somebody getting into or unfreezing your credit account and trying to open a new line of credit, like a lot of these things at the end of the day, I think, get back to the comfort and stability that people have with either their finances or their way of life. is it gonna jeopardize you know a job application that I have in the future to find something that you know I’ve put publicly online or didn’t know that I was putting publicly online through use of a service? So I think that we should expect that it’s going to be a top issue, especially as we kind of continue into the future as we integrate AI with all of these other services that are being used, whether it’s other apps or subscriptions. Mario: Yeah. Yeah. And and I also remember other studies that that you guys have run and where it isn’t like there’s a one single thing about privacy that that people are worried about. There’s a broad concern about privacy. And and for some peo

    Duck Tales: 32% of AI users tell AI things they've kept from other people. Our new study on AI's privacy problem (Ep.42)

  7. Aug 5

    Duck Tales: The DuckDuckGo Subscription Pro-tier — private AI for AI power users (Ep.41)

    In this episode, Beah (Chief Product Officer) and Chris (Partnerships) discuss the DuckDuckGo Subscription: the core protections, private AI, and how the new Pro-tier serves AI power users. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Beah: Hello and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo to discuss the stories, technology, and people that help build privacy tools for everyone. In this episode, you are going to hear about our Pro subscription plan. And my guest today is Chris Calvi. Chris, do you wanna briefly introduce yourself before we jump in? Chris: I— I would love to. Thank you for having me back. I was on— a quick ad for people who have not listened to the original subscription overview Duck Tales episode. Gabriel and I did that six months— I don’t know, actually probably nine months ago at this point. So go find that. But I’m coming to you from sunny New Jersey. And I’m a member of the partnerships team here at DuckDuckGo. And I’ve been at DuckDuckGo for about five years. I love it here. And most recently helped run efforts to add this new Pro plan to the subscription, which we’ll be talking about on this episode. Beah: Nice. I’d like to think of this episode as the Pro edition of the previous podcast episode that you recorded with Gabriel. Chris: Okay, that’s fair enough. Beah: Just to set the bar. Chris: Yeah. Beah: All right, and I’m Beah, I’m on the product team, if this is your first time meeting me. And yeah, I think we can kind of jump in. Like, let’s start with— can you just explain, Chris, what even is the subscription, what we’re talking about? Chris: Yeah, and I always like to start with, like, why the subscription first, and then I’ll say what the subscription is. So the why: you know, DuckDuckGo has a ton of fully featured, excellent, free-to-use, no-login-required functionality. And we’ve had that for, like, what, fifth— 17 years? How— how long has DuckDuckGo been on? We’ve had it for a very long time. And— and so we, though, realize that there are— there’s extra functionality and features and protections that we would like to offer to— to our users, that you can’t offer— either can offer free, for free, because they have a material c—, like, a substantial cost to us, or it can’t be, like, ad-supported like the search. So— so two years, a little over two years ago, we launched the DuckDuckGo subscription. And— and that includes four protections— that are aligned with what I just mentioned, things that we sort of, like, can offer for free. And so there’s the identity theft restoration, which internally we call IDTR. And that is essentially, if you— Beah: Not me. I call it identity theft restoration because I try to avoid— Chris: You— you don’t like to use acronyms. Beah: I’m anti-acronym, yes. Chris: As a culture, we typically are. Though I like to break the rules. I break the rules. Beah: I continue. Chris: So in the event that your identity is ever— is ever compromised, your n—, your identity stolen and is used, for instance, to open up another credit card— for— that’s not— that you didn’t open, we have a phone number that you can dial into. Actually, let me share my screen and I can show you what I’m talking about. Beah: You can do a live demo of calling a phone number and see what happens. Just don’t do that. Chris: That would be risky. No, I’m just kidding. It would be good, but it would also be boring. So let me know— does this work? Do you see? Beah: I see. Chris: All right, so to get here— after— this is kind of like jumping ahead, but let’s say you have a subscription to DuckDuckGo already, you would log in, you’d— you’d come to your— if you have the app installed, you’d come in and you would select DuckDuckGo subscription. And in subscription settings, if you didn’t already have this device connected to your subscription, you would have to do that as a first step. But once it’s connected, then you have it and you can just come in here and go to the element of the subscription that you want. So let me walk you through those pieces of the subscription. The first one I was mentioning was identity theft restoration. And so what you would do is— in this example I mentioned where somebody opened a credit card in your name, and you’d want to re—, you wanna deal with that, you’d come in here and you would call our— this has some information for you about how it’s gonna go. You will click “talk to an advisor” and then you would make a simple phone call and they would help you make things right with your— your credit card companies, your banks, your credit bureaus, things like that. So that’s identity theft restoration. The— the subscription, as I mentioned, has four key pieces of functionality— we call it four protections in one. The second one is personal information removal. So this would also be available from your— your settings screen. You’d click on here and you’d click “open personal information removal.” And similar to the identity theft restoration, this would bring you to a page where it shows you what— what it’s going to do, what it is and how it’s going to help you and how you get started. And what personal information removal is: in the US, we have this insidious issue where data brokers essentially aggregate personal information about you, about everyone in the country, and they put it on the internet or they— they’ll sell it. Sorry, they’ll put part of it on— on the public web and then sell part of it. And this includes things like your name, your address, your phone number, your date of birth. And even relatives, like— that, like, lived in the household with you. I’m sure you’ve probably seen this if you’ve ever searched for yourself. And so with personal information removal, we go through and we opt you out of the listings on over 60 of those sites. And we’re adding more all the time. And so you would just come in here, you’d click “get started,” and this would ask you a few things, a few pieces of information which stays on your device and does not go to our servers. That’s important to mention here. And then goes through a process that’s run locally from your machine and takes some time. It takes, like, anywhere from a few hours to a few weeks or longer to get you fully removed from all of these sites. So some of the sites can act quicker than others, but this will go through and— and opt you out from being listed on those data broker websites. The— the— and interrupt me with any questions you have about this, any of these functionality, but we went through— Beah: No, this is good, this good. Chris: We went through the identity theft restoration, the second one being personal information removal, and then the third one is the VPN. And the VPN is— stands for, since we don’t like acronyms, virtual private network. And what it— yeah. Beah: I guess I do allow VPN. I’m gonna allow it. Chris: I mean, but— what it— what a VPN is, is it— is it, we set up a— anonymous secure server, and we have them in 30, or over 30, countries around the world. So wherever you’re at, you can connect to the closest one. And— and what it will do is securely— it encrypts your traffic and securely and privately tunnels it through these anonymous servers and then out to the resource that you’re requesting on the other end on the internet. So, like, all of your internet traffic is basically being tunneled through this— this private secure server and then out the other end. And so this— why— what, like, what this does is it protects you in— in— in two— two, like, key ways. One is your ISP, the internet service provider that you’re connected to, won’t be able to see the sites that you are accessing, like what sites you’re visiting. And that helps you because they’re not then able to— either it makes it more difficult for them to censor the content that you can access, and/or build an ad profile about you, for like an advertising profile, and then like resell that, for instance. Beah: Okay. Chris: And then on the other end, it— it protects you— the sites that you are visiting, they’re going to see the IP address, like, kind of like who you are, of the server. And so they don’t actually know who you are unless you were to, like, log in, for instance. So it gives you protection from both— in b—, on both of those cases. But, like, the— the— the use case that is, like, most common for— for the VPN, and I should have switched over to showing you this, is the— is when you’re at, like, a public Wi-Fi location, in an airport or a hotel or, like, a— a coffee shop, you don’t need those— you don’t need them seeing what websites you’re accessing. So VPN gives you that extra security. And I— as I had mentioned, you can change the location that you’re— the server, where the server is. I just keep it on “nearest location” and I’m on it right now. I keep it on all day. I use it all the time. And so I think that’s, like, the key thing about VPNs. I don’t know if the— if I did that justice. Beah: You did. You’re doing such a good— I hate to rush you along, because you’re doing such a good job of explaining these features, but we have to get to the Pro plan. So I think you’re kind of getting to here. Chris: The key thing. Okay, so let me just get you— yes, let me get you that last thing. So the— Beah: Yes. Chris: So the fourth feature is Duck.ai. And Duck.ai is free. But if you want higher limits and be able to do more on it, then— if you hav

    Duck Tales: The DuckDuckGo Subscription Pro-tier — private AI for AI power users (Ep.41)

  8. Jul 29

    Duck Tales: How we're making it easier to use Duck.ai, our private AI chat. (Ep.40)

    In this episode, Beah (Chief Product Officer) and Mihai (Design) discuss how the Duck.ai team prioritizes what to build, how we use research and user feedback, and a demo of recent UX updates. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Beah: Hello, and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering approach. And in this particular episode, you’re going to hear from Mihai, who is here to talk about the various UX improvements that we’ve made to Duck.ai recently. Mihai is on the design team. And do you want to just give your give a quick introduction of yourself, Mihai? Mihai: Sure, thank thanks for having me. My name is Mihai. Originally from Moldova, lived across the world in many places, now based in Switzerland, joined DuckDuckGo maybe around three years ago, and I’ve been on the Duck.ai team almost from the very beginning, so I’ve seen the product evolve since then dramatically. So so yeah, we’ll talk about it today. Beah: Nice. Well thanks for joining. For anyone who hasn’t met me, I’m Beah. I’m on the product team here. I’ve been around for a while. So yeah, I’m gonna go ahead and ask Mihai some questions about his work on Duck.ai and we’ll just dive on in. So I think I said we’re talking about improving Duck.ai UX, and I’m not sure I said that if you’re not familiar, UX is just a fancy abbreviation for user experience, even though we all know experience doesn’t start with an X. So this means like, you know, making the features that we have and the functionality that we have more usable, tweaking the functionality to better like suit people’s needs and desires, and just make the product more delightful and easier to use. You feel free to amend that definition. But just to kind of like set Mihai: Feel free to amend definition. Beah: the set the tone a little bit, Mihai, tell me a little bit about like how we even decide to work on something. Like how do we prioritize UX user experience changes in Duck.ai land? Mihai: Yeah, as designers, we’re basically I I tend to think of myself as the diplomats that we we try to talk to as many people to get our work done. But we use various signals to inform these decisions. Some of some of them are more perfect, some of them are imperfect. There’s no There’s no reality where we’re working with perfect information. So sometimes we form assumptions, sometimes we rely on research. User feedback, internal feedback. Sometimes we have strategic bets that we place as an organization. We have dashboards with metrics. We have just no we know the competitive gaps that we have, and sometimes we just rely on instinct. So we Beah: That’s good. Mihai: we bundle all of these together in our decision-making process. Some of these signals they surface a problem, Beah: Yeah. Mihai: others validate the direction and it’s on us to use our own human judgment to determine the right trade-off for for these things. And in the end it narrows down to to looking at where the biggest overlap is with the user problems that we have and where we’re taking the product. Beah: Yeah, or do all the hand and just then follow it. Mihai: Yeah, maybe this is more kind of like general response, but no one decision was the same. Beah: Yeah, right once. Yeah, yeah. Mihai: So it’s it’s it’s all based on experienced people in the room making the the calls based on the information that we have. Beah: Yeah, I mean one thing that I’m guessing people I I bet a lot of people would underestimate the degree to which we’re obsessed with user feedback. Like you can’t completely rely on user feedback to tell you how to build a product because, you know, hopefully the people building the product have additional insights about what’s possible and what good looks like that users can’t articulate or shouldn’t take the time to articulate. It’s not on them. But that aside, like we obsess pretty hard on feedback. Like we read an insane amount of feedback. We have systems for categorizing it and making sense of it. I mean, I’ve been caught up in like long discussions where we are really deeply trying to interpret a comment somebody made on Reddit or in the app store and like try to figure out the best way to address it. And so I’m just pointing that out because I think that might not be obvious. Like I think there’s probably a lot of, you know, for a tech company that serves millions and millions of users, I’m not sure that’s I’m guessing that’s not the norm. So that is a big part of it. But then like beyond that, you know, there’s a bunch of other signals we use, like Mihai said, and I know in Duck.ai on that team, like you guys have a rubric, right, that you’re using to actually like evaluate in a structured way a bunch of project ideas and feedback is one of the inputs to that rubric, but there are others as well. So also very maybe distinctive of DuckDuckGo is I think it’s a hyper structured and rational decision making process. Would you agree with that, Mihai? Mihai: Yeah. It’s it’s quite unique and I think the the great thing is that a bunch of us, everyone here, they’re scoping their own projects, they have their own ideas, they uncover different things that they’re pitching in and that goes into that rubric. And at certain times we those categories of how how we score the rubric, they’re weighted. So sometimes certain elements are more important than the others. And that help us kind of look at what what’s important to us right now and what we’re getting from the users and all the projects that are strong candidates for us to tackle tackle in the next roadmap, for example. Beah: Cool. Yeah, so let’s talk about something that did bubble to the top recently. Like, can you give me an example of an a UX improvement that you’ve worked on? Mihai: Yes, so the first example is dictation. I’ll share my screen real quick. Cool, so this is our interface here. So users keep asking for dictation features, so it’s this microphone in the input field. There’s certain people that for them is they would use the input field. They’ll rather talk than type. It’s either a prompt is really long, or you they just want to do it hands free. And shipping this was actually great because it it got us in line with the competitors as well. But internally we we were kind of i it it was the group was split in terms of like if we wanna do it until that user feedback kept coming back, kept coming back. And we decided to to pull the trigger. For from the UX standpoint, what was exciting for me personally is voice interfaces, they have a big affordance problem. That it’s really hard sometimes to know what’s the system status for that and to avoid the jargon is basically you need to know what the what the app is doing at any point in time. And that was an interesting challenge on top of just delivering the feature. And if I if I trigger it, we we do three things here. We label things clearly for people to know what’s going on. You also get a visual cue, this waveform that provides feedback, and as you see me pause, that waveform reacts to my voice, but also the input field has changed its its shape not the height or anything but what’s inside it now is geared toward towards a different input which is not text and we’ve added this nice glow animation to it which is which is really cool so and then it’s trust transcribing and giving back the the text. Another one should should I should I give a few more? Alright. Beah: Nice. Beah: Yeah, yeah. Give me another example, please. That’s good. Mihai: Alright. Another one is batch deletion, which is another feature that was requested by by users. This one is a little different because our we’re famous for our fire button. You you can basically burn all your data and stuff like that. But in Duck.ai specifically we had the reality that you could either delete one chat at a time or all of them. And nothing nothing in between. And we we decided to that we need to give people more precise control. To do it in just one step. Whenever you have 30 chats, for example, and you just want to delete seven at a time, it’s quite annoying to go one by one. So this one, it’s not groundbreaking or anything. It’s been a pattern on the web for quite some time. How you can batch select the different things. This is a great example, thinking back to the signals that we spoke. It’s a great example when you need to know where to follow the process for something when whenever you’re designing something that’s more unknown and whenever you need to trust your instincts more. And this is a project that was basically that where we we used our experience more and the project was short this was delivered and and shipped quite quickly because of that. And what the feature does you can go in and select many chats you want to delete. We have different entry points to how to do it. Usually Beah: Yeah. Mihai: we know people click the fire button to open it so we surface a button there for people to actually not delete everything but select multiple ones or for maybe power users whenever something’s selected you can shift click and enter this mode and delete delete your chats. So yeah, this one the reason I’m I’m asking is we spoke about all these signals and how complex things are, but this one was th three designers, we got in the room, we did a thirty minute thing, then I spent maybe two hours on it on edge cases, we put it to review to a wider gr

    Duck Tales: How we're making it easier to use Duck.ai, our private AI chat. (Ep.40)
See All (47)

Ratings & Reviews

5
out of 5
13 Ratings

About

Behind the scenes with the DuckDuckGo team — sharing insights on product, engineering, leadership, and AI. insideduckduckgo.substack.com

Information

  • Creator
    DuckDuckGo
  • Years Active
    2025 - 2026
  • Episodes
    47
  • Rating
    Clean
  • Copyright
    © DuckDuckGo
  • Show Website
    Inside DuckDuckGo

You Might Also Like