The Agentic Edge Podcast

Aampe

0.0 (0)
Technology
Updated Biweekly

Conversations about agentic workflows, tools, and infrastructure! edge.aampe.com

10/23/2024

Your CMS is a gold mine: time to dig

Schaun Wheeler and Arpit Choudhury talk about Content Management Systems (CMS) and how they are a treasury of information that can be leveraged for hyper-personalization. They explore how generative AI is transforming CMS, the importance of data quality, and the potential of LLMs in enhancing user experiences. The discussion also covers the challenges of implementing AI in recommendation systems, the significance of integrating external data, and the future of personalization in various applications, including food delivery and travel. KEY POINTS * CMSs are often seen as repositories of content to present on webpages but they can be structured to use data for purposes beyond webpage display such as personalization, communication, and discovery. LLMs can significantly improve CMS data management. * There are opportunities in what you bring in and what you take out of the CMS. Information provided by vendors can be better used by turning it into structured data. * Generative AI can be leveraged to dynamically change the way content is displayed to users each time. * Cleaning your CMS can allow you to extract additional value from it. LLMs enable on-the-fly data cleaning and enhancement without the massive manual effort previously required. They can also analyze content tone, create detailed sub-categories, and generate descriptive tags. This data can power features like sophisticated filters and AI-driven personalization through agentic learning. * The alignment problem in AI is when there is a gap between user expectations and AI recommendations. The hardest part of aligning input to output in an LLM is getting the context right. * In a CMS, an input is anything that resides in the CMS, such as item IDs, item names, and item descriptions. An output is information that is presented to the user. * LLMs work when choosing between a limited number of options. When there are hundreds of thousands of items, LLMs aren't able to sort through all of the information consistently and coherently. * LLMs can be used to clean your CMS data and to further personalize the presentation of information. They can extract tone, feel, and other metadata that enhance both recommendations and browsing experiences. * Lots of information can be pulled in from various sources to address the discovery problem, but it must be presented selectively. Companies that help users find relevant information quickly will retain users better than those who don't. * A/B testing tells you what works for the largest minority of users, so giving everyone that experience is a bad idea. We can use AI agents to learn recommender preferences and cater to the changing needs of each user, as they are scalable and can give each user the required attention. CHAPTERS: * 00:00 Introduction * 01:22 Developments in CMS technology * 03:45 Clean your CMS * 07:34 Leveraging tags for recommendation * 09:42 The alignment problem * 14:53 Applications of LLMs * 19:14 Data extraction capabilities of LLMs * 25:29 Value propositions in food delivery * 28:36 The potential of CMS * 31:34 Integrating external data * 33:45 Agentic applications in CMS * 40:13 Inputs and outputs This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

43 min
10/17/2024

Agents, impact confidence, and prior beliefs

In order to mimic human decision making, agents need to estimate the probability that a particular intervention will move user behavior in the right direction, but also estimate how much confidence they should put in that probability assessment. The first estimate is easy - or, at least, straightforward. The second estimate is a lot harder, but it sits at the core of how agents balance exploration of new possibilities with exploitation of lessons already learned.There is no purely empirical way to estimate confidence. Frequentists estimate it implicitly, whereas Bayesians estimate it explicitly, but in both cases the estimate has to come from the very squishy realm of prior beliefs. When I was designing how our agents would assess confidence, I had to do some thinking about the properties of statistical distributions and how those map to different expectations about confidence we can be about a lesson learned from just a single intervention. No one wants to stake too much on a single interaction - in any context - but if we stake too little then we end up discounting a lot of valuable information so much that we ultimately end up ignoring it.In the attached video, I walk through some of the ways we've addressed the topic of estimate confidence with our agents. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

6 min
10/08/2024

Embedding space let agents communicate with each other

When you're on a team, you tend to figure out problems collectively - you chat with your teammates, compare notes, and swap stories about what has or has not worked in the past. What if the team is a team of agents? How do agents compare notes?An embedding space. An embedding space is a high-dimensional mapping of data points (words, users, events, etc.) to arrays of numerical values. It's like a map, but whereas a typical map can only be 2D or maybe 3D, an embedding space can have many dimensions as you want. This allows you to pick any point in the space, and quickly find other points that share similar values.Aampe agents maintain multiple embedding spaces for different look-back windows: the last 1 day, the last 7 days, the last 30 days, etc. So when an agent is working with a user and the user just won't respond to any outreach, the agent can use the embedding spaces to quickly find other users who *have* responded recently, and get ideas from those user's agents.It's impossible to visualize so many dimensions on a 2D surface, but I did a little dimensionality reduction in the attached video to try to give a little intuition about how embedding spaces work. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

5 min
10/07/2024

Get comfortable with Decision Automation

Paul Meinshausen and Schaun Wheeler talk about the key components behind decision-making and what goes into automating it in this discussion hosted by Arpit Choudhury. They emphasize that successful decision automation includes understanding the nuances of decision repeatability, outcome evaluation, user preferences, and business constraints. They discuss the importance of designing systems that can learn effectively from user interactions, the limitations of current approaches, and the ongoing need for human input to provide crucial context. KEY POINTS: 1. Decisions can be broken down into three components: * The decision set (options of the decision) * The outcome set (what happens based on decisions) * The information set (data relevant to making decisions) 2. Decisions can be described to see how they can be handed to machines. This can be done based on criteria such as whether the decision is answer set constrained and the repeatability and frequency of the decision. 3. Recognizing which problems are constrained and repeatable can help leverage past experience to tackle them systematically and save a lot of time. 4. Identifying the information relevant to making a decision helps constrain the decision set. 5. Recommender Systems are well suited for large decision sets with thousands of options. They should share relevant information to learn about users while not overwhelming them. The way information is presented affects both user decisions and system learning. 6. In decision automation, humans are needed to provide context and business constraints, and design interfaces that capture meaningful signals from users. We can show AI agents how to make decisions like we do. 7. Challenges in Decision Automation: * There's often a mismatch between available data and user preferences. * Business problems often involve opinions rather than facts, making outcome evaluation difficult. * Most software are built so that the burden of making something repeatable and learning from that repetition falls on humans. * When relevant information is not shared with users, it can make it difficult for agents to understand the information influencing users’ preferences and limit effectiveness. CHAPTERS: * 00:00 Machines and decisions * 04:30 What is a decision? * 06:32 Constrained, repeatable problems * 08:00 Components of a decision * 09:03 The answer set * 12:00 Constrained resources * 12:58 Repeatability and frequency * 15:21 Delivering recommendations * 16:55 Evaluating outcomes * 19:44 LLMs making decisions * 22:03 Facts and inference * 24:13 Not just for users * 27:14 The information set * 28:33 How recommender systems evaluate answers * 31:12 Relevant information and why we automate decisions This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

37 min
10/04/2024

Transforming distributions to encode business priorities

Agentic learners aren't tools or systems or programs. They're additional headcount. As with a a human team, one of the most important aspects of managing a team agentic learners is to know how you can give feedback and instruction. I made a video recently (link the comments) about representing user preferences as two parameters in order to do a random draw from a beta distribution. The probability parameter tells the agent how much an intervention is expected to positively impact user behavior, and the signal parameter tells the agent how confident it should be about that probability.Draw from a beta distribution that has high probability but low signal, and the result may very possibly be a low number. This is what keeps agents from getting stuck in local maxima.However, all of that deals with the explore/exploit tradeoff. That's a common tradeoff of agents to make, because an agent needs to know whether to continue to try as-yet unexplored options, or focus on options that have already proven successful (even if still other options might be even more successful). But in any realistic business context, agents also need to navigate a tradeoff between what a user prefers and what a business needs. While it doesn't do a business any good to push options on a user if the user really hates those options, it can often make sense to give a user their second- or third-choice option if doing so can meet a business objective. To do that, remember this simple formula:v ** (log(t) / log(a))Three parameters:v: the actual value drawn from the beta distribution.a: the anchor value of the distribution - I usually use 0.5, because it's central and intuitive.t: the target value to which to move the anchor.So if v = 0.5 and a = 0.5 and v = 0.66, then using that formula would transform a draw of 0.5 to 0.66. The value of the formula is that is transforms any draw from the distribution, whether it's 0.5 or 0.98 or 0.00023. It effectively uses the anchor and target values to shift the entire distribution.So if you're a business and you need your agents to prioritize the selling of a particular product line, you can raise the target value of the distribution for that product line and agents will prioritize interventions about that product, even if the user's probabilities for that product tend to be lower than the probabilities for other products. By the way, this video mentions a previous video on parameterizing beta distributions. You can find that here: This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

5 min
10/04/2024

Beta distribution draws and signal strength

How do you make a computer program show curiosity? You randomly draw from a beta distribution. Yes I'm serious. A core problem in agentic learning is the navigation of the explore/exploit tradeoff. Agents have many options to choose from when trying to engage users, and they always need to balance the goal of taking advantage of lessons already learned with the goal of trying options that haven't yet been tried. If an agent only ever explores, it never optimizes. If an agent only ever exploits, it prematurely optimizes and ends up a local maximum - something that's better than some options, but not nearly as good as it could be. Aampe agents store two different weights for every possible action in each of their action spaces - a probability of influence (roughly analogous to an expected success rate), but also a measure of signal strength, which encodes how much evidence is backing the probability. The agent needs both of those so it can hedge it's bets when it has to operate on low evidence, and can double down when it's able to operate on a high evidence. Also, this video references another video on Interrupted Time Series analysis. You can find that here: This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

6 min
10/04/2024

How agents view behavioral signals

Agents that work on the basis of behavioral data need to be able to make judgements about how successful (or not) their actions are. Unlike an A/B test or a multi-armed bandit, where you use success rates over many users to determine the relative value of different options, an agent needs to be able to try one specific action with one specific user and make a judgement call about whether that action inclined the user in the right direction. There's not such thing as a success rate when you're dealing with a single intervention for a single user. Instead, Aampe agents use a version of Interrupted Time Series analysis, simplified for use with sparse data. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

4 min
09/02/2024

You can't Google your way to a recommender system

Schaun Wheeler and DJ Rich delve into the intricacies of building recommender systems in this podcast hosted by Arpit Choudhury. The discussion highlights the steps to developing a recommender system, practical advice for startups, and the evolving landscape of recommender technologies. KEY POINTS: * Identify the Problem: Recommender systems address the "discovery problem" by helping users sift through vast amounts of options to find relevant content quickly. Recognizing this problem is crucial before diving into solutions. * System Components: Recommender systems are complex and involve multiple components such as: * Item Inventory: Detailed metadata about items (e.g., descriptions, categories). * User Interaction History: Data on user interactions with items (e.g., views, purchases). * Recommendation Model: The core model that filters and ranks items based on user preferences. * The Learner: An important component, which trains the model and separates it from the model's deployment phase. * Build vs. Buy: Should one build a recommender system from scratch or use existing solutions? For many startups, buying an off-the-shelf system can be more practical due to advances in data infrastructure and the complexity of developing a bespoke system. Buying a system can also free up resources for other critical areas. * Practical Recommendations for Startups: Instead of getting bogged down by complex models initially, startups are encouraged to start with simpler models and leverage existing infrastructure to implement a functional recommender system. * Innovations in Recommender Systems: Schaun is interested in combining traditional methods with reinforcement learning to enhance system performance. DJ is excited about research addressing causal questions and handling sequential recommendations. REFERENCES: * "Are we really making progress?" This paper is a replication study on recommender algorithms and shows that many DL approaches couldn't be reproduced or could be beaten with linear methods. * "Deep Exploration for Recommender Systems" This paper talks about sequential decisioning for RSs (where you consider more than just one item recommendation). * "Two Decades of Recommender Systems at Amazon" This paper is a retrospective on what's work well at Amazon. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com

29 min

Conversations about agentic workflows, tools, and infrastructure! edge.aampe.com

Creator

Aampe
Years Active

2K
Episodes

19
Rating

Clean
Copyright

© Aampe Inc
Show Website

The Agentic Edge Podcast