Recsperts - Recommender Systems Experts

Marcel Kurovski
  • 5.0 (3)
  • TECHNOLOGY
  • UPDATED MONTHLY
Recsperts - Recommender Systems Experts

Recommender Systems are the most challenging, powerful and ubiquitous area of machine learning and artificial intelligence. This podcast hosts the experts in recommender systems research and application. From understanding what users really want to driving large-scale content discovery - from delivering personalized online experiences to catering to multi-stakeholder goals. Guests from industry and academia share how they tackle these and many more challenges. With Recsperts coming from universities all around the globe or from various industries like streaming, ecommerce, news, or social media, this podcast provides depth and insights. We go far beyond your 101 on RecSys and the shallowness of another matrix factorization based rating prediction blogpost! The motto is: be relevant or become irrelevant! Expect a brand-new interview each month and follow Recsperts on your favorite podcast player.

  1. 2 DAYS AGO

    #26: Diversity in Recommender Systems with Sanne Vrijenhoek

    In episode 26 of Recsperts, I speak with Sanne Vrijenhoek, a PhD candidate at the University of Amsterdam’s Institute for Information Law and the AI, Media & Democracy Lab. Sanne’s research explores diversity in recommender systems, particularly in the news domain, and its connection to democratic values and goals. We dive into four of her papers, which focus on how diversity is conceptualized in news recommender systems. Sanne introduces us to five rank-aware divergence metrics for measuring normative diversity and explains why diversity evaluation shouldn’t be approached blindly—first, we need to clarify the underlying values. She also presents a normative framework for these metrics, linking them to different democratic theory perspectives. Beyond evaluation, we discuss how to optimize diversity in recommender systems and reflect on missed opportunities—such as the RecSys Challenge 2024, which could have gone beyond accuracy-chasing. Sanne also shares her recommendations for improving the challenge by incorporating objectives such as diversity. During our conversation, Sanne shares insights on effectively communicating recommender systems research to non-technical audiences. To wrap up, we explore ideas for fostering a more diverse RecSys research community, integrating perspectives from multiple disciplines. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (03:24) - About Sanne Vrijenhoek (14:49) - What Does Diversity in RecSys Mean? (26:32) - Assessing Diversity in News Recommendations (34:54) - Rank-Aware Divergence Metrics to Measure Normative Diversity (01:01:37) - RecSys Challenge 2024 - Recommendations for the Recommenders (01:11:23) - RecSys Workshops - NORMalize and AltRecSys (01:15:39) - On the Different Conceptualizations of Diversity in RecSys (01:28:38) - Closing Remarks Links from the Episode:Sanne Vrijenhoek on LinkedInInformfullyMIND: MIcrosoft News DatasetRecSys Challenge 2024NORMalize 2023: The First Workshop on the Normative Design and Evaluation of Recommender SystemsNORMalize 2024: The Second Workshop on the Normative Design and Evaluation of Recommender SystemsAltRecSys 2024: The AltRecSys Workshop on Alternative, Unexpected, and Critical Ideas in RecommendationPapers: Vrijenhoek et al. (2021): Recommenders with a Mission: Assessing Diversity in News RecommendationsVrijenhoek et al. (2022): RADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News RecommendationsHeitz et al. (2024): Recommendations for the Recommenders: Reflections on Prioritizing Diversity in the RecSys ChallengeVrijenhoek et al. (2024): Diversity of What? On the Different Conceptualizations of Diversity in Recommender SystemsHelberger (2019): On the Democratic Role of News RecommendersSteck (2018): Calibrated RecommendationsGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 36m

  2. 10/12/2024

    #25: RecSys 2024 Special

    In episode 25, we talk about the upcoming ACM Conference on Recommender Systems 2024 (RecSys) and welcome a former guest to geek about the conference. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (01:56) - Overview RecSys 2024 (07:01) - Contribution Stats (09:37) - Interview Links from the Episode:RecSys 2024 Conference WebsitePapers: RecSys '24: Proceedings of the 18th ACM Conference on Recommender SystemsGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    40 min

  3. 10/01/2024

    #24: Video Recommendations at Facebook with Amey Dharwadker

    In episode 24 of Recsperts, I sit down with Amey Dharwadker, Machine Learning Engineering Manager at Facebook, to dive into the complexities of large-scale video recommendations. Amey, who leads the Video Recommendations Quality Ranking team at Facebook, sheds light on the intricate challenges of delivering personalized video feeds at scale. Our conversation covers content understanding, user interaction data, real-time signals, exploration, and evaluation techniques. We kick off the episode by reflecting on the inaugural VideoRecSys workshop at RecSys 2023, setting the stage for a deeper discussion on Facebook’s approach to video recommendations. Amey walks us through the critical challenges they face, such as gathering reliable user feedback signals to avoid pitfalls like watchbait. With a vast and ever-growing corpus of billions of videos—millions of which are added each month—the cold start problem looms large. We explore how content understanding, user feedback aggregation, and exploration techniques help address this issue. Amey explains how engagement metrics like watch time, comments, and reactions are used to rank content, ensuring users receive meaningful and diverse video feeds. A key highlight of the conversation is the importance of real-time personalization in fast-paced environments, such as short-form video platforms, where user preferences change quickly. Amey also emphasizes the value of cross-domain data in enriching user profiles and improving recommendations. Towards the end, Amey shares his insights on leadership in machine learning teams, pointing out the characteristics of a great ML team. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (02:32) - About Amey Dharwadker (08:39) - Video Recommendation Use Cases on Facebook (16:18) - Recommendation Teams and Collaboration (25:04) - Challenges of Video Recommendations (31:07) - Video Content Understanding and Metadata (33:18) - Multi-Stage RecSys and Models (42:42) - Goals and Objectives (49:04) - User Behavior Signals (59:38) - Evaluation (01:06:33) - Cross-Domain User Representation (01:08:49) - Leadership and What Makes a Great Recommendation Team (01:13:01) - Closing Remarks Links from the Episode:Amey Dharwadker on LinkedInAmey's WebsiteRecSys Challenge 2021VideoRecSys Workshop 2023VideoRecSys + LargeRecSys 2024Papers: Mahajan et al. (2023): CAViaR: Context Aware Video RecommendationsMahajan et al. (2023): PIE: Personalized Interest Exploration for Large-Scale Recommender SystemsRaul et al. (2023): CAM2: Conformity-Aware Multi-Task Ranking Model for Large-Scale Recommender SystemsZhai et al. (2024): Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative RecommendationsSaket et al. (2023): Formulating Video Watch Success Signals for Recommendations on Short Video PlatformsWang et al. (2022): Surrogate for Long-Term User Experience in Recommender SystemsSu et al. (2024): Long-Term Value of Exploration: Measurements, Findings and AlgorithmsGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 21m

  4. 08/16/2024

    #23: Generative Models for Recommender Systems with Yashar Deldjoo

    In episode 23 of Recsperts, we welcome Yashar Deldjoo, Assistant Professor at the Polytechnic University of Bari, Italy. Yashar's research on recommender systems includes multimodal approaches, multimedia recommender systems as well as trustworthiness and adversarial robustness, where he has published a lot of work. We discuss the evolution of generative models for recommender systems, modeling paradigms, scenarios as well as their evaluation, risks and harms. We begin our interview with a reflection of Yashar's areas of recommender systems research so far. Starting with multimedia recsys, particularly video recommendations, Yashar covers his work around adversarial robustness and trustworthiness leading to the main topic for this episode: generative models for recommender systems. We learn about their aspects for improving beyond the (partially saturated) state of traditional recommender systems: improve effectiveness and efficiency for top-n recommendations, introduce interactivity beyond classical conversational recsys, provide personalized zero- or few-shot recommendations.We learn about the modeling paradigms and as well about the scenarios for generative models which mainly differ by input and modelling approach: ID-based, text-based, and multimodal generative models. This is how we navigate the large field of acronyms leading us from VAEs and GANs to LLMs. Towards the end of the episode, we also touch on the evaluation, opportunities, risks and harms of generative models for recommender systems. Yashar also provides us with an ample amount of references and upcoming events where people get the chance to know more about GenRecSys. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (03:58) - About Yashar Deldjoo (09:34) - Motivation for RecSys (13:05) - Intro to Generative Models for Recommender Systems (44:27) - Modeling Paradigms for Generative Models (51:33) - Scenario 1: Interaction-Driven Recommendation (57:59) - Scenario 2: Text-based Recommendation (01:10:39) - Scenario 3: Multimodal Recommendation (01:24:59) - Evaluation of Impact and Harm (01:38:07) - Further Research Challenges (01:45:03) - References and Research Advice (01:49:39) - Closing Remarks Links from the Episode:Yashar Deldjoo on LinkedInYashar's WebsiteKDD 2024 Tutorial: Modern Recommender Systems Leveraging Generative AI: Fundamentals, Challenges and OpportunitiesRecSys 2024 Workshop: The 1st Workshop on Risks, Opportunities, and Evaluation of Generative Models in Recommender Systems (ROEGEN@RECSYS'24)Papers: Deldjoo et al. (2024): A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)Deldjoo et al. (2020): Recommender Systems Leveraging Multimedia ContentDeldjoo et al. (2021): A Survey on Adversarial Recommender Systems: From Attack/Defense Strategies to Generative Adversarial NetworksDeldjoo et al. (2020): How Dataset Characteristics Affect the Robustness of Collaborative Recommendation ModelsLiang et al. (2018): Variational Autoencoders for Collaborative FilteringHe et al. (2016): Visual Bayesian Personalized Ranking from Implicit FeedbackGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 55m

  5. 06/06/2024

    #22: Pinterest Homefeed and Ads Ranking with Prabhat Agarwal and Aayush Mudgal

    In episode 22 of Recsperts, we welcome Prabhat Agarwal, Senior ML Engineer, and Aayush Mudgal, Staff ML Engineer, both from Pinterest, to the show. Prabhat works on recommendations and search systems at Pinterest, leading representation learning efforts. Aayush is responsible for ads ranking and privacy-aware conversion modeling. We discuss user and content modeling, short- vs. long-term objectives, evaluation as well as multi-task learning and touch on counterfactual evaluation as well. In our interview, Prabhat guides us through the journey of continuous improvements of Pinterest's Homefeed personalization starting with techniques such as gradient boosting over two-tower models to DCN and transformers. We discuss how to capture users' short- and long-term preferences through multiple embeddings and the role of candidate generators for content diversification. Prabhat shares some details about position debiasing and the challenges to facilitate exploration.With Aayush we get the chance to dive into the specifics of ads ranking at Pinterest and he helps us to better understand how multifaceted ads can be. We learn more about the pain of having too many models and the Pinterest's efforts to consolidate the model landscape to improve infrastructural costs, maintainability, and efficiency. Aayush also shares some insights about exploration and corresponding randomization in the context of ads and how user behavior is very different between different kinds of ads.Both guests highlight the role of counterfactual evaluation and its impact for faster experimentation. Towards the end of the episode, we also touch a bit on learnings from last year's RecSys challenge. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (03:51) - Guest Introductions (09:57) - Pinterest Introduction (21:57) - Homefeed Personalization (47:27) - Ads Ranking (01:14:58) - RecSys Challenge 2023 (01:20:26) - Closing Remarks Links from the Episode:Prabhat Agarwal on LinkedInAayush Mudgal on LinkedInRecSys Challenge 2023Pinterest Engineering BlogPinterest LabsPrabhat's Talk at GTC 2022: Evolution of web-scale engagement modeling at PinterestBlogpost: How we use AutoML, Multi-task learning and Multi-tower models for Pinterest AdsBlogpost: Pinterest Home Feed Unified Lightweight Scoring: A Two-tower ApproachBlogpost: Experiment without the wait: Speeding up the iteration cycle with Offline Replay ExperimentationBlogpost: MLEnv: Standardizing ML at Pinterest Under One ML Engine to Accelerate InnovationBlogpost: Handling Online-Offline Discrepancy in Pinterest Ads Ranking SystemPapers: Eksombatchai et al. (2018): Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-TimeYing et al. (2018): Graph Convolutional Neural Networks for Web-Scale Recommender SystemsPal et al. (2020): PinnerSage: Multi-Modal User Embedding Framework for Recommendations at PinterestPancha et al. (2022): PinnerFormer: Sequence Modeling for User Representation at PinterestZhao et al. (2019): Recommending what video to watch next: a multitask ranking systemGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 24m

  6. 04/08/2024

    #21: User-Centric Evaluation and Interactive Recommender Systems with Martijn Willemsen

    In episode 21 of Recsperts, we welcome Martijn Willemsen, Associate Professor at the Jheronimus Academy of Data Science and Eindhoven University of Technology. Martijn's researches on interactive recommender systems which includes aspects of decision psychology and user-centric evaluation. We discuss how users gain control over recommendations, how to support their goals and needs as well as how the user-centric evaluation framework fits into all of this. In our interview, Martijn outlines the reasons for providing users control over recommendations and how to holistically evaluate the satisfaction and usefulness of recommendations for users goals and needs. We discuss the psychology of decision making with respect to how well or not recommender systems support it. We also dive into music recommender systems and discuss how nudging users to explore new genres can work as well as how longitudinal studies in recommender systems research can advance insights. Towards the end of the episode, Martijn and I also discuss some examples and the usefulness of enabling users to provide negative explicit feedback to the system. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (03:03) - About Martijn Willemsen (15:14) - Waves of User-Centric Evaluation in RecSys (19:35) - Behaviorism is not Enough (46:21) - User-Centric Evaluation Framework (01:05:38) - Genre Exploration and Longitudinal Studies in Music RecSys (01:20:59) - User Control and Negative Explicit Feedback (01:31:50) - Closing Remarks Links from the Episode:Martijn Willemsen on LinkedInMartijn Willemsen's WebsiteUser-centric Evaluation FrameworkBehaviorism is not Enough (Talk at RecSys 2016)Neil Hunt: Quantifying the Value of Better Recommendations (Keynote at RecSys 2014)What recommender systems can learn from decision psychology about preference elicitation and behavioral change (Talk at Boise State (Idaho) and Grouplens at University of Minnesota)Eric J. Johnson: The Elements of ChoiceRasch ModelSpotify Web APIPapers: Ekstrand et al. (2016): Behaviorism is not Enough: Better Recommendations Through Listening to UsersKnijenburg et al. (2012): Explaining the user experience of recommender systemsEkstrand et al. (2014): User perception of differences in recommender algorithmsLiang et al. (2022): Exploring the longitudinal effects of nudging on users’ music genre exploration behavior and listening preferencesMcNee et al. (2006): Being accurate is not enough: how accuracy metrics have hurt recommender systemsGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 36m

  7. 11/16/2023

    #20: Practical Bandits and Travel Recommendations with Bram van den Akker

    In episode 20 of Recsperts, we welcome Bram van den Akker, Senior Machine Learning Scientist at Booking.com. Bram's work focuses on bandit algorithms and counterfactual learning. He was one of the creators of the Practical Bandits tutorial at the World Wide Web conference. We talk about the role of bandit feedback in decision making systems and in specific for recommendations in the travel industry. In our interview, Bram elaborates on bandit feedback and how it is used in practice. We discuss off-policy- and on-policy-bandits, and we learn that counterfactual evaluation is right for selecting the best model candidates for downstream A/B-testing, but not a replacement. We hear more about the practical challenges of bandit feedback, for example the difference between model scores and propensities, the role of stochasticity or the nitty-gritty details of reward signals. Bram also shares with us the challenges of recommendations in the travel domain, where he points out the sparsity of signals or the feedback delay. At the end of the episode, we can both agree on a good example for a clickbait-heavy news service in our phones. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (02:58) - About Bram van den Akker (09:16) - Motivation for Practical Bandits Tutorial (16:53) - Specifics and Challenges of Travel Recommendations (26:19) - Role of Bandit Feedback in Practice (49:13) - Motivation for Bandit Feedback (01:00:54) - Practical Start for Counterfactual Evaluation (01:06:33) - Role of Business Rules (01:11:26) - better cut this section coherently (01:17:48) - Rewards and More (01:32:45) - Closing Remarks Links from the Episode:Bram van den Akker on LinkedInPractical Bandits: An Industry Perspective (Website)Practical Bandits: An Industry Perspective (Recording)Tutorial at The Web Conference 2020: Unbiased Learning to Rank: Counterfactual and Online ApproachesTutorial at RecSys 2021: Counterfactual Learning and Evaluation for Recommender Systems: Foundations, Implementations, and Recent AdvancesGitHub: Open Bandit PipelinePapers: van den Akker et al. (2023): Practical Bandits: An Industry Perspectivevan den Akker et al. (2022): Extending Open Bandit Pipeline to Simulate Industry Challengesvan den Akker et al. (2019): ViTOR: Learning to Rank Webpages Based on Visual FeaturesGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts Website

    1h 45m

  8. 10/12/2023

    #19: Popularity Bias in Recommender Systems with Himan Abdollahpouri

    In episode 19 of Recsperts, we welcome Himan Abdollahpouri who is an Applied Research Scientist for Personalization & Machine Learning at Spotify. We discuss the role of popularity bias in recommender systems which was the dissertation topic of Himan. We talk about multi-objective and multi-stakeholder recommender systems as well as the challenges of music and podcast streaming personalization at Spotify. In our interview, Himan walks us through popularity bias as the main cause of unfair recommendations for multiple stakeholders. We discuss the consumer- and provider-side implications and how to evaluate popularity bias. Not the sheer existence of popularity bias is the major problem, but its propagation in various collaborative filtering algorithms. But we also learn how to counteract by debiasing the data, the model itself, or it's output. We also hear more about the relationship between multi-objective and multi-stakeholder recommender systems. At the end of the episode, Himan also shares the influence of popularity bias in music and podcast streaming at Spotify as well as how calibration helps to better cater content to users' preferences. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Don't forget to follow the podcast and please leave a review (00:00) - Introduction (04:43) - About Himan Abdollahpouri (15:23) - What is Popularity Bias and why is it important? (25:05) - Effect of Popularity Bias in Collaborative Filtering (30:30) - Individual Sensitivity towards Popularity (36:25) - Introduction to Bias Mitigation (53:16) - Content for Bias Mitigation (56:53) - Evaluating Popularity Bias (01:05:01) - Popularity Bias in Music and Podcast Streaming (01:08:04) - Multi-Objective Recommender Systems (01:16:13) - Multi-Stakeholder Recommender Systems (01:18:38) - Recommendation Challenges at Spotify (01:35:16) - Closing Remarks Links from the Episode:Himan Abdollahpouri on LinkedInHiman Abdollahpouri on XHiman's WebsiteHiman's PhD Thesis on "Popularity Bias in Recommendation: A Multi-stakeholder Perspective"2nd Workshop on Multi-Objective Recommender Systems (MORS @ RecSys 2022)Papers: Su et al. (2009): A Survey on Collaborative Filtering TechniquesMehrotra et al. (2018): Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommender SystemsAbdollahpouri et al. (2021): User-centered Evaluation of Popularity Bias in Recommender SystemsAbdollahpouri et al. (2019): The Unfairness of Popularity Bias in RecommendationAbdollahpouri et al. (2017): Controlling Popularity Bias in Learning-to-Rank RecommendationWasilewsi et al. (2016): Incorporating Diversity in a Learning to Rank Recommender SystemOh et al. (2011): Novel Recommendation Based on Personal Popularity TendencySteck (2018): Calibrated RecommendationsAbdollahpouri et al. (2023): Calibrated Recommendations as a Minimum-Cost Flow ProblemSeymen et al. (2022): Making smart recommendations for perishable and stockout productsGeneral Links: Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel@recsperts.comRecsperts Website

    1h 42m
See All (27)

Ratings & Reviews

5
out of 5
3 Ratings

About

Recommender Systems are the most challenging, powerful and ubiquitous area of machine learning and artificial intelligence. This podcast hosts the experts in recommender systems research and application. From understanding what users really want to driving large-scale content discovery - from delivering personalized online experiences to catering to multi-stakeholder goals. Guests from industry and academia share how they tackle these and many more challenges. With Recsperts coming from universities all around the globe or from various industries like streaming, ecommerce, news, or social media, this podcast provides depth and insights. We go far beyond your 101 on RecSys and the shallowness of another matrix factorization based rating prediction blogpost! The motto is: be relevant or become irrelevant! Expect a brand-new interview each month and follow Recsperts on your favorite podcast player.

Information

To listen to explicit episodes, sign in.

Stay up to date with this show

Sign in or sign up to follow shows, save episodes, and get the latest updates.
Select a country or region

Africa, Middle East, and India

Asia Pacific

Europe

Latin America and the Caribbean

The United States and Canada