A show about the science and engineering behind AutoML.
AutoGluon: The Story
Today we're talking with Nick Erickson from AutoGluon.
We discuss AutoGluon's fascinating origin story, its unique point of view, the science and engineering behind some of its unique contributions, Amazon's Machine Learning University, AutoGluon's multi-layer stack ensembler in all its detail, their feature preprocessing pipeline, their feature type inference, their adaptive approach to early stopping, controlling for inference speeds, the different multi-modal architectures, the ML culture at Amazon, the unique challenges of time series, the role of competitions, the decision to reject hyperparameter optimization, benchmarking in AutoML, what the research community can do to help industry along, AutoGluon's relationship with pre-trained tabular models like Tab-PFN, whether the rise of LLMs is likely to affect AutoGluon, what's stopping more people from adopting AutoML solutions, AutoGluon Cloud, the dream and reality of an auto-benchmarking tool, how to contribute to their project, and many, many other topics.
This was one of my favorite episodes. Nick, thank you for joining!
You can follow Nick on Twitter here: @innixma.
And you can follow AutoGluon on GitHub here: https://github.com/autogluon.
Some more resources on AutoGluon:
The original AutoGluon Paper: "AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data": https://arxiv.org/abs/2003.06505AutoML Fall School 2022 AutoGluon presentation, a good way to understand the philosophy behind AutoGluon: https://www.youtube.com/watch?v=VAAITEds-28AutoGluon multi-modal paper: https://dl.acm.org/doi/abs/10.1145/3534678.3542616
How to Integrate Logic and Argumentation into Human-Centric AutoML
Today we're talking with Joseph Giovanelli about his work on integrating logic and argumentation into AutoML systems.
Joseph is a PhD student at the University of Bologna. He was more recently in Hannover working on ethics and fairness with Marius’ team.
The paper he published presents his framework, HAMLET, which stands for Human-centric AutoML via Logic and Argumentation. It allows a user to iteratively specify constraints in a formal manner and, once defined, those constraints become logical premises. Those premises, when combined together, can produce conflicts with one another, thereby reducing the search space and providing deeper intuition back to the user.
To learn more about HAMLET, see the paper here: https://ceur-ws.org/Vol-3135/dataplat_short2.pdf and the repo here: https://github.com/QueueInc/HAMLET
To follow Joseph on LinkedIn, see his profile here: https://www.linkedin.com/in/joseph-giovanelli/
How to Design an AutoML System using Error Decomposition
Today we're talking with Caitlin Owen, a post-doc at the University of Otago about her work on error decomposition.
She recently published a paper titled "Towards Explainable AutoML Using Error Decomposition" about how a more granular view of the components of error can lead the construction of better AutoML systems.
Read her paper here: https://link.springer.com/chapter/10.1007/978-3-031-22695-3_13
Follow her on Twitter here: @CaitAshfordOwen
Connect with her on LinkedIn here: https://www.linkedin.com/in/caitlin-owen-5b9b08193/
The Semantic Layer and AutoML
Today we're talking with Gaurav Rao, the EVP & GM of Machine Learning and AI at AtScale, a company centered around the semantic layer.
For some time now, I've been feeling that there is a deep connection between a formal articulation of business context and the realization of the dream of AutoML, so I searched for people in the space who can help shine light on this direction.
Gaurav is one of the few who can speak about this. As you'll hear, he's extremely pedagogic and he's walking us through the origins of the concept, how it addresses some of the challenges that businesses face when trying to operationalize their ML, what it takes to build a universal semantic layer, how downstream ML applications are affected by the presence or absence of a semantic layer, and how the space of AutoML factors into this.
Connect his Gaurav and learn more about the semantic layer through his LinkedIn: https://www.linkedin.com/in/gauravraotechenthusiast/
Foundation Models: The term and its origins
Today Ankush Garg is speaking with Rishi Bommasani, PhD student at Stanford and one of the originator of the term Foundation Models.
They’re talking about the origins of the term Foundation Model, which he and his group advanced, in the paper "On the Opportunities and Risks of Foundation Models".
They’ll talk about self-supervision, issues of scale, the motivation behind the terminology, the origins of the Research for Foundation Models Institute at Stanford, outcome homogenization, emergence and phase transitions, and some of the social consequences to look out for.
Thank you both for this conversation. As the world is coming to terms with GPT-4, this will be increasingly relevant.
Paper: On the Opportunities and Risks of Foundation Models.
Rishi's twitter: @RishiBommasani
Center for Research on Foundation Models (CRFM)
The Business and Engineering of AutoML Products with Raymond Peck
Today we're talking with Raymond Peck, a senior engineer and director in the AutoML space. He spent time at H2O, dotData, Alteryx and many other places.
This is a fascinating conversation about the business, engineering, and science of machine learning automation in production. Learning about his experience is crucial for understanding the biography of the space.
We discuss the early motivations behind AutoML, the initial value propositions that propelled the first movers in the market, the market dynamics that operated in the early days, the evolution of the relevant engineering and science, how customers evaluate AutoML tools, the role of feature engineering and relational tables, the crucial role that explainability plays in AutoML, and many more topics.
Raymond is a prolific writer on LinkedIn. You should follow him here: https://www.linkedin.com/in/raymondpeck/.
Excellent niche technical content
Host does a good job of researching and getting people to open up