96 épisodes

Welcome! We at MLST are inspired by scientists and each week we have a hard-hitting discussion with the leading thinkers in the AI space. Street Talk is ridiculously technical and we believe strongly in diversity of thought in AI, covering all the main ideas in the field, avoiding hype where possible.

MLST is run by Dr. Tim Scarfe and Dr. Keith Duggar, and with regular appearances from Dr. Yannic Kilcher.

Machine Learning Street Talk (MLST‪)‬ Machine Learning Street Talk

    • Technologies

Welcome! We at MLST are inspired by scientists and each week we have a hard-hitting discussion with the leading thinkers in the AI space. Street Talk is ridiculously technical and we believe strongly in diversity of thought in AI, covering all the main ideas in the field, avoiding hype where possible.

MLST is run by Dr. Tim Scarfe and Dr. Keith Duggar, and with regular appearances from Dr. Yannic Kilcher.

    #96 Prof. PEDRO DOMINGOS - There are no infinities, utility functions, neurosymbolic

    #96 Prof. PEDRO DOMINGOS - There are no infinities, utility functions, neurosymbolic

    Pedro Domingos, Professor Emeritus of Computer Science and Engineering at the University of Washington, is renowned for his research in machine learning, particularly for his work on Markov logic networks that allow for uncertain inference. He is also the author of the acclaimed book "The Master Algorithm".



    Panel: Dr. Tim Scarfe



    TOC:

    [00:00:00] Introduction

    [00:01:34] Galaxtica / misinformation / gatekeeping

    [00:12:31] Is there a master algorithm?

    [00:16:29] Limits of our understanding 

    [00:21:57] Intentionality, Agency, Creativity

    [00:27:56] Compositionality 

    [00:29:30] Digital Physics / It from bit / Wolfram 

    [00:35:17] Alignment / Utility functions

    [00:43:36] Meritocracy  

    [00:45:53] Game theory 

    [01:00:00] EA/consequentialism/Utility

    [01:11:09] Emergence / relationalism 

    [01:19:26] Markov logic 

    [01:25:38] Moving away from anthropocentrism 

    [01:28:57] Neurosymbolic / infinity / tensor algerbra

    [01:53:45] Abstraction

    [01:57:26] Symmetries / Geometric DL

    [02:02:46] Bias variance trade off 

    [02:05:49] What seen at neurips

    [02:12:58] Chalmers talk on LLMs

    [02:28:32] Definition of intelligence

    [02:32:40] LLMs 

    [02:35:14] On experts in different fields

    [02:40:15] Back to intelligence

    [02:41:37] Spline theory / extrapolation



    YT version:  https://www.youtube.com/watch?v=C9BH3F2c0vQ



    References;



    The Master Algorithm [Domingos]

    https://www.amazon.co.uk/s?k=master+algorithm&i=stripbooks&crid=3CJ67DCY96DE8&sprefix=master+algorith%2Cstripbooks%2C82&ref=nb_sb_noss_2



    INFORMATION, PHYSICS, QUANTUM: THE SEARCH FOR LINKS [John Wheeler/It from Bit]

    https://philpapers.org/archive/WHEIPQ.pdf



    A New Kind Of Science [Wolfram]

    https://www.amazon.co.uk/New-Kind-Science-Stephen-Wolfram/dp/1579550088



    The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tom Chivers]

    https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795



    The Status Game: On Social Position and How We Use It [Will Storr]

    https://www.goodreads.com/book/show/60598238-the-status-game



    Newcomb's paradox

    https://en.wikipedia.org/wiki/Newcomb%27s_paradox



    The Case for Strong Emergence [Sabine Hossenfelder]

    https://philpapers.org/rec/HOSTCF-3



    Markov Logic: An Interface Layer for Artificial Intelligence [Domingos]

    https://www.morganclaypool.com/doi/abs/10.2200/S00206ED1V01Y200907AIM007



    Note; Pedro discussed “Tensor Logic” - I was not able to find a reference



    Neural Networks and the Chomsky Hierarchy [Grégoire Delétang/DeepMind]

    https://arxiv.org/abs/2207.02098



    Connectionism and Cognitive Architecture: A Critical Analysis [Jerry A. Fodor and Zenon W. Pylyshyn]

    https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf



    Every Model Learned by Gradient Descent Is Approximately a Kernel Machine [Pedro Domingos]

    https://arxiv.org/abs/2012.00152



    A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27 [LeCun]

    https://openreview.net/pdf?id=BZ5a1r-kVsf



    Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković]

    https://arxiv.org/abs/2104.13478



    The Algebraic Mind: Integrating Connectionism and Cognitive Science [Gary Marcus]

    https://www.amazon.co.uk/Algebraic-Mind-Integrating-Connectionism-D

    • 2 h 49 min
    #95 - Prof. IRINA RISH - AGI, Complex Systems, Transhumanism

    #95 - Prof. IRINA RISH - AGI, Complex Systems, Transhumanism

    Canadian Excellence Research Chair in Autonomous AI. Irina holds an MSc and PhD in AI from the University of California, Irvine as well as an MSc in Applied Mathematics from the Moscow Gubkin Institute. Her research focuses on machine learning, neural data analysis, and neuroscience-inspired AI. In particular, she is exploring continual lifelong learning, optimization algorithms for deep neural networks, sparse modelling and probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis. Prof. Rish holds 64 patents, has published over 80 research papers, several book chapters, three edited books, and a monograph on Sparse Modelling. She has served as a Senior Area Chair for NeurIPS and ICML.   Irina's research is focussed on taking us closer to the holy grail of Artificial General Intelligence.  She continues to push the boundaries of machine learning, continually striving to make advancements in neuroscience-inspired AI.

    In a conversation about artificial intelligence (AI), Irina and Tim discussed the idea of transhumanism and the potential for AI to improve human flourishing. Irina suggested that instead of looking at AI as something to be controlled and regulated, people should view it as a tool to augment human capabilities. She argued that attempting to create an AI that is smarter than humans is not the best approach, and that a hybrid of human and AI intelligence is much more beneficial. As an example, she mentioned how technology can be used as an extension of the human mind, to track mental states and improve self-understanding. Ultimately, Irina concluded that transhumanism is about having a symbiotic relationship with technology, which can have a positive effect on both parties.

    Tim then discussed the contrasting types of intelligence and how this could lead to something interesting emerging from the combination. He brought up the Trolley Problem and how difficult moral quandaries could be programmed into an AI. Irina then referenced The Garden of Forking Paths, a story which explores the idea of how different paths in life can be taken and how decisions from the past can have an effect on the present.

    To better understand AI and intelligence, Irina suggested looking at it from multiple perspectives and understanding the importance of complex systems science in programming and understanding dynamical systems. She discussed the work of Michael Levin, who is looking into reprogramming biological computers with chemical interventions, and Tim mentioned Alex Mordvinsev, who is looking into the self-healing and repair of these systems. Ultimately, Irina argued that the key to understanding AI and intelligence is to recognize the complexity of the systems and to create hybrid models of human and AI intelligence.

    Find Irina;

    https://mila.quebec/en/person/irina-rish/

    https://twitter.com/irinarish



    YT version: https://youtu.be/8-ilcF0R7mI 

    MLST Discord: https://discord.gg/aNPkGUQtc5



    References;

    The Garden of Forking Paths: Jorge Luis Borges [Jorge Luis Borges]

    https://www.amazon.co.uk/Garden-Forking-Paths-Penguin-Modern/dp/0241339057

    The Brain from Inside Out [György Buzsáki]

    https://www.amazon.co.uk/Brain-Inside-Out-Gy%C3%B6rgy-Buzs%C3%A1ki/dp/0190905387

    Growing Isotropic Neural Cellular Automata [Alexander Mordvintsev]

    https://arxiv.org/abs/2205.01681

    The Extended Mind [Andy Clark and David Chalmers]

    https://www.jstor.org/stable/3328150

    The Gentle Seduction [Marc Stiegler]

    https://www.amazon.co.uk/Gentle-Seduction-Marc-Stiegler/dp/0671698877

    • 39 min
    #94 - ALAN CHAN - AI Alignment and Governance #NEURIPS

    #94 - ALAN CHAN - AI Alignment and Governance #NEURIPS

    Support us! https://www.patreon.com/mlst

    Alan Chan is a PhD student at Mila, the Montreal Institute for Learning Algorithms, supervised by Nicolas Le Roux. Before joining Mila, Alan was a Masters student at the Alberta Machine Intelligence Institute and the University of Alberta, where he worked with Martha White. Alan's expertise and research interests encompass value alignment and AI governance. He is currently exploring the measurement of harms from language models and the incentives that agents have to impact the world. Alan's research focuses on understanding and controlling the values expressed by machine learning models. His projects have examined the regulation of explainability in algorithmic systems, scoring rules for performative binary prediction, the effects of global exclusion in AI development, and the role of a graduate student in approaching ethical impacts in AI research. In addition, Alan has conducted research into inverse policy evaluation for value-based sequential decision-making, and the concept of "normal accidents" and AI systems. Alan's research is motivated by the need to align AI systems with human values, and his passion for scientific and governance work in this field. Alan's energy and enthusiasm for his field is infectious. 

    This was a discussion at NeurIPS. It was in quite a loud environment so the audio quality could have been better. 

    References:



    The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tim Chivers]

    https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795



    The implausibility of intelligence explosion [Chollet]

    https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec



    Superintelligence: Paths, Dangers, Strategies [Bostrom]

    https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111



    A Theory of Universal Artificial Intelligence based on Algorithmic Complexity [Hutter]

    https://arxiv.org/abs/cs/0004001



    YT version: https://youtu.be/XBMnOsv9_pk 

    MLST Discord: https://discord.gg/aNPkGUQtc5 

    • 13 min
    #93 Prof. MURRAY SHANAHAN - Consciousness, Embodiment, Language Models

    #93 Prof. MURRAY SHANAHAN - Consciousness, Embodiment, Language Models

    Support us! https://www.patreon.com/mlst



    Professor Murray Shanahan is a renowned researcher on sophisticated cognition and its implications for artificial intelligence. His 2016 article ‘Conscious Exotica’ explores the Space of Possible Minds, a concept first proposed by philosopher Aaron Sloman in 1984, which includes all the different forms of minds from those of other animals to those of artificial intelligence. Shanahan rejects the idea of an impenetrable realm of subjective experience and argues that the majority of the space of possible minds may be occupied by non-natural variants, such as the ‘conscious exotica’ of which he speaks.  In his paper ‘Talking About Large Language Models’, Shanahan discusses the capabilities and limitations of large language models (LLMs). He argues that prompt engineering is a key element for advanced AI systems, as it involves exploiting prompt prefixes to adjust LLMs to various tasks. However, Shanahan cautions against ascribing human-like characteristics to these systems, as they are fundamentally different and lack a shared comprehension with humans. Even though LLMs can be integrated into embodied systems, it does not mean that they possess human-like language abilities. Ultimately, Shanahan concludes that although LLMs are formidable and versatile, we must be wary of over-simplifying their capacities and limitations.

    YT version: https://youtu.be/BqkWpP3uMMU

    Full references on the YT description. 



    [00:00:00] Introduction

    [00:08:51] Consciousness and  Consciousness Exotica

    [00:34:59] Slightly Consciousness LLMs

    [00:38:05] Embodiment

    [00:51:32] Symbol Grounding 

    [00:54:13] Emergence

    [00:57:09] Reasoning

    [01:03:16] Intentional Stance

    [01:07:06] Digression on Chomsky show and Andrew Lampinen

    [01:10:31] Prompt Engineering



    Find Murray online:

    https://www.doc.ic.ac.uk/~mpsha/

    https://twitter.com/mpshanahan?lang=en

    https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en



    MLST Discord: https://discord.gg/aNPkGUQtc5

    • 1h 20 min
    #92 - SARA HOOKER - Fairness, Interpretability, Language Models

    #92 - SARA HOOKER - Fairness, Interpretability, Language Models

    Support us! https://www.patreon.com/mlst

    Sara Hooker is an exceptionally talented and accomplished leader and research scientist in the field of machine learning. She is the founder of Cohere For AI, a non-profit research lab that seeks to solve complex machine learning problems. She is passionate about creating more points of entry into machine learning research and has dedicated her efforts to understanding how progress in this field can be translated into reliable and accessible machine learning in the real-world.

    Sara is also the co-founder of the Trustworthy ML Initiative, a forum and seminar series related to Trustworthy ML. She is on the advisory board of Patterns and is an active member of the MLC research group, which has a focus on making participation in machine learning research more accessible.

    Before starting Cohere For AI, Sara worked as a research scientist at Google Brain. She has written several influential research papers, including "The Hardware Lottery", "The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation", "Moving Beyond “Algorithmic Bias is a Data Problem”" and "Characterizing and Mitigating Bias in Compact Models". 

    In addition to her research work, Sara is also the founder of the local Bay Area non-profit Delta Analytics, which works with non-profits and communities all over the world to build technical capacity and empower others to use data. She regularly gives tutorials on machine learning fundamentals, interpretability, model compression and deep neural networks and is dedicated to collaborating with independent researchers around the world.

    Sara Hooker is famous for writing a paper introducing the concept of the 'hardware lottery', in which the success of a research idea is determined not by its inherent superiority, but by its compatibility with available software and hardware. She argued that choices about software and hardware have had a substantial impact in deciding the outcomes of early computer science history, and that with the increasing heterogeneity of the hardware landscape, gains from advances in computing may become increasingly disparate. Sara proposed that an interim goal should be to create better feedback mechanisms for researchers to understand how their algorithms interact with the hardware they use. She suggested that domain-specific languages, auto-tuning of algorithmic parameters, and better profiling tools may help to alleviate this issue, as well as provide researchers with more informed opinions about how hardware and software should progress. Ultimately, Sara encouraged researchers to be mindful of the implications of the hardware lottery, as it could mean that progress on some research directions is further obstructed. If you want to learn more about that paper, watch our previous interview with Sara.

    YT version: https://youtu.be/7oJui4eSCoY

    MLST Discord: https://discord.gg/aNPkGUQtc5

    TOC:

    [00:00:00] Intro

    [00:02:53] Interpretability / Fairness

    [00:35:29] LLMs



    Find Sara:

    https://www.sarahooker.me/

    https://twitter.com/sarahookr

    • 51 min
    #91 - HATTIE ZHOU - Teaching Algorithmic Reasoning via In-context Learning #NeurIPS

    #91 - HATTIE ZHOU - Teaching Algorithmic Reasoning via In-context Learning #NeurIPS

    Support us! https://www.patreon.com/mlst



    Hattie Zhou, a PhD student at Université de Montréal and Mila, has set out to understand and explain the performance of modern neural networks, believing it a key factor in building better, more trusted models. Having previously worked as a data scientist at Uber, a private equity analyst at Radar Capital, and an economic consultant at Cornerstone Research, she has recently released a paper in collaboration with the Google Brain team, titled ‘Teaching Algorithmic Reasoning via In-context Learning’. In this work, Hattie identifies and examines four key stages for successfully teaching algorithmic reasoning to large language models (LLMs): formulating algorithms as skills, teaching multiple skills simultaneously, teaching how to combine skills, and teaching how to use skills as tools. Through the application of algorithmic prompting, Hattie has achieved remarkable results, with an order of magnitude error reduction on some tasks compared to the best available baselines. This breakthrough demonstrates algorithmic prompting’s viability as an approach for teaching algorithmic reasoning to LLMs, and may have implications for other tasks requiring similar reasoning capabilities.



    TOC

    [00:00:00] Hattie Zhou

    [00:19:49] Markus Rabe [Google Brain]



    Hattie's Twitter - https://twitter.com/oh_that_hat

    Website - http://hattiezhou.com/



    Teaching Algorithmic Reasoning via In-context Learning [Hattie Zhou, Azade Nova, Hugo Larochelle, Aaron Courville, Behnam Neyshabur, and Hanie Sedghi]

    https://arxiv.org/pdf/2211.09066.pdf



    Markus Rabe [Google Brain]:

    https://twitter.com/markusnrabe

    https://research.google/people/106335/

    https://www.linkedin.com/in/markusnrabe



    Autoformalization with Large Language Models [Albert Jiang Charles Edgar Staats Christian Szegedy Markus Rabe Mateja Jamnik Wenda Li Yuhuai Tony Wu]

    https://research.google/pubs/pub51691/



    Discord: https://discord.gg/aNPkGUQtc5

    YT: https://youtu.be/80i6D2TJdQ4

    • 21 min

Classement des podcasts dans Technologies

Caroline Mignaux
Micode
France Culture
Laurent Kretz | Orso Media
Audrey Couleau et Mat alias @profduweb
Lex Fridman

D’autres se sont aussi abonnés à…

Sam Charrington
Changelog Media
NVIDIA
Andreessen Horowitz
Santa Fe Institute, Michael Garfield
Kyle Polich