83 episodes

Welcome! We at MLST are inspired by scientists and each week we have a hard-hitting discussion with the leading thinkers in the AI space. Street Talk is ridiculously technical and we believe strongly in diversity of thought in AI, covering all the main ideas in the field, avoiding hype where possible.

MLST is run by Dr. Tim Scarfe and Dr. Keith Duggar, and with regular appearances from Dr. Yannic Kilcher.

Machine Learning Street Talk (MLST‪)‬ Machine Learning Street Talk

    • Technology
    • 4.7 • 33 Ratings

Welcome! We at MLST are inspired by scientists and each week we have a hard-hitting discussion with the leading thinkers in the AI space. Street Talk is ridiculously technical and we believe strongly in diversity of thought in AI, covering all the main ideas in the field, avoiding hype where possible.

MLST is run by Dr. Tim Scarfe and Dr. Keith Duggar, and with regular appearances from Dr. Yannic Kilcher.

    #82 - Dr. JOSCHA BACH - Digital Physics, DL and Consciousness [UNPLUGGED]

    #82 - Dr. JOSCHA BACH - Digital Physics, DL and Consciousness [UNPLUGGED]

    AI Helps Ukraine - Charity Conference

    A charity conference on AI to raise funds for medical and humanitarian aid for Ukraine

    https://aihelpsukraine.cc/



    YT version: https://youtu.be/LgwjcqhkOA4



    Support us!

    https://www.patreon.com/mlst 



    Dr. Joscha Bach (born 1973 in Weimar, Germany) is a German artificial intelligence researcher and cognitive scientist focusing on cognitive architectures, mental representation, emotion, social modelling, and multi-agent systems. 

    http://bach.ai/

    https://twitter.com/plinz



    TOC:

    [00:00:00] Ukraine Charity Conference and NeurIPS 2022

    [00:03:40] Theory of computation, Godel, Penrose

    [00:11:44] Modelling physical reality

    [00:15:19] Is our universe infinite?

    [00:24:30] Large language models, and on DL / is Gary Marcus hitting a wall?

    [00:45:17] Generative models / Codex / Language of thought

    [00:58:46] Consciousness (with Friston references)



    References:



    Am I Self-Conscious? (Or Does Self-Organization Entail Self-Consciousness?) [Friston]

    https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full



    Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi]

    https://arxiv.org/abs/2202.07206



    Deep Learning Is Hitting a Wall [Gary Marcus]

    https://nautil.us/deep-learning-is-hitting-a-wall-238440/



    Turing machines

    https://en.wikipedia.org/wiki/Turing_machine

    Lambda Calculus

    https://en.wikipedia.org/wiki/Lambda_calculus

    Godel's incompletness theorem

    https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems

    Oracle machine

    https://en.wikipedia.org/wiki/Oracle_machine

    • 1 hr 15 min
    #81 JULIAN TOGELIUS, Prof. KEN STANLEY - AGI, Games, Diversity & Creativity [UNPLUGGED]

    #81 JULIAN TOGELIUS, Prof. KEN STANLEY - AGI, Games, Diversity & Creativity [UNPLUGGED]

    Support us (and please rate on podcast app)

    https://www.patreon.com/mlst 



    In this show tonight with Prof. Julian Togelius (NYU) and Prof. Ken Stanley we discuss open-endedness, AGI, game AI and reinforcement learning.  



    [Prof Julian Togelius]

    https://engineering.nyu.edu/faculty/julian-togelius

    https://twitter.com/togelius



    [Prof Ken Stanley]

    https://www.cs.ucf.edu/~kstanley/

    https://twitter.com/kenneth0stanley



    TOC:

    [00:00:00] Introduction

    [00:01:07] AI and computer games

    [00:12:23] Intelligence

    [00:21:27] Intelligence Explosion

    [00:25:37] What should we be aspiring towards?

    [00:29:14] Should AI contribute to culture?

    [00:32:12] On creativity and open-endedness

    [00:36:11] RL overfitting

    [00:44:02] Diversity preservation

    [00:51:18] Empiricism vs rationalism , in gradient descent the data pushes you around

    [00:55:49] Creativity and interestingness (does complexity / information increase)

    [01:03:20] What does a population give us?

    [01:05:58] Emergence / generalisation snobbery



    References;

    [Hutter/Legg] Universal Intelligence: A Definition of Machine Intelligence

    https://arxiv.org/abs/0712.3329



    https://en.wikipedia.org/wiki/Artificial_general_intelligence

    https://en.wikipedia.org/wiki/I._J._Good

    https://en.wikipedia.org/wiki/G%C3%B6del_machine



    [Chollet] Impossibility of intelligence explosion

    https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec



    [Alex Irpan] - RL is hard

    https://www.alexirpan.com/2018/02/14/rl-hard.html

    https://nethackchallenge.com/

    Map elites

    https://arxiv.org/abs/1504.04909



    Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space

    https://arxiv.org/abs/1912.02400



    [Stanley] - Why greatness cannot be planned

    https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237



    [Lehman/Stanley] Abandoning Objectives: Evolution through the Search for Novelty Alone

    https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf

    • 1 hr 9 min
    #80 AIDAN GOMEZ [CEO Cohere] - Language as Software

    #80 AIDAN GOMEZ [CEO Cohere] - Language as Software

    We had a conversation with Aidan Gomez, the CEO of language-based AI platform Cohere. Cohere is a startup which uses artificial intelligence to help users build the next generation of language-based applications. It's headquartered in Toronto. The company has raised $175 million in funding so far.

    Language may well become a key new substrate for software building, both in its representation and how we build the software. It may democratise software building so that more people can build software, and we can build new types of software. Aidan and I discuss this in detail in this episode of MLST.

    Check out Cohere -- https://dashboard.cohere.ai/welcome/register?utm_source=influencer&utm_medium=social&utm_campaign=mlst

    Support us!

    https://www.patreon.com/mlst 

    YT version: https://youtu.be/ooBt_di8DLs

    TOC:

    [00:00:00] Aidan Gomez intro

    [00:02:12] What's it like being a CEO?

    [00:02:52] Transformers

    [00:09:33] Deepmind Chomsky Hierarchy

    [00:14:58] Cohere roadmap

    [00:18:18] Friction using LLMs for startups

    [00:25:31] How different from OpenAI / GPT-3

    [00:29:31] Engineering questions on Cohere

    [00:35:13] Francois Chollet says that LLMs are like databases

    [00:38:34] Next frontier of language models

    [00:42:04] Different modes of understanding in LLMs

    [00:47:04] LLMs are the new extended mind

    [00:50:03] Is language the next interface, and why might that be bad?

    References:

    [Balestriero] Spine theory of NNs

    https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf

    [Delétang et al] Neural Networks and the Chomsky Hierarchy

    https://arxiv.org/abs/2207.02098

    [Fodor, Pylyshyn] Connectionism and Cognitive Architecture: A Critical Analysis

    https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf

    [Chalmers, Clark] The extended mind

    https://icds.uoregon.edu/wp-content/uploads/2014/06/Clark-and-Chalmers-The-Extended-Mind.pdf

    [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models

    https://arxiv.org/abs/2210.13966

    [Jay Alammar]

    Illustrated stable diffusion

    https://jalammar.github.io/illustrated-stable-diffusion/

    Illustrated transformer

    https://jalammar.github.io/illustrated-transformer/

    https://www.youtube.com/channel/UCmOwsoHty5PrmE-3QhUBfPQ

    [Sandra Kublik] (works at Cohere!)

    https://www.youtube.com/channel/UCjG6QzmabZrBEeGh3vi-wDQ

    • 51 min
    #79 Consciousness and the Chinese Room [Special Edition] (CHOLLET, BISHOP, CHALMERS, BACH)

    #79 Consciousness and the Chinese Room [Special Edition] (CHOLLET, BISHOP, CHALMERS, BACH)

    This video is demonetised on music copyright so we would appreciate support on our Patreon! https://www.patreon.com/mlst 

    We would also appreciate it if you rated us on your podcast platform. 

    YT: https://youtu.be/_KVAzAzO5HU

    Panel: Dr. Tim Scarfe, Dr. Keith Duggar

    Guests: Prof. J. Mark Bishop, Francois Chollet, Prof. David Chalmers, Dr. Joscha Bach, Prof. Karl Friston, Alexander Mattick, Sam Roffey

    The Chinese Room Argument was first proposed by philosopher John Searle in 1980. It is an argument against the possibility of artificial intelligence (AI) – that is, the idea that a machine could ever be truly intelligent, as opposed to just imitating intelligence.

    The argument goes like this:

    Imagine a room in which a person sits at a desk, with a book of rules in front of them. This person does not understand Chinese.

    Someone outside the room passes a piece of paper through a slot in the door. On this paper is a Chinese character. The person in the room consults the book of rules and, following these rules, writes down another Chinese character and passes it back out through the slot.

    To someone outside the room, it appears that the person in the room is engaging in a conversation in Chinese. In reality, they have no idea what they are doing – they are just following the rules in the book.

    The Chinese Room Argument is an argument against the idea that a machine could ever be truly intelligent. It is based on the idea that intelligence requires understanding, and that following rules is not the same as understanding.

    in this detailed investigation into the Chinese Room, Consciousness and Syntax vs Semantics, we interview luminaries J.Mark Bishop and Francois Chollet and use unreleased footage from our interviews with David Chalmers, Joscha Bach and Karl Friston. We also cover material from Walid Saba and interview Alex Mattick from Yannic's Discord. 

    This is probably my favourite ever episode of MLST. I hope you enjoy it!  With Keith Duggar. 

    Note that we are using clips from our unreleased interviews from David Chalmers and Joscha Bach -- we will release those shows properly in the coming weeks. We apologise for delay releasing our backlog, we have been busy building a startup company in the background.



    TOC: 

    [00:00:00] Kick off

    [00:00:46] Searle

    [00:05:09] Bishop introduces CRA

    [00:00:00] Stevan Hardad take on CRA 

    [00:14:03] Francois Chollet dissects CRA

    [00:34:16] Chalmers on consciousness

    [00:36:27] Joscha Bach on consciousness

    [00:42:01] Bishop introduction

    [00:51:51] Karl Friston on consciousness

    [00:55:19] Bishop on consciousness and comments on Chalmers 

    [01:21:37] Private language games (including clip with Sam Roffey)

    [01:27:27] Dr. Walid Saba on the chinese room (gofai/systematicity take)

    [00:34:36] Bishop: on agency / teleology

    [01:36:38] Bishop: back to CRA

    [01:40:53] Noam Chomsky on mysteries 

    [01:45:56] Eric Curiel on math does not represent

    [01:48:14] Alexander Mattick on syntax vs semantics



    Thanks to: Mark MC on Discord for stimulating conversation, Alexander Mattick, Dr. Keith Duggar, Sam Roffey. Sam's YouTube channel is https://www.youtube.com/channel/UCjRNMsglFYFwNsnOWIOgt1Q

    • 2 hr 9 min
    MLST #78 - Prof. NOAM CHOMSKY (Special Edition)

    MLST #78 - Prof. NOAM CHOMSKY (Special Edition)

    Patreon: https://www.patreon.com/mlst

    Discord: https://discord.gg/ESrGqhf5CB

    In this special edition episode, we have a conversation with Prof. Noam Chomsky, the father of modern linguistics and the most important intellectual of the 20th century. 

    With a career spanning the better part of a century, we took the chance to ask Prof. Chomsky his thoughts not only on the progress of linguistics and cognitive science but also the deepest enduring mysteries of science and philosophy as a whole - exploring what may lie beyond our limits of understanding. We also discuss the rise of connectionism and large language models, our quest to discover an intelligible world, and the boundaries between silicon and biology.

    We explore some of the profound misunderstandings of linguistics in general and Chomsky’s own work specifically which have persisted, at the highest levels of academia for over sixty years.  

    We have produced a significant introduction section where we discuss in detail Yann LeCun’s recent position paper on AGI, a recent paper on emergence in LLMs, empiricism related to cognitive science, cognitive templates, “the ghost in the machine” and language. 



    Panel: 

    Dr. Tim Scarfe

    Dr. Keith Duggar

    Dr. Walid Saba 



    YT version: https://youtu.be/-9I4SgkHpcA



    00:00:00 Kick off

    00:02:24 C1: LeCun's recent position paper on AI, JEPA, Schmidhuber, EBMs

    00:48:38 C2: Emergent abilities in LLMs paper

    00:51:32 C3: Empiricism

    01:25:33 C4: Cognitive Templates

    01:35:47 C5: The Ghost in the Machine

    01:59:21 C6: Connectionism and Cognitive Architecture: A Critical Analysis by Fodor and Pylyshyn

    02:19:25 C7: We deep-faked Chomsky

    02:29:11 C8: Language

    02:34:41 C9: Chomsky interview kick-off!

    02:35:39 Large Language Models such as GPT-3

    02:39:14 Connectionism and radical empiricism

    02:44:44 Hybrid systems such as neurosymbolic

    02:48:47 Computationalism silicon vs biological

    02:53:28 Limits of human understanding

    03:00:46 Semantics state-of-the-art

    03:06:43 Universal grammar, I-Language, and language of thought

    03:16:27 Profound and enduring misunderstandings

    03:25:41 Greatest remaining mysteries science and philosophy

    03:33:10 Debrief and 'Chuckles' from Chomsky

    • 3 hr 37 min
    #77 - Vitaliy Chiley (Cerebras)

    #77 - Vitaliy Chiley (Cerebras)

    Vitaliy Chiley  is a Machine Learning Research Engineer at the next-generation computing hardware company Cerebras Systems. We spoke about how DL workloads including sparse workloads can run faster on Cerebras hardware.



    [00:00:00] Housekeeping

    [00:01:08] Preamble

    [00:01:50] Vitaliy Chiley Introduction

    [00:03:11] Cerebrus architecture

    [00:08:12] Memory management and FLOP utilisation

    [00:18:01] Centralised vs decentralised compute architecture

    [00:21:12] Sparsity

    [00:23:47] Does Sparse NN imply Heterogeneous compute?

    [00:29:21] Cost of distributed memory stores?

    [00:31:01] Activation vs weight sparsity

    [00:37:52] What constitutes a dead weight to be pruned?

    [00:39:02] Is it still a saving if we have to choose between weight and activation sparsity?

    [00:41:02] Cerebras is a cool place to work

    [00:44:05] What is sparsity? Why do we need to start dense? 

    [00:46:36] Evolutionary algorithms on Cerebras?

    [00:47:57] How can we start sparse? Google RIGL

    [00:51:44] Inductive priors, why do we need them if we can start sparse?

    [00:56:02] Why anthropomorphise inductive priors?

    [01:02:13] Could Cerebras run a cyclic computational graph?

    [01:03:16] Are NNs locality sensitive hashing tables?



    References;

    Rigging the Lottery: Making All Tickets Winners [RIGL]

    https://arxiv.org/pdf/1911.11134.pdf



    [D] DanNet, the CUDA CNN of Dan Ciresan in Jurgen Schmidhuber's team, won 4 image recognition challenges prior to AlexNet

    https://www.reddit.com/r/MachineLearning/comments/dwnuwh/d_dannet_the_cuda_cnn_of_dan_ciresan_in_jurgen/ 



    A Spline Theory of Deep Learning [Balestriero]

    https://proceedings.mlr.press/v80/balestriero18b.html 

    • 1 hr 7 min

Customer Reviews

4.7 out of 5
33 Ratings

33 Ratings

Ryan7102 ,

Excellent guests and down-to-earth discussion

Naom Chomsky interview was AMAZING. I listened twice and followed up on every thread he mentioned. Props to interviewers for asking leading questions, and fixing up the audio. IF this podcast does nothing else again ever, it will have been worth it to capture this great man's (final?) thoughts on AI, which I've heard him discuss nowhere else (sadly he's gone down a political rabbit hole in his writings). Huge props to you gentlemen!

Todd Morrill ,

The real deal

I love these guys. They’re the real deal. They dig into tough topics with A-list guests. And as someone who builds deep learning models and has seen them come up short enough times, I absolutely appreciate all the time devoted to discussions of formal reasoning and symbolic systems.

Top Podcasts In Technology

Lex Fridman
Jason Calacanis
Cal Newport
The New York Times
NPR
Jack Rhysider

You Might Also Like

Sam Charrington
Changelog Media
NVIDIA
Kyle Polich
Santa Fe Institute, Michael Garfield
Sean Carroll | Wondery