The Cartesian Cafe Timothy Nguyen

Marcus Hutter is an artificial intelligence researcher who is both a Senior Researcher at Google DeepMind and an Honorary Professor in the Research School of Computer Science at Australian National University. He is responsible for the development of the theory of Universal Artificial Intelligence, for which he has written two books, one back in 2005 and one coming right off the press as we speak. Marcus is also the creator of the Hutter prize, for which you can win a sizable fortune for achieving state of the art lossless compression of Wikipedia text.
Patreon (bonus materials + video chat): https://www.patreon.com/timothynguyen
In this technical conversation, we cover material from Marcus’s two books “Universal Artificial Intelligence” (2005) and “Introduction to Universal Artificial Intelligence” (2024). The main goal is to develop a mathematical theory for combining sequential prediction (which seeks to predict the distribution of the next observation) together with action (which seeks to maximize expected reward), since these are among the problems that intelligent agents face when interacting in an unknown environment. Solomonoff induction provides a universal approach to sequence prediction in that it constructs an optimal prior (in a certain sense) over the space of all computable distributions of sequences, thus enabling Bayesian updating to enable convergence to the true predictive distribution (assuming the latter is computable). Combining Solomonoff induction with optimal action leads us to an agent known as AIXI, which in this theoretical setting, can be argued to be a mathematical incarnation of artificial general intelligence (AGI): it is an agent which acts optimally in general, unknown environments. The second half of our discussion concerning agents assumes familiarity with the basic setup of reinforcement learning.
I. Introduction
00:38 : Biography
01:45 : From Physics to AI
03:05 : Hutter Prize
06:25 : Overview of Universal Artificial Intelligence
11:10 : Technical outline
II. Universal Prediction
18:27 : Laplace’s Rule and Bayesian Sequence Prediction
40:54 : Different priors: KT estimator
44:39 : Sequence prediction for countable hypothesis class
53:23 : Generalized Solomonoff Bound (GSB)
57:56 : Example of GSB for uniform prior
1:04:24 : GSB for continuous hypothesis classes
1:08:28 : Context tree weighting
1:12:31 : Kolmogorov complexity
1:19:36 : Solomonoff Bound & Solomonoff Induction
1:21:27 : Optimality of Solomonoff Induction
1:24:48 : Solomonoff a priori distribution in terms of random Turing machines
1:28:37 : Large Language Models (LLMs)
1:37:07 : Using LLMs to emulate Solomonoff induction
1:41:41 : Loss functions
1:50:59 : Optimality of Solomonoff induction revisited
1:51:51 : Marvin Minsky
III. Universal Agents
1:52:42 : Recap and intro
1:55:59 : Setup
2:06:32 : Bayesian mixture environment
2:08:02 : AIxi. Bayes optimal policy vs optimal policy
2:11:27 : AIXI (AIxi with xi = Solomonoff a priori distribution)
2:12:04 : AIXI and AGI. Clarification: ASI (Artificial Super Intelligence) would be a more appropriate term than AGI for the AIXI agent.
2:12:41 : Legg-Hutter measure of intelligence
2:15:35 : AIXI explicit formula
2:23:53 : Other agents (optimistic agent, Thompson sampling, etc)
2:33:09 : Multiagent setting
2:39:38 : Grain of Truth problem
2:44:38 : Positive solution to Grain of Truth guarantees convergence to a Nash equilibria
2:45:01 : Computable approximations (simplifying assumptions on model classes): MDP, CTW, LLMs
2:56:13 : Outro: Brief philosophical remarks
 
Further Reading:M. Hutter, D. Quarrel, E. Catt. An Introduction to Universal Artificial IntelligenceM. Hutter. Universal Artificial IntelligenceS. Legg and M. Hutter. Universal Intelligence: A Definition of Machine Intelligence
 
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Richard Borcherds is a mathematician and professor at University of California Berkeley known for his work on lattices, group theory, and infinite-dimensional algebras. His numerous accolades include being awarded the Fields Medal in 1998 and being elected a fellow of the American Mathematical Society and the National Academy of Sciences.
Patreon (bonus materials + video chat): https://www.patreon.com/timothynguyen
In this episode, Richard and I give an overview of Richard's most famous result: his proof of the Monstrous Moonshine conjecture relating the monster group on the one hand and modular forms on the other. A remarkable feature of the proof is that it involves vertex algebras inspired from elements of string theory. Some familiarity with group theory and representation theory are assumed in our discussion.
I. Introduction
00:25: Biography
02:51 : Success in mathematics
04:04 : Monstrous Moonshine overview and John Conway
09:44 : Technical overview
II. Group Theory
11:31 : Classification of finite-simple groups + history of the monster group
18:03 : Conway groups + Leech lattice
22:13 : Why was the monster conjectured to exist + more history 28:43 : Centralizers and involutions
32:37: Griess algebra
III. Modular Forms
36:42 : Definitions
40:06 : The elliptic modular function
48:58 : Subgroups of SL_2(Z)
IV. Monstrous Moonshine Conjecture Statement
57:17: Representations of the monster
59:22 : Hauptmoduls
1:03:50 : Statement of the conjecture
1:07:06 : Atkin-Fong-Smith's first proof
1:09:34 : Frenkel-Lepowski-Meurman's work + significance of Borcherd's proof
V. Sketch of Proof
1:14:47: Vertex algebra and monster Lie algebra
1:21:02 : No ghost theorem from string theory
1:25:24 : What's special about dimension 26?
1:28:33 : Monster Lie algebra details
1:32:30 : Dynkin diagrams and Kac-Moody algebras
1:43:21 : Simple roots and an obscure identity
1:45:13: Weyl denominator formula, Vandermonde identity
1:52:14 : Chasing down where modular forms got smuggled in
1:55:03 : Final calculations
VI. Epilogue
1:57:53 : Your most proud result?
2:00:47 : Monstrous moonshine for other sporadic groups?
2:02:28 : Connections to other fields. Witten and black holes and mock modular forms.
 
Further reading: V Tatitschef. A short introduction to Monstrous Moonshine. https://arxiv.org/pdf/1902.03118.pdf
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Thought I'd share some exciting news about what's happening at The Cartesian Cafe in 2024 and also a personal message to viewers on how they can support the cafe.
Patreon:
https://www.patreon.com/timothynguyen

Tim Maudlin is a philosopher of science specializing in the foundations of physics, metaphysics, and logic. He is a professor at New York University, a member of the Foundational Questions Institute, and the founder and director of the John Bell Institute for the Foundations of Physics.
Patreon (bonus materials + video chat):https://www.patreon.com/timothynguyen
In this very in-depth discussion, Tim and I probe the foundations of science through the avenues of locality and determinism as arising from the Einstein-Poldosky-Rosen (EPR) paradox and Bell's Theorem. These issues are so intricate that even the Nobel Prize committee incorrectly described the significance of Bell's work in their press release for the 2022 prize in physics. Viewers motivated enough to think deeply about these ideas will be rewarded with a conceptually proper understanding of the nonlocal nature of physics and its manifestation in quantum theory.
I. Introduction 00:00 :
00:25: Biography
05:26: Interdisciplinary work
11:54 : Physicists working on the wrong things
16:47 : Bell's Theorem soft overview
24:14: Common misunderstanding of "God does not play dice."
25:59: Technical outline
II. EPR Paradox / Argument
29:14 : EPR is not a paradox
34:57 : Criterion of reality
43:57 : Mathematical formulation
46:32 : Locality: No spooky action at a distance
49:54 : Bertlmann's socks
53:17 : EPR syllogism summarized
54:52 : Determinism is inferred not assumed
1:02:18 : Clarifying analogy: Coin flips
1:06:39 : Einstein's objection to determinism revisited
III. Bohm Segue
1:11:05 : Introduction
1:13:38: Bell and von Neumann's error
1:20:14: Bell's motivation: Can I remove Bohm's nonlocality?
IV. Bell's Theorem and Related Examples
1:25:13 : Setup
1:27:59 : Decoding Bell's words: Locality is the key!
1:34:16 : Bell's inequality (overview)
1:36:46 : Bell's inequality (math)
1:39:15 : Concrete example of violation of Bell's inequality
1:49:42: GHZ Example
V. Miscellany
2:06:23 : Statistical independence assumption
2:13:18: The 2022 Nobel Prize
2:17:43: Misconceptions and hidden variables
2:22:28: The assumption of local realism? Repeat: Determinism is a conclusion not an assumption.
VI. Interpretations of Quantum Mechanics
2:28:44: Interpretation is a misnomer
2:29:48: Three requirements. You can only pick two.
2:34:52: Copenhagen interpretation?
 
Further Reading:
J. Bell. Speakable and Unspeakable in Quantum Mechanics
T. Maudlin. Quantum Non-Locality and Relativity
Wikipedia: Mermin's device, GHZ experiment
 
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Antonio (Tony) Padilla is a theoretical physicist and cosmologist at the University of Nottingham. He serves as the Associate Director of the Nottingham Centre of Gravity, and in 2016, Tony shared the Buchalter Cosmology Prize for his work on the cosmological constant. Tony is also a star of the Numberphile YouTube channel, where his videos have received millions of views and he is also the author of the book Fantastic Numbers and Where to Find Them: A Cosmic Quest from Zero to Infinity.
Patreon: https://www.patreon.com/timothynguyen
This episode combines some of the greatest cosmological questions together with mathematical imagination. Tony and I go through the math behind some oft-quoted numbers in cosmology and calculate the age, size, and number of atoms in the universe. We then stretch our brains and consider how likely it would be to find your Doppelganger in a truly large universe, which takes us on a detour through black hole entropy. We end with a discussion of naturalness and the anthropic principle to round out our discussion of fantastic numbers in physics.
Part I. Introduction
00:00 : Introduction
01:06 : Math and or versus physics
12:09 : Backstory behind Tony's book
14:12 : Joke about theoreticians and numbers
16:18 : Technical outline
Part II. Size, Age, and Quantity in the Universe
21:42 : Size of the observable universe
22:32 : Standard candles
27:39 : Hubble rate
29:02 : Measuring distances and time
37:15 : Einstein and Minkowski
40:52 : Definition of Hubble parameter
42:14 : Friedmann equation
47:11 : Calculating the size of the observable universe
51:24 : Age of the universe
56:14 : Number of atoms in the observable universe
1:01:08 : Critical density
1:03:16: 10^80 atoms of hydrogen
1:03:46 : Universe versus observable universe
Part III. Extreme Physics and Doppelgangers
1:07:27 : Long-term fate of the universe
1:08:28 : Black holes and a googol years
1:09:59 : Poincare recurrence
1:13:23 : Doppelgangers in a googolplex meter wide universe
1:16:40 : Finitely many states and black hole entropy
1:25:00 : Black holes have no hair
1:29:30 : Beckenstein, Christodolou, Hawking
1:33:12 : Susskind's thought experiment: Maximum entropy of space
1:42:58 : Estimating the number of doppelgangers
1:54:21 : Poincare recurrence: Tower of four exponents.
Part IV: Naturalness and Anthropics
1:54:34 : What is naturalness? Examples.
2:04:09 : Cosmological constant problem: 10^120 discrepancy
2:07:29 : Interlude: Energy shift clarification. Gravity is key.
2:15:34 : Corrections to the cosmological constant
2:18:47 : String theory landscape: 10^500 possibilities
2:20:41 : Anthropic selection
2:25:59 : Is the anthropic principle unscientific? Weinberg and predictions.
2:29:17 : Vacuum sequestration
Further reading: Antonio Padilla. Fantastic Numbers and Where to Find Them: A Cosmic Quest from Zero to Infinity
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org

Boaz Barak is a professor of computer science at Harvard University, having previously been a principal researcher at Microsoft Research and a professor at Princeton University. His research interests span many areas of theoretical computer science including cryptography, computational complexity, and the foundations of machine learning. Boaz serves on the scientific advisory boards for Quanta Magazine and the Simons Institute for the Theory of Computing and he was selected for Foreign Policy magazine’s list of 100 leading global thinkers for 2014.
www.patreon.com/timothynguyen
Cryptography is about maintaining the privacy and security of communication. In this episode, Boaz and I go through the fundamentals of cryptography from a foundational mathematical perspective. We start with some historical examples of attempts at encrypting messages and how they failed. After some guesses as to how one might mathematically define security, we arrive at the one due to Shannon. The resulting definition of perfect secrecy turns out to be too rigid, which leads us to the notion of computational secrecy that forms the foundation of modern cryptographic systems. We then show how the existence of pseudorandom generators (which remains a conjecture) ensures that such computational secrecy is achievable, assuming P does not equal NP. Having covered private key cryptography in detail, we then give a brief overview of public key cryptography. We end with a brief discussion of Bitcoin, machine learning, deepfakes, and potential doomsday scenarios.
I. Introduction
00:17 : Biography: Academia vs Industry
10:07 : Military service
12:53 : Technical overview
17:01 : Whiteboard outline
II. Warmup
24:42 : Substitution ciphers
27:33 : Viginere cipher
29:35 : Babbage and Kasiski
31:25 : Enigma and WW2
33:10 : Alan Turing
III. Private Key Cryptography: Perfect Secrecy
34:32 : Valid encryption scheme
40:14 : Kerckhoffs's Principle
42:41 : Cryptography = steelman your adversary
44:40 : Attempt #1 at perfect secrecy
49:58 : Attempt #2 at perfect secrecy
56:02 : Definition of perfect secrecy (Shannon)
1:05:56 : Enigma was not perfectly secure
1:08:51 : Analogy with differential privacy
1:11:10 : Example: One-time pad (OTP)
1:20:07 : Drawbacks of OTP and Soviet KGB misuse
1:21:43 : Important: Keys cannot be reused!
1:27:48 : Shannon's Impossibility Theorem
IV. Computational Secrecy
1:32:52 : Relax perfect secrecy to computational secrecy
1:41:04 : What computational secrecy buys (if P is not NP)
1:44:35 : Pseudorandom generators (PRGs)
1:47:03 : PRG definition
1:52:30 : PRGs and P vs NP
1:55:47: PRGs enable modifying OTP for computational secrecy
V. Public Key Cryptography
2:00:32 : Limitations of private key cryptography
2:09:25 : Overview of public key methods
2:13:28 : Post quantum cryptography
VI. Applications
2:14:39 : Bitcoin
2:18:21 : Digital signatures (authentication)
2:23:56 : Machine learning and deepfakes
2:30:31 : A conceivable doomsday scenario: P = NP
Further reading: Boaz Barak. An Intensive Introduction to Cryptography
Twitter: @iamtimnguyen
Webpage: http://www.timothynguyen.org