core.py Pablo Galindo and Łukasz Langa

Oof, no episode in April, huh? Yeah, we're getting close to Python 3.13 beta 1. PyCon US is also coming up real soon. Let's use this opportunity then to talk about a feature we're teaming up on: a better interactive interpreter!



## Outline



(00:00:00)  INTRO

(00:01:53)  PART 1: History of Terminals

(00:03:20)  /dev/tty

(00:04:51)  The first cool word

(00:05:45)  Chrząszcz

(00:06:20)  Control code characters in ASCII

(00:11:54)  PART 2: Python REPL Today

(00:12:34)  There is no REPL

(00:15:28)  So what is there instead?

(00:19:13)  readline

(00:25:38)  Source in the REPL

(00:31:13)  Implementing a REPL from scratch? Prepare to support arg: 5

(00:36:09)  PART 3: PR OF THE WEEK

(00:37:09)  Introducing: Complaining Pablo

(00:38:23)  Tests are always green if you skip them

(00:39:57)  Getting dirty with escape sequences

(00:41:28)  Typing finds bugs

(00:42:29)  Shiny new features of the new REPL

(00:45:55)  Contributing back to PyPy

(00:48:10)  We still have two weeks, right?

(00:49:59)  Is Python synthwave enough?

(00:51:57)  Do we have a bug?

(00:55:31)  What's lurking in pydoc?

(00:59:38)  PART 4: WHAT'S HAPPENING IN CPYTHON?

(01:02:39)  PEP 744: The JIT

(01:06:05)  Incremental GC is now actually in

(01:08:21)  Tier 2 interpreter updates

(01:10:29)  Python supported on iOS with PEP 730

(01:13:11)  Better error messages for name shadowing

(01:15:17)  Queue.shutdown()

(01:17:14)  ctypes adopts heap types

(01:18:26)  Free-threading updates

(01:20:14)  Dataclass creation is faster

(01:20:44)  OUTRO

Let's talk about the Steering Council, running a small consultancy business, the Walrus, and pet peeves with our special guest today!



## Outline



(00:00:00)  INTRO

(00:00:56)  PART 1: Emily Morehouse

(00:02:15)  Running a small consultancy business

(00:04:39)  What features of JS do you miss in Python?

(00:05:50)  Łukasz outnumbered in a world of Steering Council members

(00:06:12)  Upgrading to new Python versions

(00:07:00)  It depends on who deployed the project

(00:09:44)  Second term as a Steering Council member

(00:11:33)  Barry, play some bass for us

(00:13:04)  Let's hear a recent war story

(00:15:17)  Is this progress bar even working?

(00:17:40)  The Villain Origin Story

(00:21:37)  Emily, The Bringer of Doom

(00:22:37)  Consensus within the Steering Council

(00:25:52)  Syntax changes in Python are rare, right? Right?

(00:28:22)  On implementing PEP 572

(00:32:52)  How would PyCon 2020 in Pittsburgh feel?

(00:34:18)  How can you be mad about the Walrus?

(00:36:10)  Favorite parts of the standard library

(00:38:10)  Is hacking on Python a good experience to newcomers?

(00:40:26)  Emily's pet peeve about Python, take 1

(00:42:17)  Emily's favorite change in Python in recent years

(00:44:34)  Emily's pet peeve about Python, take 2

(00:46:34)  Łukasz's pet peeve

(00:48:25)  Surprise extra question

(00:49:42)  At core.py we are professionals

(00:51:00)  PART 2: PR of the Week

(00:54:00)  CALL TO ACTION: Upgrade Python.org to Django 4!

(00:56:22)  PART 3: What's Going On in CPython?

(00:56:38)  Faster Python updates

(01:00:10)  Free threading: GIL can be disabled but we're not done yet!

(01:04:17)  New defaults for SSL context flags

(01:05:39)  python -m asyncio and sys.__interactivehook__

(01:06:24)  Surprise question: what is sys.__interactivehook__ even doing?

(01:08:11)  OUTRO

The suspense was killing us! OK, the old parser was then... but what about NOW? We're finally answering this question... in more detail than you dared to ask for. PEG, memoization, funky secrets, and how a certain auto-formatter self-inflicted an existential crisis on itself. It's all there, told in barely 100 minutes! Can you believe it?



# Timestamps



(00:00:00)  INTRO

(00:00:54)  PART 1: What even is PEG?

(00:04:02)  You can't prove anything!

(00:05:03)  What's a "parsing expression"?

(00:08:23)  Our old LL1 parser wasn't doing its job

(00:09:37)  "Soft keywords" in LL1: A Horror Story

(00:13:16)  PART 2: How PEG was adopted by Python

(00:17:10)  Why not LALR?

(00:22:11)  The PEG paper wasn't enough either, if we're honest

(00:26:26)  Less obvious advantages of the new parser

(00:31:28)  Black is stuck with LL1, can it cope?

(00:36:24)  Hedging against Łukasz, the bringer of doom

(00:41:14)  PART 3: How does the PEG parser of CPython work?

(00:44:30)  Pedantic Pablo on "exponential"

(00:45:14)  Fresh news from literally yesterday last week

(00:46:39)  Pedantic Pablo on "infinite"

(00:47:32)  Memoization in the PEG parser

(00:50:41)  Parse once, and if it fails, try again!

(00:52:14)  How to model a grammar of programming mistakes?

(00:56:36)  Why is there C code in my grammar file?

(00:59:57)  Bro, do you even lift?

(01:01:45)  How soft keywords work today: it's not free lunch

(01:04:29)  Funky grammar secrets

(01:09:07)  PART 4: PR OF THE WEEK

(01:09:15)  audioop.c license shenanigans

(01:14:56)  The secret profiler inside CPython (tests)

(01:22:45)  PART 5: WHAT'S GOING ON IN CPYTHON?

(01:23:30)  Free-threading changes

(01:28:15)  Faster Python changes

(01:35:39)  End of an era: docs get rid of Python 2 migration info

(01:36:45)  Python --help output is now nicer

(01:38:43) SQLite as a dbm backend

(01:41:08)  OUTRO

Context-free grammars, non-deterministic finite automatons, left-to-right leftmost derivations... what even is all that?! Today we're talking about how Python parses your source code. We start gently with how this worked in the past. Come listen to Łukasz's high-level explanations and Pedantic Pablo's "well actuallys".



# Timestamps



(00:00:00) INTRO

(00:01:35) You can still download Python 1.0!

(00:02:19) The original tokenizer

(00:03:10) What even is a tokenizer?

(00:04:08) FUN FACTS ABOUT THE TOKENIZER

(00:04:34) Circumflex

(00:05:16) Python's invisible braces

(00:08:29) Backticks in the syntax

(00:11:00) Where are the comments stored?

(00:12:27) GRAMMAR

(00:13:37) What is a grammar?

(00:16:25) The long-forgotten 'access' keyword

(00:20:25) Making LL1 do things it wasn't meant to do

(00:23:24) SURPRISE QUESTION 1: soft keywords

(00:24:46) What's a context-free grammar?

(00:26:51) A note about backslashes

(00:29:33) The Dragon Book(s)

(00:31:27) PARSING: What is it?

(00:35:23) How to generate a parser?

(00:39:00) LL Cool Parser

(00:41:15) What if we used LR?

(00:44:01) Let's have three tokenizers!

(00:47:50) 2to3 and its legacy

(00:52:38) Black and its blib2to3

(00:54:04) The pesky 'with' statement and the death of LL1

(01:00:05) PR OF THE WEEK: GH-113745

(01:05:41) SURPRISE QUESTION 2: Subclasses of SyntaxError

(01:07:02) WHAT'S GOING ON IN CPYTHON?

(01:09:16) Sam Gross nominated as a core dev

(01:10:13) Free-threading progress

(01:13:11) Faster CPython changes

(01:17:29) ntpath.isreserved()

(01:20:11) Pablo and the DWARF

(01:22:02) OUTRO

Happy New Year! In this episode we're talking about exceptions, how they work, and how they evolved. Expect the unexpected.



# Timestamps



(00:00:00)  INTRO

(00:01:43)  How does a 'try' block work?

(00:04:00)  How many 'try' blocks can you fit on a bus?

(00:05:56)  How does Python store the current exception?

(00:09:30)  Pre-history: exceptions as strings

(00:12:59)  Try out string exceptions with CentOS 5

(00:14:28)  PEP 341: Combining 'finally' and 'except' in one 'try' block

(00:16:15)  Core Hacker in Residence

(00:16:51)  PEP 3109: Raising exceptions in Python 3K

(00:19:17)  Automatic tuple unpacking for raised exceptions?!

(00:21:55)  PEP 3110: Catching exceptions in Python 3K

(00:26:00)  Foreshadowing: exception groups

(00:27:10)  PEP 3134: Exception chaining

(00:29:12)  __cause__, __context__, __traceback__

(00:31:50)  Back in the day we had to walk uphill both ways

(00:32:56)  PEP 409: Suppressing exception chaining

(00:34:44)  Raise from None? Or raise from Ellipsis?

(00:37:11)  __supress_context__

(00:38:13)  Semantic difference between 'pass', 'None', and '....'

(00:41:02)  NotImplemented vs NotImplementedError

(00:43:02)  Zero-cost exceptions in Python 3.11

(00:51:12)  Reconstructing exception table entries dynamically

(00:52:51)  Objects/exception_handling_notes.txt

(00:54:19)  PEP 654: Historical context

(00:56:58)  PEP 654: BaseExceptionGroup and ExceptionGroup

(00:58:29)  PEP 654: except*

(01:03:23)  PEP 678: Exception notes

(01:06:09)  PEP 657: Cooler Errors

(01:08:36)  A message to language implementers

(01:09:15)  Fine-grained error locations in tracebacks

(01:12:05)  This is useful for tracing coverage and Specializator

(01:13:50)  Hacker in Residence fired?

(01:16:25)  WHAT'S GOING ON IN CPYTHON

(01:16:58)  Copy&patch JIT PR open

(01:21:30)  Free-threading progress: GC split, stop-the-world

(01:24:22)  The buildbots are red

(01:25:27)  Faster CPython changes: interpreter code generator refactors

(01:26:27)  Eval Game crash fix

(01:28:41)  Three developers in residence!

(01:29:54)  OUTRO



# Links



https://compilercrim.es/rust-np/

This time we're hosting a special guest: Carl Meyer from Meta. What is Cinder, how does it work, and how does it intersect with the future of Python 3? Find out in today's episode. 100% serious stuff!



# Timestamps



(00:00:00)  INTRO

(00:00:53)  Carl Meyer's war story

(00:02:27)  CINDER

(00:03:22)  Static Python makes things significantly faster

(00:08:15)  Cinder JIT and how it's tuned for Instagram

(00:11:57)  Strict Python and the joy of import side effects

(00:16:35)  The static typing controversy

(00:18:52)  Upstreaming changes from Cinder?

(00:22:53)  PEP 709: Comprehension inlining

(00:28:35)  pip install CinderX

(00:31:19)  Immortal instances

(00:35:15)  asyncio.eager_task_factory()

(00:39:39)  Carl's pet peeve with Python

(00:44:49)  PR OF THE WEEK: PyPy's REPL in CPython

(00:52:07)  WHAT'S GOING ON IN CPYTHON

(00:52:22)  Python 3.12.1

(00:53:17)  Python 3.11.7

(00:54:45)  multiprocessing.SharedMemory track

(00:56:49)  Fine-grained error locations for multi-line expressions

(01:00:03)  libedit tab completion is fixed

(01:02:14)  Colored exception tracebacks

(01:05:11)  Removing testing modules from sys.modules, correctly

(01:06:47)  SBOMs are a very serious matter

(01:09:08)  Arrays by value on ARM

(01:12:24)  Remove development environments and CAPS LOCK

(01:15:30)  Interpreter cases generator refactored

(01:16:17)  Free-threading news

(01:20:01)  OUTRO