Python behind the scenes: a list of resources
This is a collection of documents, posts and talks that I found useful while studying CPython and working on the CPython behind the scenes series. There are some really good resources, but they didn't help to answer many of my questions. That's why I eventually decided to share my own experience of working through the CPython's source code.
-
Python Documentation. Python documentation is the first place to go to for almost any Python-related topic. While it lacks a special part covering internals of the interpreter, Python/C API reference describes CPython's public interfaces and some of the implementation details. It's accompanied by the tutorial for C programmers, which shows how to extend a Python program with C or to embed Python inside a C application. I bet you'll understand how CPython works quite well just by accomplishing thoughtfully one of these tasks.
-
PEPs. You won't find a PEP that describes the overall design of the interpreter, but most of the major changes to the language have a corresponding proposal. PEPs are great. They provide you with both technical and historical contexts. Authors of documentation have a good habit of referencing a relevant PEP when appropriate. For example, Python/C API reference mentions at least once PEP 432 that describes a transition to a new CPython initialization sequence and PEP 587, which is its partial implementation.
- Inside The Python Virtual Machine book by Obi Ike-Nwosu. The most comprehensive and, as I can judge, accurate source on the CPython internals.
- Your Guide to the CPython Source Code by Anthony Shaw. The post's title speaks for itself. If you want to dig into the source code straight away, this should be your choice!
- CPython internals: A ten-hour codewalk through the Python interpreter source code by Philip Guo. If you want a more gentle introduction, these video lectures is probably the best place to start. They cover CPython 2.7, but the principles remain the same.
- Python’s Innards series by Yaniv Aknin covers the early version of the CPython 3 VM, which is still very relevant.
- Python internals posts by Eli Bendersky. The posts on symbol tables are especially good.
- Stupid Python Ideas blog. A collection of posts on various Python-related topics. Some touch upon the CPython internals. Highly recommended but can be hard to navigate through.
- A Python Interpreter Written in Python by Allison Kaptur. The article describes a toy Python VM that is able to execute real Python bytecode. It might be useful as a first introduction to the interpreter. I don't see a good reason, though, to spend much time studying a toy example when CPython itself is not overwhelmingly complicated.
- To GIL or not to GIL talk by Eric Snow. The main theme of the talk is subinterpretes. What I like about it is how Eric gives an overview of CPython architecture in the beginning.