Professional Documents
Culture Documents
11
YOU WILL LEARN ABOUT:
1. Changes to Python in the 3.11 release
2. How the 1.2x speed-up was achieved
3. New, more precise messaging in tracebacks
4. The addition of TOML to the standard library
5. New features with Exceptions and the new TaskGroup
mechanism
OVERVIEW
● Python 3.11 took a little longer than usual, but there is good stuff in it
● Big focus on performance improvement: 1.2x speed-up on average
● Further improvement to error messaging
● TOML has been added to the standard library (read-only)
● New features in Exceptions:
● ExceptionGroup
● Adding notes
OVERVIEW
● New mechanism in asyncio for creating tasks
● Context manager based
● Takes advantage of Exception groups
● Additions to the typing module:
● Self
● Variadic types
● Dataclass transform
NEXT UP...
FASTER CODE
● On-going Faster CPython Project
● Python 3.11 is 10-60% faster than Python 3.10
● Average 1.22x speed-up on standard benchmark suite
ADAPTIVE INTERPRETER
● PEP 659: Specializing Adaptive Interpreter
● Interpreter now adapts types or values for performance gains
● “Specializes code aggressively, but over a very small region”
● Byte-code gets adjusted after running several times
● LOAD_ATTR → LOAD_ATTR_ADAPTIVE
● LOAD_ATTR_INSTANCE_VALUE
● LOAD_ATTR_MODULE
● LOAD_ATTR_SLOT
FASTER STARTUP
● Python caches your bytecode in the __pycache__ directory
● At start-up it:
● Reads the cache
● Unmarshals the objects
● Allocates the code on the heap
● Evaluates the code
● Python 3.11 is freezing key core modules
● Code is statically allocated
● Can be directly loaded
● 10-15% improvement on interpreter loading times
NEXT UP...
BETTER TRACEBACKS
● PEP 657: Include Fine-Grained Error Locations in Tracebacks
● Continuation of error message improvement from Python 3.10
● Traceback messages now have more detail as to where in the statement a
problem occurred
NEXT UP...
TOML
● TOML: Tom’s Obvious, Minimal Language
● Configuration file format
● Similar to Windows INI files
● Third-party libraries, now part of the standard library
● Being used by many packaging tools
DATA TYPES
● Comments
● Key-value pairs
● Arrays
● Tables
● Inline tables
● Integers, floats, booleans
● Dates, times
PYTHON’S TOML
● Commonly used by a lot of Python tools:
● black, mypy, pytest, tox, pylint, isort, ...
● Only supports binary files to ensure UTF-8 encoding
● Interface is consistent with json.load and pickle.load
● Python’s standard library implementation is read-only
● Third-party style-preserving TOML library: TOML Kit
https://github.com/sdispater/tomlkit
NEXT UP...
Exceptional exceptions
TABLE OF CONTENTS
1. Overview
2. Faster, Faster, Faster
3. Improved Traceback Messages
4. TOML
5. Exceptions & Task Groups
6. Typing Features
7. Odds and Ends
8. Summary and Further Investigation
EXCEPTION NOTES
● PEP 678: Enriching Exceptions with Notes
● Add information to an exception
● Especially helpful for re-raising
EXCEPTION GROUPS
● Exception handling code may itself cause an exception
● This can be hard to understand
● An ExceptionGroup is a way of grouping exceptions together
● The new except* syntax allows you to react to a subset of an
ExceptionGroup
TASK GROUPS
● The ascynio library has a new way of creating tasks
● TaskGroup is used as a context manager
● Uses an ExceptionGroup when problems happen in an asynchronous
task
COROUTINE PRIMER
● ascynio is an abstraction for parallel execution of code
● Uses the async and await keywords
● Tasks need to be created, then waited upon until completion
abcABdeC
COMPARISON
async def bootstrap():
tasks = [
asyncio.create_task(write_letters(string.ascii_lowercase)),
asyncio.create_task(write_letters(string.ascii_uppercase)),
]
await asyncio.gather(*tasks)
NEXT UP...
SELF TYPE
● typing.Self indicates the use of a class’s own type
● Useful with factory methods
import json
from dataclasses import dataclass
from typing import Self
@dataclass
class Person:
name: str
pe
age: int
ty
y
M
@classmethod
def from_json(cls, data: str) -> Self:
p = json.loads(data)
return Person(p["name"], p["int"])
VARIADIC TYPE
● typing.TypeVar indicates parameterization with a single type
● typing.TypeVarTuple indicates parameterization with an arbitrary number of types
gs
g
in
T = TypeVar('T')
in
Th
Th
Ts = TypeVarTuple('Ts')
y
e
an
On
M
def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]:
return (*tup[1:], tup[0])
DATACLASS TRANSFORM
● Many libraries use data class like semantics
● Django, DRF, Pydantic, Ninja...
● typing.datclass_transform indicates to type checkers that a class
should be treated as a data class
● Decorator for: functions, classes, metaclasses
● Example:
● A transformed ModelBase class indicates children are data-class-
like
● A CustomerModel(ModelBase) class with id and name
attributes
● Type checker expects the __init__ method to take id and name
class Movie(TypedDict):
title: str
year: NotRequired[int]
NEXT UP...
NEGATIVE ZERO
● Floating point numbers often aren’t what you expect
>>> 0.1 + 0.2
0.30000000000000004
NEXT UP...
Should you?
TABLE OF CONTENTS
1. Overview
2. Faster, Faster, Faster
3. Improved Traceback Messages
4. TOML
5. Exceptions & Task Groups
6. Typing Features
7. Odds and Ends
8. Summary and Further Investigation
SUMMARY
● Python 3.11 has lots of new things
● Big focus on making Python faster
● Continued improvement on error messages, tracebacks
● Addition of TOML into the standard library
● Exception groups and notes
● New task creation mechanism for asyncio
● More additions to the typing module