Professional Documents
Culture Documents
Farhad Malik
Oct 4 · 11 min read
The latest Python 3.9.0 final version is out on the Monday, 2020–10–05
Just like most of the Python fans, I am super excited to explore and use the latest
features. This article will provide an overview of the must-know features of Python 3.9.
Image By Author
I read through the Python 3.9 release notes and the associated discussions. Based on
the information, I wanted to write a comprehensive guide so everyone can get a
glimpse of the features along with their detailed workings.
Before I begin, I have to say, I am very excited to explore version 3.9 as some of the
features are definitely going to be used in my applications.
In A Nutshell
From the dictionary update/merge to the addition of new string methods to the
introduction of zoneinfo library, a number of new features have been added.
Furthermore, a new stable and high-performant parser has been introduced.
The standard library is updated with numerous new features along with the addition
of new modules, zoneinfo and graphlib . A number of modules have been improved
The | operator is used to merge dictionaries whereas the |= operator can be used to
update the dictionaries.
PEP: 584
Code:
For Merge: |
For Update: |=
>>> a |= b
>>> a
{’blog’: 2, 'python’: 3,’farhad’:’malik’}
The key rule to remember is that if there are any key conflicts then the rightmost-value
will be kept. It means that the last seen value always wins. This is the current behavior
of other dict operations too.
Detailed Explanation:
As we can see above, the two new operators, | and |= , have been added to the built-
in dict class.
The | operator is used to merge dictionaries whereas the |= operator can be used to
update the dictionaries.
If we assess version 3.8, we will notice that there are few ways of merging and
updating dictionaries.
approach is that it modifies first_dict in place. One way to solve this issue is to copy
the first_dict in a temporary variable and then perform an update. However, it adds
extra unnecessary code just to get the update/merge to work.
We can also use {**first_dict, **second_dict} . The issue with this approach is that it
is not easily discoverable and it’s harder to understand what the code is intended to
perform. The other issue with this approach is that the mappings types are ignored and
the type is always dict. As an instance, if the first_dict is a defaultdict and the
second_dict is of type dict then it will fail.
Lastly, the collections library contains a ChainMap function. It can take in two
dictionaries such as ChainMap(first_dict, second_dict) and return a merged
dictionary but again, this library is not commonly-known.
It also fails for subclasses of dict that have an incompatible __init__ method.
Image By Author
PEP: 617
Detailed Explanation:
The current CPython parser is LL(1) based. Subsequently, the grammar is LL(1) based
to allow it to be parsed by the LL(1) parser. The LL(1) parser is a top-down parser.
Furthermore, it parses the inputs from left to right. The current grammar is context-
free grammar hence the context of the tokens is not taken into account.
The Python 3.9 version is proposing to replace it with a new PEG-based parser which
means it will lift the current LL(1) grammar Python restrictions. Additionally, the
current parser has been patched with a number of hacks that are going to be removed.
As a result, it will reduce the maintenance cost in long run.
As an instance, although the LL(1) parsers and grammars are simple to implement, the
restrictions do not allow them to express common constructs in a natural way to the
language designer and the reader. The parser only looks at one token ahead to
distinguish possibilities.
The choice operator | is ordered. For an instance, if the following rule is written:
rule: A|B|C
The LL(1) parser, a context-free grammar parser, will generate constructions such that
given an input string will deduce whether it needs to expand A or B or C. The PEG
parser is different. It will check if the first alternative succeeds. If it fails only then it
will continue with the second or the third.
The PEG parser generates exactly one valid-tree for a string. Hence it’s not ambiguous
like the LL(1) parser.
The PEG parser also directly generates the AST nodes for a rule via grammar actions.
This means it avoids the generation of the intermediate steps.
The key point to take is that the PEG parser has been extensively tested and validated.
The PEG parser performance is fine-tuned. As a result, for most instructions, it comes
within 10% of the memory and speed consumption of the current parser. This is mainly
because the intermediate syntax tree is not constructed.
I am eliminating the mention of the low-level details for the sake of keeping the
article simple and readable. The link is provided at the
bottom if more information is required to be understood.
Image By Author
PEP: 616
Code:
'farhad_python'.removeprefix('farhad_')
#returns python
'farhad_python'.removesuffix('_python')
#returns farhad
Detailed Explanation:
One of the common tasks in a data science application that involves manipulating text
is to remove the prefix/suffix of the strings. Two new functions have been added to the
str object. These functions can be used to remove unneeded prefix and suffix from a
string.
If we study the functions, internally they check if the string starts with a prefix (or ends
with a suffix) and if it does then they return a string without a prefix (or after a suffix)
using str[:] slicing feature.
With these functions being part of the standard library, we get an API that is consistent,
less fragile, high performant and is more descriptive.
The release has enabled support for the generics syntax in all standard collections
currently available in the typing module.
We can use the list or dict built-in collection types as generic types instead of using
the typing.List or typing.Dict in the signature of our function.
Therefore, the code now looks cleaner and it has made it easier to understand/explain
the code.
PEP: 585
Detailed Explanation:
Although Python is a dynamically typed language, the annotation of the types in the
Python program enables introspection of the type. Subsequently, the annotation can be
used for API generation of runtime type checking.
This release has enabled support for the generics syntax in all standard collections
currently available in the typing module.
A generic type is usually a container e.g. list. It is a type that can be parameterized.
Parameterized generic is an instance of a generic with the expected types for container
elements e.g. list[str]
We can use the list or dict built-in collection types as generic types instead of using
the typing.List or typing.Dict.
For example, we could guide the Python runtime type checking by annotating the
code:
Over the past few releases, a number of static typing features have been built
incrementally on top of the existing Python runtime. Some of these features were
constrained by existing syntax and runtime behavior. As a consequence, there was a
duplicated collection hierarchy in the typing module due to generics.
For instance, we will see typing.List , typing.Dictionary along with built-in list ,
PEP: 615
IANA time zones are often called tz or zone info . There are a large number of IANA
time zones with different search paths to specify the IANA timezone to a date-time
object. As an instance, we can pass in the name of the search path as the
Continent/City to a datetime object to set its tzinfo .
Detailed Explanation:
We use the datetime library to create a datetime object and specify its
timezone by setting the tzinfo propety. However, we can end up creating complex
timezone rules when using the datetime.tzinfo baseline.
Most of time, we only need to set the object and set its timezone to either UTC, system
local time zone, or IANA time zone.
Code:
This parameter cancels all of the pending futures that have not started. Prior to
version 3.9, the process would wait for them to complete before shutting down the
executor.
Explanation:
The new parameter cancel_futures have been added to both ThreadPoolExecutor and
ProcessPoolExecutor . The way it works is when the value of the parameter is True then
Once there are no pending work items then it shuts down the worker.
Link: https://bugs.python.org/issue30966
As an instance,
With regards to the multiprocessing library improvements, a new method close() has
been added to the multiprocessing.SimpleQueue class.
This method explicitly closes the queue. This will ensure that the queue is closed and
does not stay around for longer than expected. The key to remember is that the
methods get(), put(), empty() must not be called once the queue is closed.
Link: https://bugs.python.org/issue30966
It has been fixed now. The __import__() now raises ImportError instead of
ValueError .
We can generate random numbers but what if we needed to generate random bytes?
Prior to 3.9 version, the developers had to get creative to generate the random bytes.
Although we can use os.getrandom(), os.urandom() or secrets.token_bytes() but we
can’t generate pseudo-random patterns.
As an instance, to ensure the random numbers are generated with expected behaviour
and the process is reproducible, we generally use the seed with random.Random
module.
This bug confused the users and caused inconsistent behavior in applications.
The 3.9 release has fixed this issue and it is now consistent with "".replace("", s) .
The way replace function works is that for a given max replace occurrence argument,
it replaces a set of characters from the string by a new set of characters.
To further explain the issue, prior to the version 3.9, the replace function had
inconsistent behaviour
"".replace("", "blog", 1)
Returns ’'
One would expect to see blog
"".replace("", "|", 1)
Returns ’'
One would expect to see |
"".replace("", "prefix")
Howver returns ’prefix'
A number of redundant features have been eliminated in the Python 3.9 too, such as
Py_UNICODE_MATCH.
I am very excited to explore 3.9 as some of the features are definitely going to be used
in my applications.
If you want to read more about those improvements, please read the official guide
here.
Your email
By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more
information about our privacy practices.