PART IV CHAPTER 16: Function Basics

Functions and CHAPTER 17: Scopes

CHAPTER 18: Arguments
Generators CHAPTER 19: Advanced Function Topics
CHAPTER 20: Comprehensions and
Generations
CHAPTER 21: The Benchmarking Interlude

1
CHAPTER 16: Function Basics

2
CHAPTER 16: Function Basics
● Why Use Functions?
● Coding Functions
● A First Example: Definitions and Calls
● A Second Example: Intersecting Sequences

3
CHAPTER 16: Why Use Functions?
● Maximizing code reuse and minimizing redundancy
○ code in a single place and use it in many places
○ reduce code redundancy
○ reduce maintenance effort
● Procedural decomposition
○ split systems into pieces that have well- defined roles.

4
CHAPTER 16: Coding Functions

5
CHAPTER 16: def Statements
● creates a function object and assigns it to a name

def name(arg1, arg2,... argN): # headline

statements # body: block of statements

def name(arg1, arg2,... argN):

return value # often contain a return statement, otherwise
return None object which usually ignored at the call

6
CHAPTER 16: def Executes at Runtime
● A def is a true executable statement:
○ when it runs, it creates a new function object and assigns it to a name:
an assign statement (=)
a=3
○ a def can appear anywhere a statement can—even nested in other statements

if test:
def func(): … # Define func this way
else:
def func(): … # Or else this way
func() # Call the version selected and built

● The function is objects, it can be assigned to a different name and called by the new name
othername = func # Assign function object
othername() # Call func again

7
CHAPTER 16: A First Example
Definition Calls
● Create and assign function ● Work according to what we pass
into it: polymorphism
● Body executed when called
>>> times(2, 4)
def times(x, y): 8
return x * y >>> x = times(3.14, 4)
>>> x
12.56

>>> times('Ni', 4)
'NiNiNiNi'

8
CHAPTER 16: A Second Example
Intersecting Sequences Calls
● polymorphism
Definition

def intersect(seq1, seq2): >>> x = intersect([1, 2, 3], (1, 4))

res = [] >>> x
for x in seq1: [1]
if x in seq2:
res.append(x)
return res

Rewrite using list comprehension ???

res=[x for x in seq1 if x in seq2]

9
CHAPTER 17: SCOPES

10
CHAPTER 17: Python Scope Basics
● Namespace: a place where names (variables) are looked up
● The scope of visibility: the location of the assignment of a name
● Functions define a local scope and modules define a global scope with the
following properties:
○ Assigned names are local unless declared global or nonlocal.
○ Each call to a function creates a new local scope.

11
 CHAPTER 17: Python Scope Basics
The LEGB scope lookup rule:
1. The search order:
Local -> Enclosing (nested)
-> Global -> Built-in
2. The first occurrence wins.

12
CHAPTER 17: Scope Example
In a module file:
● X and func assigned in module: global # Global scope
● Y and Z assigned in function: locals X = 99
● X is a global
def func(Y):
# Local scope
Z=X+Y
return Z

func(1)
# func in module: result=100

13
CHAPTER 17: The Built-in Scope
The built-in scope is just a built-in module
called builtins
>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError',
'AttributeError', 'BaseException',
'BlockingIOError', 'BrokenPipeError',
'BufferError', 'BytesWarning', ...many
more names omitted...
'ord', 'pow', 'print', 'property', 'quit',
'range', 'repr', 'reversed', 'round',
'set', 'setattr', 'slice', 'sorted',
'staticmethod', 'str', 'sum', 'super',
'tuple', 'type', 'vars', 'zip']
14
CHAPTER 17: The global Statement
● Global names are variables X = 88 y, z = 1, 2
assigned at the top level of the
enclosing module file.
def func(): def all_global():
● Global names must be declared
if they are assigned within a global X global x
function. X = 99 x=y+z
● Global names may be
referenced within a function func() all_global()
without being declared.
print(X) # 99 print(x) # 3

15
CHAPTER 17: The global Statement
Program Design: Minimize Global X = 99
Variables def func1():
global X
Local variables disappear when the X = 88
function returns, but globals do not.
def func2():
Global variables can retain shared
global X
state information.
X = 77

Don’t overuse global variables since What is value of X?

it can make hard to debug, or to Depending on the function call order ->
understand, reuse track all the program -> hard to
manage
16
CHAPTER 17: The global Statement
Program Design: Minimize Cross-File # first.py
X = 99 # doesn’t know about second.py
Changes
# second.py
import first
Cross-File Changes
print(first.X)
first.X = 88 # too implicit to change X
In some cases, if need cross-file
changes, provide accessor function
# first.py
X = 99
def setX(new): # Accessor make external changes explicit
global X
X = new

# second.py
import first
first.setX(88) # Call the function instead of changing directly
17
CHAPTER 17: Scopes and Nested Functions
LEGB lookup rule: X = 99
E as enclosing (nested) scopes
def f1():
X = 88
def f2():
print(X)
f2()

f1() # 88

18
CHAPTER 17: Scopes and Nested Functions
Factory Functions: Closures
A simple function factory: a function that creates and returns another function.

def maker(N):
def action(X): # Make and return action
return X ** N # action retains N from enclosing scope return action
return action

f = maker(2)
print(f(4)) # 16

g = maker(3)
print(g(4)) # 64
19
CHAPTER 17: The nonlocal Statement in 3.X
def f1(): def f1():
x = 88 x = 88

def f2(): def f2():

print(x) # x readable print(x) # local variable 'X' referenced before assignment
x = 10
f2()
f2()
f1()
f1()

20
CHAPTER 17: The nonlocal Statement in 3.X
def f1():
x = 88 ● only inside a function
def f2():
nonlocal x def func():
print(x) nonlocal name1, name2, …
x = 10 # writable with nonlocal

f2() ● allow names to be assigned

print('x after f2 ', x) ● names already exist
f1() # ???

21
CHAPTER 18: Arguments

22
CHAPTER 18: Argument-Passing Basics (1)
Similar to the argument-passing model of the C language
● Immutable arguments are effectively passed “by value.”
● Mutable arguments are effectively passed “by pointer.”

1. Arguments and Shared References

2. Avoiding Mutable Argument Changes
3. Simulating Output Parameters and Multiple Results

23
CHAPTER 18: Arguments and Shared References (1.1)
>>> def changer(a, b):
a= 2
b[0] = 'spam'

>>> X = 1
>>> L = [1, 2]
>>> changer(X, L)
>>> X, L
(1, ['spam', 2])

24
CHAPTER 18: Avoiding Mutable Argument Changes (1.2)

>>> def changer(a, b): >>> changer(X, L[:])

a= 2
b[0] = 'spam' >>> changer(X, tuple(L))

>>> X = 1 def changer(a, b):

>>> L = [1, 2] a=2
>>> changer(X, L) b = b[:]
>>> X, L b[0] = 'spam'
(1, ['spam', 2])

25
CHAPTER 18: Simulating Output Parameters
and Multiple Results (1.3)
>>> def multiple(x, y):
x=2
y = [3, 4]
return x, y

>>> X = 1
>>> L = [1, 2]
>>> X, L = multiple(X, L)
>>> X, L
(2, [3, 4])

26
CHAPTER 18: Argument Matching Basics (2)

27
CHAPTER 18: Argument Matching Basics (2)
Detailed steps of argument matching:
1. Assign nonkeyword arguments by position.
2. Assign keyword arguments by matching names.
3. Assign extra nonkeyword arguments to *name tuple.
4. Assign extra keyword arguments to **name dictionary.
5. Assign default values to unassigned arguments in header.

28
CHAPTER 18: Keyword and Default Examples
Keywords

>>> def f(a, b, c):

print(a, b, c)

>>> f(1, 2, 3)

# a gets 1 by position, b and c passed by name

>>> f(1, c=3, b=2)

29
CHAPTER 18: Keyword and Default Examples
Default

>>> def f(a, b=2, c=3): # a required, b and c optional

print(a, b, c)

>>> f(1) # Use defaults 12 3

>>> f(a=1)
123

>>> f(1, 4)
143

>>> f(1, 4, 5)
145

>>> f(1, c=6)

30
CHAPTER 18: Keyword and Default Examples
Combining keywords and defaults:

def func(spam, eggs, toast=0, ham=0):

print((spam, eggs, toast, ham))

func(1, 2) # Output: (1, 2, 0, 0)

func(1, ham=1, eggs=0) # Output: (1, 0, 0, 1)

func(spam=1, eggs=0) # Output: (1, 0, 0, 0)

func(toast=1, eggs=2, spam=3) # Output: (3, 2, 1, 0)

func(1, 2, 3, 4) # Output: (1, 2, 3, 4)

31
CHAPTER 18: Arbitrary Arguments Examples
● Collect all the positional arguments >>> def f(*args):
into a new tuple and assigns the print(args)
variable args to that tuple >>> f()
()
>>> f(1)
(1,)
>>> f(1, 2, 3, 4)
(1, 2, 3, 4)

32
CHAPTER 18: Arbitrary Arguments Examples
● ** feature only works for >>> def f(**args): print(args)
keyword arguments >>> f()
● it collects them into a new {}
dictionary >>> f(a=1, b=2)
{'a': 1, 'b': 2}

33
CHAPTER 18: Arbitrary Arguments Examples
>>> def f(a, *pargs, **kargs):
print(a, pargs, kargs)

>>> f(1, 2, 3, x=1, y=2)

1 (2, 3) {'y': 2, 'x': 1}

34
 CHAPTER 19: Advanced Function Topics

1. Function Design Concepts

2. Recursive Functions
3. Function Objects: Attributes and Annotations
4. Anonymous Functions: lambda
5. Functional Programming Tools

35
CHAPTER 19: Function Design Concepts
Few general guidelines on coupling (how functions communicate):
● Coupling: use arguments for inputs and return for outputs.
● Coupling: use global variables only when truly necessary.
● Coupling: don’t change mutable arguments unless the caller expects it.
● Cohesion: each function should have a single, unified purpose.
● Size: each function should be relatively small.
● Coupling: avoid changing variables in another module file directly.

Good function designers prefer to use only arguments for inputs and return
statements for outputs, whenever possible.

36
CHAPTER 19: Function Design Concepts (1.1)

Figure 19-1. Function execution environment. 37

CHAPTER 19: Recursive Functions
● Recursive functions: functions
that call themselves in order to
loop
>>> def mysum(L):
print(L) # Trace recursive levels
if not L: # L shorter at each level
return 0
else:
return L[0] + mysum(L[1:])

>>> mysum([1, 2, 3, 4, 5])

[1, 2, 3, 4, 5]
[2, 3, 4, 5]
[3, 4, 5]
[4, 5]
[5]
[]
15

38
CHAPTER 19: Recursive Functions
(1) Which ones can work on
def mysum(L): list, string, file? Why?
return 0 if not L else L[0] + mysum(L[1:])
list(1, 2, 3)
(2) string(2,3)
def mysum(L):
file(3)
return L[0] if len(L) == 1 else L[0] + mysum(L[1:])

(3)
def mysum(L):
first, *rest = L # L = [1,2,3,4,5]
return first if not rest else first + mysum(rest)

39
 CHAPTER 19: Loop Statements Versus Recursion
>>> def mysum(L): while:
print(L) # Trace recursive levels >>> L = [1, 2, 3, 4, 5]
if not L: # L shorter at each level >>> sum = 0
return 0 >>> while L:
else: sum += L[0]
return L[0] + mysum(L[1:]) L = L[1:]
>>> sum

for
>>> L = [1, 2, 3, 4, 5]
>>> sum = 0
>>> for x in L: sum += x
>>> sum 15
40
CHAPTER 19:
Function Objects: Attributes and Annotations
Function Introspection

>>> def func(a):

b = 'spam'
return b * a
>>> func(8)
'spamspamspamspamspamspamspamspam'

>>> func.__name__
'func'

>>> dir(func)
['__annotations__', '__call__', '__class__', …]

41
CHAPTER 19:
Function Objects: Attributes and Annotations
Function Attributes

>>> func.count = 0
>>> func.count += 1
>>> func.count

42
CHAPTER 19: Anonymous Functions: lambda
lambda argument1, argument2,... argumentN : expression using arguments

Some properties of lambda expressions.

● lambda is an expression, not a statement.
● lambda’s body is a single expression, not a block of statements.

>>> def f(x, y, z): return x + y + z

>>> f(2, 3, 4)
9

>>> f = lambda x, y, z: x + y + z
>>> f(2, 3, 4)
9
43
CHAPTER 19: Anonymous Functions: lambda
lambda allows to embed a def f1(x): return x ** 2
function’s definition within the code def f2(x): return x ** 3
that uses it. def f3(x): return x ** 4

L = [ lambda x: x ** 2, L = [f1, f2, f3]

lambda x: x ** 3, for f in L:
lambda x: x ** 4] print(f(2))
for f in L: print(L[0](3))
print(f(2))
print(L[0](3))

44
CHAPTER 19: Anonymous Functions: lambda
Multiway branch switches: The finale

>>> key = 'got'

>>> {'already': (lambda: 2 + 2),
'got': (lambda: 2 * 4),
'one': (lambda: 2 ** 6)}[key]()
8
>>> def f1(): return 2 + 2
>>> def f2(): return 2 * 4
>>> def f3(): return 2 ** 6
>>> key = 'one'
>>> {'already': f1, 'got': f2, 'one': f3}[key]()
64
45
CHAPTER 19: Anonymous Functions: lambda
>>> def action(x):
return (lambda y: x + y) # Make and return function, remember x
>>> act = action(99)
>>> act
<function action.<locals>.<lambda> at 0x00000000029CA2F0>
>>> act(2) # Call what action returned
101

>>> action = (lambda x: (lambda y: x + y)) # nested

>>> act = action(99)
>>> act(3)
102
>>> ((lambda x: (lambda y: x + y))(99))(4)
103
46
CHAPTER 19: Functional Programming Tools
Functional programming tools apply functions to sequences and other iterables.
Mapping Functions over Iterables: map
- apply a passed-in function to each item in an iterable object
- return a list containing all the function call results.

47
CHAPTER 19: Functional Programming Tools
>>> counters = [1, 2, 3, 4] >>> def inc(x): return x + 10
>>> updated = [] >>> list(map(inc, counters))
>>> for x in counters: [11, 12, 13, 14]
updated.append(x + 10)
>>> updated >>> list(map((lambda x: x + 10),
[11, 12, 13, 14] counters))

List comprehension ???

updated=[x+10 for x in counters]

48
CHAPTER 19: Functional Programming Tools
● With multiple sequences, map expects an N-argument function for N
sequences.

>>> pow(3, 4)
81
>>> list(map(pow, [1, 2, 3], [2, 3, 4])) # 1**2, 2**3, 3**4
[1, 8, 81]

49
CHAPTER 19: Functional Programming Tools
● Selecting Items in Iterables: filter def myfunc(x):
>>> res = []
>>> for x in range(−5, 5): if x > 0: return x
if x > 0:
res.append(x)
>>> res
[1, 2, 3, 4]
print(list(filter(myfunc, range(-5, 5))))
List comprehension ???
res =[x for x in range(-5,5) if x>0]

list(filter((lambda x: x > 0), range(-5,

5)))

50
CHAPTER 19: Functional Programming Tools
Combining Items in Iterables: reduce

>>> L = [1,2,3,4]
>>> res = L[0]
>>> for x in L[1:]:
res = res + x
>>> res
10

>>> from functools import reduce

>>> reduce((lambda x, y: x + y), [1, 2, 3, 4])
10

51
CHAPTER 20: Comprehensions and Generations

1. List Comprehensions and Functional Tools

2. Generator Functions and Expressions
3. Comprehension Syntax Summary

52
CHAPTER 20: List Comprehensions and Functional Tools

List Comprehensions Versus map

● map maps a function over an iterable
● list comprehensions map an expression over a sequence or other iterable

>>> res = []
>>> for x in 'spam':
res.append(ord(x))
>>> res
[115, 112, 97, 109]

>>> res = list(map(ord, 'spam')) # Apply function to sequence (or other)

>>> res
[115, 112, 97, 109]

>>> res = [ord(x) for x in 'spam'] # Apply expression to sequence (or other)
>>> res
[115, 112, 97, 109]
53
CHAPTER 20: List Comprehensions and Functional Tools

Adding Tests and Nested Loops: filter

>>> res = []
>>> for x in range(5):
if x % 2 == 0:
res.append(x)
>>> res
[0, 2, 4]
>>> [x for x in range(5) if x % 2 == 0]
[0, 2, 4]
>>> list(filter((lambda x: x % 2 == 0), range(5)))
[0, 2, 4]

54
CHAPTER 20: List Comprehensions and Functional Tools

Formal comprehension syntax

[ expression for target1 in iterable1 if condition1
for target2 in iterable2 if condition2 Advice:
…
for targetN in iterableN if conditionN ] Keep It Simple
>>> res = []
>>> for x in range(5):
if x % 2 == 0:
for y in range(5):
if y % 2 == 1:
res.append((x, y))
>>> res
[(0, 1), (0, 3), (2, 1), (2, 3), (4, 1), (4, 3)]

>>> [(x, y) for x in range(5) if x % 2 == 0 for y in range(5) if y % 2 == 1]

[(0, 1), (0, 3), (2, 1), (2, 3), (4, 1), (4, 3)]

55
CHAPTER 20: Generator Functions and Expressions
Generator Functions
● Generator functions: a special kind of function that return a generator.
○ A generator is an object t supports the iteration protocol to yield one piece of the result list at
a time instead of building the result list in memory
○ Generators can be better: memory use, performance in larger programs
● A generator function yields a value, a normal function returns one

56
CHAPTER 20: Generator Functions and Expressions

>>> def gensquares(N):

for i in range(N):
yield i ** 2

>>> for i in gensquares(5):

print(i, end=' : ')
0 : 1 : 4 : 9 : 16 :

>>> x = gensquares(4)
>>> x
<generator object gensquares at 0x000000000292CA68>
>>> next(x)

57
CHAPTER 20: Generator Functions and Expressions

● Generator expressions are a memory-space optimization

○ they do not require the entire result list to be constructed all at once, as the square- bracketed
list comprehension does.

>>> [x ** 2 for x in range(4)] # List comprehension: build a list

[0, 1, 4, 9]
>>> (x ** 2 for x in range(4)) # Generator expression: make an iterable
<generator object <genexpr> at 0x00000000029A8288>

58
CHAPTER 20: Generator Functions and Expressions
Generator expressions versus map

>>> list(map(abs, (−1, −2, 3, 4)))

[1, 2, 3, 4]

>>> list(abs(x) for x in (−1, −2, 3, 4))

[1, 2, 3, 4]

Same results but generator expression, map don’t make temporary lists
>>> [x * 2 for x in [abs(x) for x in (−1, −2, 3, 4)]] # Nested comprehensions
[2, 4, 6, 8]

>>> list(map(lambda x: x * 2, map(abs, (−1, −2, 3, 4)))) # Nested maps

[2, 4, 6, 8]

>>> list(x * 2 for x in (abs(x) for x in (−1, −2, 3, 4))) # Nested generators
[2, 4, 6, 8]
59
CHAPTER 20: Generator Functions and Expressions

Generator expressions versus filter

>>> line = 'aa bbb c'

>>> ''.join(x for x in line.split() if len(x) > 1)
'aabbb'
>>> ''.join(filter(lambda x: len(x) > 1, line.split()))
'aabbb'

60
CHAPTER 20: Generator Functions Versus Generator Expressions

● Generator function
○ A function def statement contains a yield statement
○ It returns a new generator object when called
● Generator expressions
○ A comprehension expression enclosed in parentheses ()
○ It returns a new generator object when run

>>> G = (c * 4 for c in 'SPAM')

>>> list(G)
['SSSS', 'PPPP', 'AAAA', 'MMMM']

>>> def timesfour(S):

for c in S:
yield c * 4
>>> G = timesfour('spam')
>>> list(G)
['ssss', 'pppp', 'aaaa', 'mmmm']
61
CHAPTER 20: Generators Are Single-Iteration Objects

● Generator functions and >>> G = (c * 4 for c in 'SPAM')

expressions are their own >>> I1 = iter(G)
iterators >>> next(I1)
● They support just one active 'SSSS'
iteration >>> next(I1)
○ can’t have multiple iterators of 'PPPP'
either positioned at different >>> I2 = iter(G)
locations in the set of results. >>> next(I2)
'AAAA'
>>> list(I1)
['MMMM']
>>> next(I2)
StopIteration
62
CHAPTER 20: Generators Are Single-Iteration Objects

Generation in Built-in Types, Tools, and Classes

● dictionaries are iterables with iterators that produce keys on each iteration
>>> D = {'a':1, 'b':2, 'c':3}
>>> x = iter(D)
>>> next(x)
'c'

>>> for key in D: print(key, D[key])

● file iterators, python simply loads lines from the file

>>> for line in open('temp.txt'): print(line, end='')

63
CHAPTER 20: Generators Are Single-Iteration Objects

Generators and library tools: Directory walkers

>>> import os
>>> for (root, subs, files) in os.walk('.'): # Directory walk generator
for name in files:
if name.startswith('call'):
print(root, name)

Advance: Explicit is better than implicit.

64
CHAPTER 20: Generators Are Single-Iteration Objects

Example: Generating Scrambled (1) List comprehension ???

Sequences (2) Generator function ???
(3) Generator expressions ???
Scrambling sequences

>>> def scramble(seq):

res = []
for i in range(len(seq)):
res.append(seq[i:] + seq[:i])
return res
>>> scramble('spam')
['spam', 'pams', 'amsp', 'mspa']
65
CHAPTER 20: Comprehension Syntax Summary
List, set and dictionary comprehensions

● sets:
○ {1, 3, 2} is equivalent to set([1, 3, 2])
○ {f(x) for x in S if P(x)} : set comprehension
○ set(f(x) for x in S if P(x)) : generator expression
● dictionaries:
○ dict(zip(keys, vals))
○ {key: val for (key, val) in zip(keys, vals)}: dict comprehension
○ {x: f(x) for x in items} is like dict((x, f(x)) for x in items)

66
CHAPTER 20: Comprehension Syntax Summary
Scopes and Comprehension
Variables
● temporary loop variable names
in generator, set, dictionary,
and list comprehensions are
local to the expression
>>> (X for X in range(5))
<generator object <genexpr> at
0x00000000028E4798>
>>> X
NameError: name 'X' is not defined
>>> [x * x for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> (x * x for x in range(10))
<generator object at 0x009E7328>
>>> {x * x for x in range(10)}
{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
>>> {x: x * x for x in range(10)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6:
36, 7: 49, 8: 64, 9: 81}
67
 CHAPTER 21: The Benchmarking Interlude

Timing Iteration Alternatives

Timing Iterations and Pythons with timeit

68
CHAPTER 21: The Benchmarking Interlude
Timing Iteration Alternatives
● Timing Module: Homegrown (check file timer.py)
○ Homegrown timing tools for function calls.
○ Does total time, best-of time, and best-of-totals time
● Timing Script
○ Test the relative speed of iteration tool alternatives. (check file timeseqs.py)

69
CHAPTER 21: The Benchmarking Interlude
Timing Iterations and Pythons with timeit

>>> import timeit

>>> timeit.timeit(stmt='[x ** 2 for x in range(1000)]', number=1000)

>>> def testcase():

y = [x ** 2 for x in range(1000)] # Callable objects or code strings
>>> timeit.timeit(stmt=testcase, number=1000)

70
CHAPTER 21: The Benchmarking Interlude
Timing Iterations and Pythons with timeit

import pybench

pythons = [
(1, 'python3'),
(0, 'python'),
]
stmts = [
(0, 0, "[x ** 2 for x in range(1000)]"), # list comprehension
(0, 0, "res=[]\nfor x in range(1000): res.append(x ** 2)"), # for loop
(0, 0, "list(map(lambda x: x ** 2, range(1000)))"), # map
(0, 0, "list(x ** 2 for x in range(1000))"), # generation expression
]

pybench.runner(stmts, pythons, tracecmd=True)

71