1. Foundations of Numerics from Advanced Mathematics
Calculus
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 1
TU M¨ unchen
1.1. Calculus
Functions Revisited
• notions of a function, its range, and its image
• graph of a function
• isolines and isosurfaces
• sums and products of functions
• composition of functions
• inverse of a function: when existing?
• simple properties: (strictly) monotonous
• explicit and implicit deﬁnition
• parametrized representations (curves, ...)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 2
TU M¨ unchen
Exercise Functions
Which of the following curves is a graph of a function f (x)?
x x x
Graphically determine the image of the function graph(s).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 3
TU M¨ unchen
Exercise Functions – Solution
Which of the following curves is a graph of a function f (x)?
x x x
Graphically determine the image of the function graph(s).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 4
TU M¨ unchen
Exercise Functions
Sketch the isolines of the function f : R
2
→R, (x, y) → x
2
+ y
2
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 5
TU M¨ unchen
Exercise Functions
Sketch the isolines of the function
f : R
2
→R
2
, (x, y) → x
2
+ y
2
.
x
y
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 6
TU M¨ unchen
Exercise Functions
Assume, we have the functions f , g : R →R with
f (x) = sin(x), g(x) = cos(x).
(f
2
+ g
2
)(x) =?
(f ◦ g)(x) = ?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 7
TU M¨ unchen
Exercise Functions – Solution
Assume, ew have the functions f , g : R →R with
f (x) = sin(x), g(x) = cos(x).
(f
2
+ g
2
)(x) = sin
2
(x) + cos
2
(x) = 1.
(f ◦ g)(x) = sin(cos(x)).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 8
TU M¨ unchen
Examples for Functions
Explicit function:
f : R →R, x → (1 − x)
2
+ e
x
Implicit function:
f : R
+
0
→R
+
0
, x → y with x
2
+ y
2
= 1
Parametrized function:
f : R →R, x → gy(gx
−1
(x)) with g : R →R
2
, g(t ) =
( gx(t ), gy(t ) ).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 9
TU M¨ unchen
Continuity
• remember the ε and the δ!
• deﬁnition of local (“in x
0
”) and global continuity (“∀x”)
• what about sums, products, quotients, ... of continuous functions?
• what about compositions of continuous functions?
• what about continuity of the inverse?
• intermediate value theorem
• continuous functions on compact sets – maximum and minimum value
• uniform continuity
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 10
TU M¨ unchen
Exercise Continuity
We have two continuous functions f : R →R and g : R →R. f
is continuous iff . . .
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 11
TU M¨ unchen
Exercise Continuity – Solution
We have two continuous functions f : R →R and g : R →R. f
is continuous iff
∀ > 0, x ∈ R ∃δ > 0 : f (x) − f (y) < ∀y : x − y < δ.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 12
TU M¨ unchen
Exercise Continuity
We have two continuous functions f : R →R and g : R →R.
Are f + g, f − g, f · g,
f
g
, and f ◦ g continuous?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 13
TU M¨ unchen
Exercise Continuity – Solution
We have two continuous functions f : R →R and g : R →R.
Are f + g, f − g, f · g,
f
g
, and f ◦ g continuous?
The continuity of all these functions can be shown easily using
f (x) ± g(x) − (f (y) ± g(y)) ≤ f (x) − f (y) + g(x) − g(y)
f (x) · g(x) − f (y) · g(y) = (f (x) − f (y))g(x) + f (y)(g(x) − g(y)) ≤
g(x)f (x) − f (y) + f (y)f (x) − f (y)
¸
¸
¸
1
g(x)
−
1
g(y)
¸
¸
¸ =
¸
¸
¸
g(y)−g(x)
g(x)·g(y)
¸
¸
¸
Proof the continuity of f ◦ g on your own! It’s easy, but a few lines to write.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 14
TU M¨ unchen
Limits
• meaning of ε → 0 and N → ∞ and x → ∞
• accumulation point of a set
• limit (value) of a set
• limits from the left or from the right, respectively: f (x+), f (x−)
• limits at inﬁnity: lim
x→∞
f (x)
• inﬁnite limits: f (x) → ∞
• how can discontinuities look like?
– jumps: f (x+) = f (x−)
– holes: f (x+) = f (x−) = f (x)
– second kind: f (x) = 0 in x = 0 and f (x) = sin
_
1
x
_
elsewhere
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 15
TU M¨ unchen
Exercise Limits
Determine the accumulation point of S =
1
n
; n ∈ N
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 16
TU M¨ unchen
Exercise Limits – Solution
Determine the accumulation point of
1
n
; n ∈ N
.
The accumulation point is 0 since for all > 0 there is a e ∈ S with e − 0 < .
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 17
TU M¨ unchen
Limits – Visualization
N → ∞:
x → ∞:
→ 0:
0
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 18
TU M¨ unchen
Limits – Visualization
Examples for lim
x→∞
f (x):
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 19
TU M¨ unchen
Limits – Visualization
Two examples for f (x) → ∞:
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 20
TU M¨ unchen
Limits – Visualization
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 21
TU M¨ unchen
Sequences
• deﬁnition of a sequence: a function f deﬁned on N
• if f (n) = a
n
, write
(a
n
) or a
1
, a
2
, a
3
, ...
• bounded / monotonously increasing / monotonously decreasing sequences
• notion of convergence of a sequence: existence of a limit for n → ∞
• Cauchy sequence
• subsequences
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 22
TU M¨ unchen
Sequences – Visualization
∀ > 0 ∃
¯
N ∈ N :
a
N
− a
M
 < ∀ N, M >
¯
N.
This is a Cauchy sequence!
This is NOT!
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 23
TU M¨ unchen
Series
• notion of an (inﬁnite) series
– elements of a series
– partial sums of a series
– convergence deﬁned by convergence of the sequence of the partial sums
– convergence and absolute convergence
• examples:
– geometric series:
∞
k=1
x
k
=
1
1−x
– harmonic series:
∞
k=1
1
k
= ∞
– alternating harmonic series:
∞
k=1
(−1)
k−1 1
k
= ln(2)
• criteria for convergence: quotient and root criterion
• power series:
∞
k=0
a
k
(z − a)
k
– coefﬁcients a
k
and centre point a
– radius of convergence R: absolute convergence for z − a < R
– identity theorem for power series
• rearrangement
• sums of series, nested series, products of series (Cauchy product)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 24
TU M¨ unchen
Series – Convergence Criteria
quotient criterion: limsup
n→∞
a
n+1

a
n

= q < 1 (convergence)
(Cauchy’s) root criterion:
limsup
n→∞
n
a
n
 = C
< 1 absolute conv.
> 1 divergence
= 1 (abs.) con/divvergence
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 25
TU M¨ unchen
Series – Identitity Theorem for Power Series
If the radii of convergence of the power series
∞
n=0
a
n
(z −z
0
)
n
and
∞
n=0
b
n
(z − z
0
)
n
are positive and the sums of the series
are equal in inﬁnitely many points which have z
0
as an
accumulation point, then the both series are identical, i.e.
a
n
= b
n
for each n = 0, 1, 2, . . ..
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 26
TU M¨ unchen
Differentiation
• ﬁrst step: functions f of one real variable, complex values allowed
• derivative or differential quotient of f :
– deﬁned via limit process of difference quotients
– write f
or
˙
f or
df
dx
– geometric meaning?
– local and global differentiability
– derivative from the left / from the right
• rules for the daily work:
– derivative of f + g, fg, and f /g?
– derivative of f (g) (chain rule)?
– derivative of the inverse function?
• higher derivatives f
(k)
(x); meaning
• notion of continuous differentiability
• smoothness of a function
• space of ktimes continuously differentiable functions: C
k
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 27
TU M¨ unchen
Visualization Differention
f(x+h)−f(x)
h
f(x)−f(x−h)
h
x
h 0
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 28
TU M¨ unchen
Exercise Differentiation Rules
(f + g)
= ?
(f · g)
= ?
(f ◦ g)
= ?
f
−1
= ?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 29
TU M¨ unchen
Exercise Differentiation Rules – Solution
(f + g)
= f
+ g
.
(f · g)
= f
· g + f · g
. ?
(f ◦ g)
= (f
◦ g) · g
.
f
−1
= (f
)
−1
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 30
TU M¨ unchen
Differential Calculus of one Real Variable
• notion of a global/local minimum/maximum
• local extrema and the ﬁrst derivative
• mean value theorem:
∃ξ ∈ (a, b) : f
(ξ) =
f (b) − f (a)
b − a
• monotonous behaviour and the ﬁrst derivative
• local extrema and the second derivative
• rule of de l’Hospital
• notions of convexity and concavity
• convexity/concavity and the second derivative
• notion of a turning point
• turning points and the second derivative
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 31
TU M¨ unchen
Visualization Mean Value Theorem
a b
f(b)
f(a)
ξ
a b
f(b)
f(a)
ξ ξ
1
2
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 32
TU M¨ unchen
Meaning of Derivatives
ﬁrst derivative second derivative
increasing > 0 –
decreasing < 0 –
maximum = 0 < 0
minimum = 0 > 0
convex – > 0
concave – < 0
turning point – = 0
con ex
conCAVE
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 33
TU M¨ unchen
Function Classes (1)
• polynomials
– deﬁnition, degree, sums and products, division with rest, identity theorem,
roots and their multiplicity
• rational functions
– poles and their multiplicity, partial fraction decomposition
• exponential function and logarithm
– characterising law of the exponential function:
exp(s + t ) = exp(s) · exp(t ) or y
= y
(functional equation of natural growth)
– series expansion of the exponential function, speed of growth
– natural logarithm as exp’s inverse:
y = exp(x) = e
x
, x = ln(y)
– functional equation: ln(xy) = ln(x) + ln(y)
– exponential function and logarithm for general basis a:
a
x
:= e
x ln a
, log
a
(y) :=
ln y
ln a
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 34
TU M¨ unchen
Function Classes (2)
• hyperbolic functions
– cosh(z), sinh(z), ...
• trigonometric functions
– sin(x), cos(x): solutions of y
(2)
+ y = 0
– geometric meaning?
– Euler’s formula: e
ix
= cos(x) + i · sin(x)
– derivatives, addition theorem
– periodicity
– series expansion
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 35
TU M¨ unchen
Integral Calculus of one Variable
• Riemann integral, upper and lower sums
• approximation by staircase functions
• properties:
– linearity
– monotonicity
• mean value theorem:
ξ ∈ (a, b) :
_
b
a
f (x)dx = (b − a) · f (ξ)
• main theorem of differential and integral calculus:
– deﬁne F(x) :=
_
x
a
f (t )dt
– then
_
b
a
f (t )dt = F(b) − F(a)
• rules for everyday work:
– partial integration:
_
uv
dx = uv −
_
vu
dx
– substitution:
_
b
a
f (t (x))t
(x)dx =
_
t (b)
t (a)
f (t )dt
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 36
TU M¨ unchen
Visualization RiemannIntegral
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 37
TU M¨ unchen
Local Approximation: Taylor Polynomials and Series
• local approximation of functions with polynomials
• generalization of the tangent approximation used for the deﬁnition of the derivative
• Taylor polynomials:
– let f be ntimes differentiable in a
– we look for a polynomial T with T
(k)
= f
(k)
for k = 0, 1, ..., n
– obviously:
T(x) :=
n
k=0
1
k!
f
(k)
(a)(x − a)
k
– unique, degree n, write T
n
f (x; a)
– remainder R
n+1
(x) := f (x) − T
n
f (x : a)
R
n+1
(x) =
f
(n+1)
(ξ)
(n + 1)!
(x − a)
n+1
• Taylor series:
– for inﬁnitely differentiable functions (exp, sin, cos, ...)
– sum up to ∞ instead of n only
– examples
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 38
TU M¨ unchen
Visualization Taylor Polynomials
T
(1)
T
(2)
a a
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 39
TU M¨ unchen
Global Approximation: Uniform Convergence
• convergence of sequences of functions f
n
deﬁned on D:
• pointwise: for each x ∈ D; then
f (x) := lim
n→∞
f
n
(x)
deﬁnes a function
– problems: are properties such as continuity or differentiability inherited from
the f
n
to f , and how to calculate derivatives or integrals of f ?
– i.e., can the order of limit processes be changed?
• therefore the notion of uniform convergence:
– deﬁnition: f
n
− f
D
→ 0 for n → ∞
– with that, the inheritance and changeorder problems from above are solved!
– criteria: Cauchy, ...
• approximation theorem of Weierstrass: each continuous function f on a
compact set can be arbitrarily well approximated with some polynomial
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 40
TU M¨ unchen
Simple Differential Equations
• notion of a differential equation
– ordinary: one variable
– partial: more than one variable (several spatial dimensions or space and
time)
• examples:
– growth:
˙
y = k · y or
˙
y = k(t , y) · y
– oscillation:
¨
y + y = 0 or similar
• example of an analytic solution strategy: separation of variables
y
= g(x) · h(y), y(x) = y
0
– formal separation:
dy
h(y)
= g(x)dx
– integration of the left and right side
– some requirements for applicability
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 41
TU M¨ unchen
Periodic Functions
• target now: periodic functions, period typically 2π
• trigonometric polynomials
• deﬁnition:
T(x) :=
n
k=−n
c
k
e
ikx
=
a
0
2
+
n
k=1
(a
k
cos(kx) + b
k
sin(kx))
(coefﬁcients c
k
, a
k
, and b
k
are unique)
• formula for the coefﬁcients:
c
k
=
1
2π
_
2π
0
T(x)e
−ikx
dx
• T is real iff all a
k
, b
k
are real iff c
k
= c
−k
• Weierstrass: 2πperiodic continuous functions can be arbitrarily well
approximated by trigonometric polynomials
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 42
TU M¨ unchen
Fourier Series
• consider vector space of 2πperiodic complex functions f on R
• Fourier coefﬁcients:
ˆ
f (k) :=
1
2π
_
2π
0
f (x)e
−ikx
dx
• Fourier polynomial: S
n
f (x) :=
n
k=−n
ˆ
f (k)e
ikx
• Fourier series:
∞
−∞
ˆ
f (k)e
ikx
• sinecosine representation of S
n
f :
S
n
f (x) =
a
0
2
+
n
k=1
(a
k
cos(kx) + b
k
sin(kx))
• coefﬁcients:
a
k
=
ˆ
f (k) +
ˆ
f (−k) =
1
π
_
π
−π
f (x) cos(kx)dx
b
k
= i (
ˆ
f (k) −
ˆ
f (−k)) =
1
π
_
π
−π
f (x) sin(kx)dx
• all a
k
vanish for odd f , all b
k
vanish for even f
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 43
TU M¨ unchen
Functions of Several Variables
• f now deﬁned on R
n
or a subset of it
• notion of differentiability: now via existence of a linear map, the differential
• directional derivatives
• partial derivatives
• prominent differentiability criterion: existence and continuity of all partial
derivatives
• the gradient of a scalar function f and its interpretation
• the Jacobian of a vectorvalued function f
• mean value theorem
• higher partial derivatives, Taylor approximation, Hessian
• local minima and maxima, criteria
• saddle points
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 44
TU M¨ unchen
Visualization Saddle Point
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 45
TU M¨ unchen
Integration over Domains
• a huge ﬁeld, from which we only mention a few results
• theorem of Fubini:
– shows that, in many cases, a multidimensional integration domain can be
tackled dimension by dimension
– statement (we neglect the requirements, for which more integration theory is
needed):
_
X×Y
f (x, y)d(x, y) =
_
Y
__
X
f (x, y)dx
_
dy =
_
X
__
Y
f (x, y)dy
_
dx
– related to Cavalieri’s principle
– will also be of relevance for numerical quadrature
• transformation theorem:
– a generalisation of integration by substitution
– statement, again without requirements:
_
U
f (T(x)) ·
_
_
det T
(x)
_
_
dx =
_
V
f (y)dy
– allows for a change of the coordinate system (polar coordinates), e.g.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 46
TU M¨ unchen
Gauss Theorem
• we further generalise integration, now allowing for integration over hypersurfaces
(a sphere, e.g.)
• this is important for the physical modelling in many scenarios (heat ﬂux through a
pot’s surface, ...)
• the famous Gauss theorem allows to combine integrals over volumes and
surfaces, which occurs in the derivation of many physical models (conservation
laws) and, hence, is of special relevance for CSE
• prerequisites:
– a vector ﬁeld: a vectorvalued function on R
n
(example: the velocity ﬁeld in
ﬂuid mechanics)
– the divergence of a vector ﬁeld F:
div F(x) =
n
i =1
∂
i
F
i
(x)
• ﬁnally the Gauss theorem:
– several regularity assumptions needed
_
G
div Fdx =
_
∂G
F
dS
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 47