# Problem Set 1

1
3.2

a) Take any three vectors x, y, z in Rl and two real number α, β ∈ R. Deﬁne the zero vector θ = (0, .., 0) ∈ Rl . To check that it is a vector space, deﬁne the sum of two vectors as the vector of the sum element by element; and the scalar multiplication as the multiplication of every element by an scalar. It is trivial to check that this operations are closed in a ﬁnite dimesional Rl . It is not very diﬃcult then to check conditions a to h of the deﬁnition of a real vector space in page 43 of SLP by using the element by element operations. For example for property c we have that α (x + y) = α (x1 + y1 , ..., xl + yl ) = (αx1 + αy1 , ..., αxl + αyl ) = αx + αy b) Straigthforward extension of part a) c) Deﬁne the addition of two sequences as the element by element addition, and scalar multiplication as the multiplication of every element by the same real number. Then proceed as in part a. d) Take f, g : [a, b] → R and α ∈ R. Let θ (x) = 0. Deﬁne the addition of functions by (f + g) (x) = f (x) + g (x), and scalar multiplication by (αf ) (x) = αf (x). A function f is continuous if xn → x implies that f (xn ) → f (x). To see that f + g is continuous, take a sequence xn → x in [a, b]. Then
xn →x

lim (f + g) (xn ) = lim f (xn ) + g (xn ) = f (x) + g (x) = (f + g) (x)
xn →x

Now you can proceed as in part c, checking that the properties are deﬁned for every point of the function. 1

y) = |x − y| = |y − x| = ρ (y. a) Take three diﬀerent integers x. y). Then 1 ∈ I but α1 6∈ I. f) Choose α ∈ (0. Notice also that x. when z = y. y) b) First. 0) + (0. f) The ﬁrst two properties come from the absolute value plus f (0) = 0. y}. z) + ρ(z.and z.3. b] → R+ . 1) is not in the unit circle. y) ≤ ρ(x. The ﬁrst two properties of the metric are immediate from the deﬁnition of absolute value.b] t∈[a. and z (t). z) = max |x (t) − z (t)| = max |x (t) − y (t) + y (t) − z (t)| t∈[a. z = x and when z 6∈ {x. To prove the triangle inequality let ρ (x. 2 . x). Violates the deﬁnition of vector space g) Let f : [a. notice that you have to consider three cases. z) t∈[a. Then (1. For the ﬁrst two cases the triangle inequality holds with equality. so does not belong to the set of nonnegative functions. 1).e) Take the vectors (0. then αf ≤ 0. y continuous in [a.b] t∈[a. y. It is also true that ρ (x. and with equality only when x = y. Also ρ (x. y) = |x − y| ≤ |x − z| + |z − y| ≤ ρ (x. x) Finally ρ(x. b] implies that the functions are bounded.b] ≤ max {|x (t) − y (t)| + |y (t) − z (t)|} ≤ max |x (t) − y (t)| + max |y (t) − z (t)| = ρ (x. The proposed metric is then real valued (not extended real). The non-negative property holds trivially for the absolute value. y) < 2 for all x. ρ (x. For the last one it holds with inequality ρ (x. y (t). z) + ρ (z. 0). y) ≥ 0. y) = ρ (y. Finally to show that ρ (x. 3.b] t∈[a. y) + ρ (y. Take α < 0. 1) and (1. c) Take three functions x (t) . e) Similar to a. and f is strictly increasing (there is only one zero). y.b] d) Very similar to c.

a) The norm is non-negative comes from the deﬁnition of the square root of sum of squares. The second one. z) + ρ (z. For the triangle inequality we have ||x + y|| = sup (|xk + yk |) ≤ sup (|xk | + |yk |) ≤ sup |xk | + sup |yk | k k k k = ||x|| + ||y|| 3 . The second property we have ||αx|| = supk |αxk | = |α| supk |xk | = |a| ||x||. let ρ (x. d) The proposed norm is non-negative and is real valued (bounded sequences). xk = 0.4. y) 3.c) are very similar exploiting the absolute value properties. is implied by X X x2 = α2 ||x|| (αxi )2 = α2 ||αx||2 = i To prove the triangle inequality we follow : ||x + y||2 = X (xi + yi )2 = X X 2 xi x2 + 2 i xi yi + By Cauchy-Schwartz we have that X x2 + 2 i X xi yi + X 2 yi ≤ = ||x||2 + 2 ||x|| ||y|| + ||y||2 = (||x|| + ||y||)2 X 2 xi + 2 P P P 2 1/2 xi yi ≤ [( x2 ) ( yi )] so i h³X ´ ³X 2 yi X 2 yi ´i1/2 + X 2 yi This implies that ||x + y|| ≤ ||x|| + ||y||. y) = f (|x − y|) = f (|x − z + z − y|) By strictlt increasing f we have f (|x − z + z − y|) ≤ f (|x − z| + |z − y|) And by concavity we have f (|x − z| + |z − y|) ≤ f (|x − z|) + f (|z − y|) = ρ (x. b). The ﬁrst property. notice that ||x|| = 0 only if for all k. ||αx|| = |α| ||x||.To see the triangle inequality.

and construct a sequence of {xki } that is always bounded away from x by ε. for any ε. This is ρ (xn . 0) . n2 > N . Let N2 > k1 and can ﬁnd a k2 such that |xk2 − x| > ε and so on . a) The metric in 3. then there is a subsequence that does not converge to x.6. such that |xn − xm | < ε = 1. Compute the P i 2 (xn − xi ) . then for any ε. and for any N1 we can ﬁnd an k1 > N such that |xk1 − x| > ε. y) = 0 and x = y.. each sequence ¡ ¢ of kth elements converges to some xk . notice that if {xn } is a Cauchy sequence under the norm. Deﬁne x = x1 . Where x is the limit of the sequence. x) + ρ (xn2 . (which is the Cauchy deﬁnition). Then ρ (xn1 . and xn → y. x) < ε/2 for n1 > N . . ρ (xn . b) If xn → x. This implies that xn = xm = x. xn2 ) ≤ ρ (xn1 . y) given that xn → x. We can construct such a sequence by showing that if {xn } does not converge. To show that if every subsequence converges to x. xk . d) The fact that xn → x implies that every subsequence converges is easy. 3. then {xn } converges to x we need to show that if xn does not converge to x. 0) = ε + ρ (xN . M } < ∞ for all n so the sequence is bounded. 0). 3. a) By the triangle inequality we know that ρ (x. c) Pick an ε. The 3.5.. 0) ≤ ρ (xn1 . then there exists an N such that for n1 > N we have that ρ (xn1 . then each of the kth elements of xn are also a Cauchy sequence. Just choose ε < 1.3b metric is similar. implying that ρ (x. the right hand side can be made as small as wanted. we can ﬁnd N such that ρ (xn1 . xN ) + ρ (xN .e) Proceed as in part d.4a is complete. xn2 ) < ε. x) = ε for all n1 . x)2 = 4 . x) + ρ (xn .3a is complete. 0)}. y) ≤ ρ (xn . 0) ≤ max{ε + ρ (xN . So we have then that for all n.. Let M = maxn<N {ρ (xn . Given that the real line is complete. The distance between to xn1 and xn2 with n2 > N is then ρ (xn1 . To show that. The metric in 3. Given that every distance from xn to x. f) Proceed as in part d.

any convergent sequence in S 0 converges in S 0 . xm ) → 0. For 3. we have to show that the limit function is non-decreasing. x) ≤ to k k ρ (xn . it has to be the case that fn (t) − fn (t0 ) > ε. .f.4. v = ¢ ¢ ¡ ¡ = ρ T n v0 . Just analyze the limit of for example xn (t) = 1 + an t as an → 0 with an > 0. The proof was done in class for more general space. but the limit of xn (t) is discountinuos at t = 0.}. Let n x = {x1 . In this case.. T n+1 v0 + βρ (T n v0 . The fact©that n ª a sequence of sequences is Cauchy (under ª the norm) implies that xk n © is Cauchy as well. This limit is just xn (t) = 1.3. In 3. Now. we m can make the left hand side of the inequality as small as possible. just think of the counter example for a = 0. that the Cauchy sequence of {fn } converges to f . x) → x. ρ (xn . Given that any Cauchy sequence in S converges in S (by completeness). then ρ (xn . ©It ¡is easy¢ª see that ρ (xn .c replacing with ”non-decreasing” we can construct the limit function point by point as before. For the case of 3.e. This implies that xk converges to some xk . to do that suppose that is not.4d. xm ) + supk ρ xm . which is not strictly increasing.. xm ) + ρ (xm . x2 .4c is similar. x) ≤ ρ (xn . T n+1 v0 + ρ T n+1 v0 . v) ≤ ρ T n v0 . implying that ρ (xn . The 3. See that ¡ ¢ ¡ ¢ ρ (T n v0 . Then 0 < ε < f (t) − f (t0 ) = f (t) − fn (t) + [fn (t) − fn (t0 )] + fn (t0 ) − f (t0 ) ≤ 2 ||f − fn || + fn (t) − fn (t0 ) Given that f → fn . b) Since S 0 is closed in S.. T n+1 v0 + ρ T n+1 v0 .3e is not complete: rational sequences can converge to irrational numbers. 3.4. T v) 5 .4b and 3. Given that xk → xk and {xn } is Cauchy.xi → xi .b = 1 and xn (t) = tn . 3. 3.3c is not complete. a contradiction of non-decreasing property of fn . but for some t0 > t we have that f (t) − f (t0 ) > ε.9. and hence converges in S0. Let xk be the kth element of the n sequence. T v ¡ ¢ ≤ ρ T n v0 . implies that any Cauchy sequence in S 0 converges. x)2 → 0 which proves the limit result. The metric in n 3. x .

the result is immediate. for every x0 ∈ X and every ε > 0 there exists an − ∈ Π (x0 ) x → such that ¡ ¢ ε x v (x0 ) ≤ un − + β n+1 v (xn+1 ) + → 2 ¡ ¢ ¡ ¢ ε ε x x v (x0 ) ≤ u − + lim sup β n+1 v (xn+1 ) + ≤ u − + → → 2 2 n→∞ ¡ ¢ Given that u − ≤ v∗ (x0 ) this gives us that x → v (x0 ) ≤ v ∗ (x0 ) + ε/2 and hence v (x0 ) ≤ v ∗ (x0 ) for all x0 .h. Take arbitrary sequences xn → x and yn ∈ Γ (xn ).3. But this is impossible. then for some N .p. Γ (x) is non-empty. we have that ynk − f (x) > ε for all nk > N . If v (x0 ) = ∞ the proof is along the lines of the last part of theorem 4. with f (x) = x. Hence f is l. a) Same as part b. Choose any y ∈ Γ (x) and consider the sequence xn → x. Since v satisﬁes the FE.3) a) Let v (x0 ) be ﬁnite. this implies that {xn } is bounded. as shown in the proof of Theorem 4. Given x. Given that {xn } converges.13. Let y = lim ynk . Then by continuity of f we have that limyn = γ lim f (xn ) = γf (x) = y. at x. Because any bounded sequence of real numbers has a convergent subsequence there exists a convergent subsequence {ynk }. So yn ∈ Γ (xn ). c) Proceed coordinate by coordinate as in b) 4. This implies that {yn } is bounded as well. because ynk ∈ Γ (xnk ).3 Taking the limit as n → ∞ this gives 6 . Let γ ≡ y/f (x) ≤ 1 and yn = γf (xn ).Rearranging term we get the inequility we need. 3. Since 0 ∈ Γ (x). Suppose no. b) Choose any x. If v (x0 ) = −∞.c. Γ (x) is compact valued. This implies then that ynk − f (xnk ) > 2ε for all nk > M > N . Now we need to show that lim ynk ≤ f (x).

Γ (x) is non-empty and ﬁnite-valued for all x implies that the optimal policy correspondence is non-empty. we have that v = v∗ . for all x ∈ X Hence T : B (X) → B(X). So. There is a unique ﬁxed point.3 we have that ¡ ¢ v (x0 ) ≥ un − + β n+1 v (xn+1 ) x → ¡ ¢ ¡ ¢ ¡ ¢ ¡ ¢ x x x v (x0 ) ≥ lim un −0 + lim β n v x0n+1 = un −0 ≥ u − → → → n→∞ n→∞ For all − ∈ Π (x0 ). Monotonicity of Thn implies that Thn wn ≤ 2 Thn (T wn ) = Thn wn . and the maximum is always attained. g with f ≤ g. y) + βf (y) Discounting: It is easy to show that T (f + a) (x) = (T f ) (x) + βa. T is a contraction. We show now that T is a contraction mapping.4) a) Let K be a bound on F and M be a bound on f. Iterating in this operator we have that 7 .Then (T f ) (x) ≤ K + βM . y ∗ ) + βf (y ∗ ) ≤ F (x. y ∗ ) + βg(y ∗ ) ≤ (T g) (x) where y ∗ ∈ arg max F (x.b) By the argument in theorem 4. 4. y) + βf (y) = F (x. b) Similar to part (a) c) Note that wn (x) = (Thn wn ) (x) ≤ max [F (x. This implies that x → v (x0 ) ≥ v∗ (x0 ) = x − ∈Π(x0 ) → and together with the result in part a). y) + βwn (y)] ¡ ¢ = (T wn ) (x) = Thn+1 wn (x) N T wn ≤ Thn+1 wn So we have that wn ≤ T wn . Monotonicity : Let f. Then (T f ) (x) = y∈Γ(x) ¡ ¢ sup u − x → max F (x.

||wn − v|| ≤ ||T wn−1 − v|| ≤ β ||wn−1 − v|| ≤ β ||T wn−2 − v|| ≤ β 2 ||wn−2 − v|| ≤ β n ||w0 − v|| and hence wn → v as n → ∞.N But wn+1 = limN →∞ Thn+1 wn .. ≤ T wn ≤ v By the contraction mapping.. Hence T wn ≤ wn+1 . And w0 ≤ T w0 ≤ w1 ≤ T w1 ≤ . 8 .