This action might not be possible to undo. Are you sure you want to continue?

Welcome to Scribd! Start your free trial and access books, documents and more.Find out more

Kahan

Notes on Jensen’s Inequality for Math. H90

September 27, 2000 6:23 am

A Convex Region in a vector space is a region which, together with any two points in that region, includes all of the straight line segment joining them. For example, interiors of elipses and triangles and parallelograms are convex regions in the plane, but a star or annulus is not. The Convex Hull of a set { z1, z2, z3, ... } of given points or vectors is the smallest convex region that contains all of them. The convex hull can be shown to be the set of all … w1 z1 + w2 z2 + w3 z3 + … + wn zn Positively Weighted Averages -------------------------------------------------------------------------------w1 + w2 + w3 + … + wn of finite subsets, here { z1, z2, z3, ..., zn } , of those points with positive Weights w1 > 0 , w2 > 0 , w3 > 0 , ..., wn > 0 . In fact, Caratheodory’s theorem says that the convex hull is the union of all simplices whose vertices are chosen from the given point set, and every such simplex can easily be shown to consist of the set of positively weighted averages of its vertices. In the plane simplices are triangles; in 3-space simplices are tetrahedra; in N-space a simplex is the convex hull of N+1 points not lying in a (hyper)plane of dimension less than N . Usually n > N . Another interpretation of the foregoing positively weighted average is as the Center of Mass or Center of Gravity of a collection of positive masses w1, w2, w3, ..., wn positioned respectively at the points z1, z2, z3, ..., zn . Note that the center of mass of positive masses lies strictly inside their convex hull when n > 2 , and strictly between them when n = 2 . All the foregoing assertions above about convex regions seem too obvious for plane regions to need explicit proofs here, but proofs for convex regions in spaces of arbitrarily high dimensions may be unobvious. (See texts by R.V. Benson, H.G. Egglestone, S.R. Lay, F.A. Valentine, … .) A real function U(x) of the vector argument x is called a Convex Function when its graph is the lower boundary of a convex region. Consequently every Secant ( a straight line that crosses a graph at least twice ) lies above the graph of U(x) between every two crossings, so there can be only two crossings. Moreover the graph of U(x) lies above its every Support-Line, which is a straight line, like a tangent, that touches the graph somewhere without crossing it nor running above it. The letters U and V without serifs are examples of graphs of convex functions; the letter V has no tangent at its sharp bottom vertex, but it has infinitely many support-lines there. A round bowl, a satellite-dish, and an ice-cream cone can also be examples of convex graphs. A convex function’s domain must be convex; do you see why? Every convex function must be continuous in the interior of its domain (this isn’t obvious), and can be proved to be differentiable almost everywhere, and one-sidedly differentiable once everywhere inside its domain, but not necessarily twice differentiable anywhere; however, the second derivative must be nonnegative definite wherever it exists. For instance, if x is a scalar this means U"(x) ≥ 0 .

What this means for vector arguments x is best explained with the aid of a Taylor Series: U(z+h) = U(z) + U'(z)·h + (U"(z) + Ø(z, h))·h·h/2 , where Ø(z, h) → 0 as h → o , wherever the second derivative U"(z) exists. Here U" is a bilinear operator: U"·p·b is a real-valued functional linear in each of the vectors p and b separately. If we think of U(x) as a function of the components ξ1, ξ2, ξ3, … of x , then U"(x)·p·b = ∑j ∑k (∂2U(x)/∂ξj∂ξk) πj ßk = pTHb where π1, π2, π3, … and ß1, ß2, ß3, … are the components of p and of b respectively, and H is the Hessian matrix of second partial derivatives of U . A wellknown theorem due to H.A. Schwarz asserts that ∂2U(x)/∂ξj∂ξk = ∂2U(x)/∂ξk∂ξj provided either side exists and is continuous, as is normally the case; then H = HT is symmetric and U"(x)·p·b = U"(x)·b·p . The convexity of U implies that U(x) ≥ U(z) + U'(z)·(x–z) for every x in the domain of U , which says the graph of U(x) lies above a (hyper)plane tangent at z ; substituting this inequality into the Taylor series implies that (x–z)TH(x–z) ≥ 0 for all such x , especially those very near z , which implies that H and thus U"(z) is nonnegative definite as asserted earlier. Conversely, if U"(z) is nonnegative definite at every z inside its domain then U(x) is convex, though to prove this we must replace U" + Ø by a weighted average of U" on the segment from x to z .

Notes issued 25-27 Sept. 2000

Page 1/3

.. yk) for k = 0. ... Replacing a convex function U(x) by a concave function C(x) merely reverses the inequality after y0 .. as the lower boundary of its convex hull.Prof. zn . In particular. also falls below or on the boundary of the convex hull of the points z1. xn . ….. xn } and G a Weighted Geometric Mean with the same set of weights { w1.. ….+ -----. all these points lie on the graph of U(x) which. . . 2000 Page 2/3 . Theorem: H ≤ G ≤ A . n . x2. and then C"(x) must be nonpositive definite wherever C"(x) exists. . and H is a Weighted Harmonic Mean with the same weights. ÿ) = -------------------------------------------------------------------------------. An important one is the Inequality among the Arithmetic. To prove H ≤ G apply the inequality of the arithmetic and geometric means to the reciprocals of x1. Then the weighted average w1 x1 + w2 x2 + w3 x3 + … + wn xn x0 := --------------------------------------------------------------------------------w1 + w2 + w3 + … + wn lies in the domain of U also.. so z0 lies directly below or on the positively weighted average w1 z1 + w2 z2 + w3 z3 + … + wn zn (x0. wn } . 1.. Jensen’s Inequality w1 y1 + w2 y2 + w3 y3 + … + wn yn is this Theorem: y0 ≤ ÿ := --------------------------------------------------------------------------------w1 + w2 + w3 + … + wn Its proof goes roughly as follows: Let zk = (xk. can you see why? (It takes a while.. . w1 + w2 + w3 + … + wn G := ( x 1 w1 ⋅ x2 w2 ⋅ x3 w3 ⋅ … ⋅ zn ) wn 1 ⁄ ( w1 + w2 + w3 + … + wn) ... Now set yk := U(xk) for k = 0...... H90 September 27. W. w1 w2 w3 wn -----. . x2.) Jensen’s Inequality has many applications. z2.. with equality only when x1 = x2 = .+ -----... .. wn and n positive numbers x1.. . w2. x2. = xn . z0 lies below or on that last boundary. Geometric and Harmonic Means: Given n positive weights w1. Notes issued 25-27 Sept. Only an Affine-Linear (or Non-Homogeneous Linear) function cTx + b can be both concave and convex everywhere on its domain.. Jensen’s Inequality becomes equality only when n = 1 or function U is affine-linear over at least the convex hull of the given arguments xj . G and H . 2000 6:23 am Analogously. Suppose now that x1. Kahan Notes on Jensen’s Inequality for Math. To prove that G ≤ A apply Jensen’s Inequality to the concave function log(x) to deduce that log(G) ≤ log(A) . 1. wn are all positive weights. n . w1 + w2 + w3 + … + wn This confirms Jensen's Inequality. a real function C(x) of a vector argument x is called Concave when its graph is the upper boundary of a convex region. w2.. and suppose w1.. define w1 x1 + w2 x2 + w3 x3 + … + wn xn A := --------------------------------------------------------------------------------.. w2. x2.. xn are distinct vector arguments in the domain of a convex function U(x) . 2. xn. and w1 + w2 + w3 + … + wn H := ----------------------------------------------------------.+ … + -----x1 x2 x3 xn Here A is a Weighted Arithmetic Mean of the set { x1.. 2..

. and v1. Sn really does decrease as n increases.. and justifies calling the angle arccos(a•b/(||a||·||b||) = arccos(( ∑j aj·bj )/√( ∑j aj2 · ∑k bk2 )) the unsigned angle between two column vectors a and b in Euclidean N-space.. Note that the inequality stays true in the limits as p → 1 or → +∞ .. 1 – 1/n. or powers of 2 on a computer?) . 1 + 1/n... (The sequence √(snSn) → e faster... . This implies that every term in the increasing sequence sn is less than every term in the decreasing sequence Sn . W. v3.. aN] is defined usefully as ||aT||q := q√(∑j |aj|q) in conjunction with ||b||p := p√(∑j |bj|p) for column vectors b := [b1... The mnemonic for Minkowski’s inequality is “ ||b + c||p ≤ ||b||p + ||c||p ” ... In non-Euclidean spaces generally no useful notion of angle need exist.. Meanwhile sn < Sn because sn/Sn = (1 – 1/n2)n < 1 . 1 } have a smaller geometric mean than arithmetic mean. now with wj = vjp and xj = (uj/vj)p and the concave function (1 + p√x)p . Therefore sn really does increase with n .Prof.. v2. 2000 6:23 am An elementary application of the inequalities among the arithmetic and geometric means yields a proof that e = 2. The special case p = q = 2 is called Cauchy’s Inequality. First observe that sn < sn+1 because the geometric mean of n+1 numbers { 1 + 1/n... … and v1. Then p√( ∑ (u + v )p ) ≤ p√( ∑ u p ) + p√( ∑ v p ) . H90 September 27.. apply Jensen's Inequality to the convex function xq with weights wj = ujp and arguments xj = uj·vj/wj . u3.. and all of u1..1/n)n as integer n approaches +∞ .. but often the length of row-vectors aT := [a1. u2. Kahan Notes on Jensen’s Inequality for Math... 1 + 1/n. so both sequences must converge. and suppose all of u1. u3. Another important application of Jensen’s Inequality is a proof of … Hölder’s Inequality: Suppose q/(q-1) = p > 1 (and so p/(p-1) = q > 1 too)..718281828459… exists as the limit of the increasing sequence sn := (1 + 1/n)n and also of the decreasing sequence Sn := 1/(1 . ... as claimed. v2. apply Jensen's Inequality again. and ê ≥ ë .. These definitions of length ||…||… make sense because they satisfy … Minkowski’s Triangle Inequality: Suppose p > 1 . Notes issued 25-27 Sept... b2.. with equality just when u1p/v1q = u2p/v2q = u3p/v3q = … .... Next observe that 1/Sn < 1/Sn+1 because the n+1 numbers { 1 – 1/n.. u2. and then “ aTb ≤ ||aT||q·||b||p ” is the mnemonic for Hölder’s inequality. 2000 Page 3/3 . sn increases to some limit ë while Sn decreases to some limit ê as n → +∞ .. but still requires n to be huge to achieve appreciable accuracy. To prove this. called e ... To prove this... are positive. Note again that the inequality remains true in the limits as p → 1 or → +∞ . bN]T . …. Then ∑j uj·vj ≤ p√(∑j ujp)· q√(∑j vjq) .. 1 – 1/n. Therefore both sequences converge to the same limit. Must ê = ë ? For all n > 2 consider 0 ≤ ê – ê < Sn – sn = ( 1 – (1 – 1/n2)n )·Sn = ( 1 – (1/Sn·n)1/n )·Sn < ( 1 – 1/S41/n )·S2 → 0 as n → +∞ ... a2. ... j j j j j j j with equality just when u1/v1 = u2/v2 = u3/v3 = … . So ê – ë = 0 .. . Can you see why roundoff may blight the computation of terms in any of the three sequences for very huge n unless it is restricted to powers of 10 on a calculator.. 1 } is less than their arithmetic mean.. … are positive. v3.. .

- Current Affairs Listing of Complete Year Prelims
- Cls Jeead-14-15 Xi Mat Target-2 Set-2 Chapter-6
- Contain
- 1008311283233572 Syl Log is m
- ++History gist by Ghanshyam thori
- ++Environmental Studies UGC
- Vision IAS Febuary 2015 Current Affairs
- M 4Y Lines and Angles Assignment-1.
- HRT-3 for 9th test math
- M 3Y Quiz Inequalities
- 3.0HRT-2-2013(ACEG)
- m 4y Heron's Formula Assignment
- Assnm Sequence & Series Module-4.
- Aiimiit Progression 1
- notes.420.02
- 26241890 Champions Speak
- C 2Y METALLURGY Extraction Purification Assignment 1
- Van Gogh - The Passionate Eye (Art eBook)
- The Complete Photographer - A Masterclass in Every Genre (DK Publishing) (2010)
- Dpp (1) 13th Maths Ans
- M 4Y Trigonometry Assignment-1
- War of the Worlds
- Ethics Note Topper AIR 386 Rahul Venkat
- 12th Paper
- Rcox Thesis

jenson identity

jenson identity

- 509S05Final-soln
- Homework 7.pdf
- Maths vit
- Ar Ve Son Spectral Solutions
- Householder Matrices
- matematika
- Chapter 1(2)
- C1 Revision Guide AWK
- Section_3.5
- How to Graph Functions
- Determinants 1
- Generalization Popvicious Inequality
- If Em
- MIB.LECTURE9
- 11 13 02 Optimization
- cmap2
- hw1
- 3.3.5 Quadratic Residue
- Linear Equations
- Chapter 1
- Lp and Sobolev Notes
- Lp and Sobolev Notes
- SLE Seminar
- Hilbert spae discription
- ec744note03
- Practice Test 1 Sol
- Maths Probability Limit Theorems lec8/8.pdf
- Convex Functions
- lec3.ppt

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue listening from where you left off, or restart the preview.

scribd