: Jensen, Log Sum

: :

.....................................................................................................................3
Abstract....................................................................................................................................4
1. (INTRODUCTION).............................................................................................5
2. Jensen..............................................................................................6
3. Log sum (Log sum inequality).........................................................9
4. (CONCLUDING REMARKS).................................................................12
(REFERENCES)...............................................................13

:
Jensen Log sum. Jensen
.
,
, .
, log sum
.

:
Jensen, Log sum

3
Abstract

This project describes two inequalities, such as: Jensen's inequality and Log sum
inequality. Jensen's inequality is generalization of several well-known and lesser-
known inequalities. From this inequality derive much inequality as inequality between
geometric and arithmetic areas, Chebyshev's inequality etc.
In mathematics, the log sum inequality is an inequality which is useful for proving
several theorems in information theory.

Key words:
Jensen's inequality, Log sum inequality.

4
1. (INTRODUCTION)

Jensen
,
. Jensen
Johan Jensen ,
.
Jensen 1906. ,
,
.

;
.

. ,
Jensen.
Jensen
.

, log sum
.

5
2. Jensen

Jensen
Johan Jensen ,
.
Jensen 1906. ,
,
.

;
.

. ,
Jensen.
Jensen
.
:

( )
I R. g : I R. g e ,
x, y I d [0, 1] : g(x + (1 )y) g(x) + (1 )g(x).
.

1.1 ( )
I R. g : I R. :
1. g e I,
2. g 0 I, g .
Jensen
:

6
 1.2
I R,

x0 I I0 : I 0 ( x ) ( x ) x I I 0 ( x0 ) = ( x 0) .

:
:

1. h > 0 , = :
( x ) ( xh) ( x +h ) (x)

h h

2. h1 >h2 =h2 / h1

-h1 :
( x ) ( xh) ( x ) (xh2)

h1 h2

( x +h1 ) (x) ( x +h2 ) (x)

h1 h2

+
(2) , h 0

,
+ ( x ) ( xh ) ( x+ h ) (x )
h 0 h 0 +
h h
' '
( x )=lim + ( x )=lim

' '
(1) ( x ) + ( x ) x . z

a [ ' ( z ) , ' + ( z ) ] l

l z ( x )= ( z ) + a(xz)

l ( z )= ( z ) ,

: z ( x ) ( x ) xI .

7
 1.3 ( Jensen)

g I, R. x I

1 , g (X) .

g ( EX ) E (g ( X ) ) .

I . l ( x )=ax+ b e g

() EX. :

1. l ( EX )=g ( EX)

2. l ( x ) g ( x ) x I

(2) , :
Eg ( x ) E l ( x )=E ( ax +b ) =aEX +b=l( EX)

1.

Jensen .

.

1.4

8
 a1 , , an >0 .

ai
n

1
n
= ai

n i=1 , i=1 GM AM.


GM =

{ a1 , , an },

1
P ( X=ai )=
n . , =. log GM log .

X
Jensen, log log E X =logAM ,
E

X
log =log GM . ,
E

ai


1/ n

ai


GM =log
log

9
3. Log sum (Log sum inequality)

, log sum e
.

a1, a 2 , , an b1 , b2 ,b n :

n
a n ai
a i ln bi
i=1 i
( )
i=1
ai ln i1
n

bi
i=1

a 0
( 0 ln 0=0, aln = a>0 ln =0
0 0

ai
=cost ,i=1,2,3, n
bi

, ai >0 bi > 0 .

'' 1
f ( t )=t logt f (t )= loge >0
t

t . , :

i f ( t i) f ( i t i) i 0, i=1.
i

b ai
i= n
t i= ,
bi log
bj
j=1

sum:

10
 ai ai ai a
log log i
bj bi b j bi

.

1. D( p q) 0 ,

p ( x )=q( x ) . log sum :

p ( x ) ( p ( x ) ) log p ( x ) 1
p|q )= p ( x ) log =1 log =0
q(x) q(x) 1
D

p( x)
=c . p q
q ( x)

p|q )=0
, c=1, D

p ( x )=q ( x ) x .

2
H ( p ) e p .

:
H ( p )=log |x|D( p u)

u , D , H
.

. x 1 p1 x 2 p2

, .

11
 = {2
1
1

Z =X , Z X 1 X 2

. Z p1 +(1) p2 .

H ( Z ) H ( Z| ) , .

,
p
( 2)
H ( Z| )= p ()H ( Z| )=H ( Z|=1 )+ (1 ) H ( Z|=2 )=H ( p1 ) + ( 1 ) H
0

:
H ( p1 +1) p2 H ( p1 )+(1 ) H ( p2 )

( X , Y ) p ( x , y )= p ( x ) p ( x , y ) . I( X ;Y)

p(x) y) y)

p p

p (x ) .

:
,

p(x) p(x }) ,
p ( y )= p( x , y )=
x x

p(x) x)
.
p

:
I ( X ; Y )=H ( Y )H ( Y | X )=H ( Y ) p ( x ) H (Y X=x )
X

12
 , p(x) ,

p( y ) p (x ) H (Y )

p( y) p (x ) .

y

p (x )
, y

p1

y


y

p1 ( x , y ) = p ( x ) p 1

y


p ( x ) , p 1 ( y ) , p ( x ) , p2 ( y ) . y
, .

y)
p

( p ( x ) ).

p ( x , y )= p1 ( x , y )+(1 ) p2 (x , y )

Y :
p ( y )= p1 ( y ) + ( 1 ) p 2 ( y ) .

13

.
:
q ( x )= p ( x ) p ( y )= p ( x ) ( p1 ( y )+ ( 1 ) p2 ( y )) q 1 ( x , y ) + ( 1 ) q2 ( x , y )

p |q ) .
, I ( X ; Y )=D D

( p , q ) , . .

4. (CONCLUDING REMARKS)

Jensen .


;
.

14
5. (REFERENCES)

1. Thomas. M. Cover, J. A. Thomas. (2006) Elements of Information Theory,

John Wiley & Sons.
2. http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_The
ory/lecture3.pdf
3. https://en.m.wikipedia.org/wiki/Jensen%27s_inequality
4. http://www.math.mcgill.ca/~dstephens/556/Handouts/Math556-05-
Inequalities.pdf
5. http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_The
ory/lecture3.pdf

15
16