73 views

Uploaded by man420

fgd

- Empirical likelihood estimation of interest rate diusion model
- Parameter Estimation for the Lognormal Distribution.pdf
- The Big Problems File
- Lecture 2 - Mathematical Statistics
- Very Sparse Random Projections
- Econometrics Appendices B and C
- Structural Reliability
- Bias in Statistics_ Definition, Selection Bias & Survivorship Bias - Statistics How To
- Rouder Bayes R
- Econometrics_ch4.ppt
- Small vs big data
- CHAPTER 1 Estimation S2
- Maximum Likelihood Estimation in a Class of Nonregular Cases
- Infocom submission
- week 5 actl
- Estimasi Gamma Part 2
- Almost Unbiased Ratio and Product Type Estimator of Finite Population Variance Using the Knowledge of Kurtosis of an Auxiliary Variable in Sample Surveys
- Samples variance estimator
- Hanif Lecture
- 05696052

You are on page 1of 43

Method of Moments

Estimation

Patrick Zheng

01/30/14

Introduction

Goal: Find a good POINT estimation of population

parameter

Data: We begin with a random sample of size n

taken from the totality of a population.

We shall estimate the parameter based on the sample

probability distribution of the sample, which is

characterized by the parameter.

The distribution is always easy to identify

The parameter is unknown.

2

Notations

Sample: , ,,

Distribution: iid f(x, )

Parameter:

Example

e.g., the distribution is normal (f=Normal) with

unknown parameter and (=(, )).

e.g., the distribution is binomial (f=binomial) with

unknown parameter p (= p).

3

The importance of point estimates lies in the

fact that many statistical formulas are based

on them, such as confidence interval and

formulas for hypothesis testing, etc..

A good estimate should

1.

2.

3.

4.

Be unbiased

Have small variance

Be efficient

Be consistent

4

Unbiasedness

An estimator is unbiased if its mean equals

the parameter.

It does not systematically overestimate or

underestimate the target parameter.

Sample mean(x)/proportion( p ) is an unbiased

estimator of population mean/proportion.

Small variance

We also prefer the sampling distribution of

the estimator has a small spread or

variability, i.e. small standard deviation.

Efficiency

An estimator is said to be efficient if its

Mean Square Error (MSE) is minimum

among all competitors.

MSE( )

E(

where Bias( )

)2

E( )

Bias 2 ( )

v ar( ),

Relative Efficiency( , ) =

(

(

)

)

If <1, is more efficient than .

7

Example: efficiency

Suppose X1 , X2 ,...Xn iid~ N( ,

If 1 X1 ,then

Bias 2 ( 1 )

MSE( 1 )

If 2

X1

MSE( 2 )

X2

... X n

n

Bias 2 ( 2 )

Since R.E.( 1 , 2 )

v ar( 1 )

).

,then

v ar( 2 )

MSE( 2 )

=

MSE( 1 )

0

2

/n

2

1

=

n

/ n.

1,

Consistency

An estimator is said to be consistent if

sample size goes to +, will converge in

probability to .

0, Pr(|

0 as n

Chebychevs rule

0, Pr(|

E(

)2

2

MSE( )

2

goes to +, then is consistent.

9

Example: Consistency

Suppose X1 , X2 ,...Xn iid~ N( ,

Estimator

since

0, Pr(|

2

/n

2

X1

X2

).

... X n

n

)2

E(

2

is consistent,

MSE( )

2

0 as n

10

There are many methods available for

estimating the parameter(s) of interest.

Three of the most popular methods of

estimation are:

The method of moments (MM)

The method of maximum likelihood (ML)

Bayesian method

11

12

One of the oldest methods; very simple

procedure

What is Moment?

Based on the assumption that sample

moments should provide GOOD ESTIMATES

of the corresponding population moments.

13

How it works?

'

1

X; m

'

2

(1/ n)

n

i

2

X

1 i

14

X1 , X 2 ,...X n iid~ N( ,

step 1,

'

1

E(X)

'

1

step 2,

X; m

step 3,

Set

'

1

m1' ,

).

'

2

E(X 2 )

(1/ n)

;

'

2

'

2

2

X

.

1 i

m '2 , therefore,

X,

2

(1/ n)

n

i

2

X

1 i

X, 2

(1/ n)

n

i

2

2

X

X

1 i

15

x)

p

1 p

if x 1

if x 0

16

17

Note

MME may not be unique.

In general, minimum number of moment

conditions we need equals the number of

parameters.

Question: Can these two estimators be combined

in some optimal way?

Answer: Generalized method of moments.

18

Easy to compute and always work:

The method often provides estimators when other

methods fail to do so or when estimators are hard

to obtain (as in the case of gamma distribution).

MME is consistent.

19

They are usually not the best estimators

available. By best, we mean most efficient,

i.e., achieving minimum MSE.

(see next page for example)

20

Suppose we observe 3,5,6,18 from a U(0,)

Since E(X)= /2,

3+5+6+18

MME of is 2 X=2*

=16, which is

4

not acceptable, because we have

already observed a value of 18.

21

Likelihood

22

Proposed by geneticist/statistician:

Sir Ronald A. Fisher in 1922

Idea: We attempt to find the values of the

parameters which would have most likely

produced the data that we in fact observed.

23

What is likelihood?

when =1.

24

25

(discrete case)

26

(continuous case)

27

Definition of MLE

maximizing a function of single or several parameters.

One way to do the maximization is to take derivative.

28

29

30

Example contd

31

32

Example contd

33

34

Pros of Method of ML

When sample size n is large (n>30), MLE is

unbiased, consistent, normally distributed,

and efficient (regularity conditions)

Efficient means it produces the minimum MSE

than other methods including Method of Moments

35

Cons of Method of ML

MLE can be highly biased for small samples.

Sometimes, MLE has no closed-form solution.

MLE can be sensitive to starting values, which

might not give a global optimum.

Common when is of high dimension

36

1.

aforementioned)

2.

Newtons method, quasi-Newton method

(Broyden 1970), direct search method (Nelder

and Mead 1965), etc.

These methods can be implemented by R function

optimize(), optim()

37

Newtons Method

a method for finding successively better

approximations to the roots (or zeroes) of a

real-valued function.

Pick an close to the root of a continuous

function ()

Take the derivative of () to get ()

Plug into

(

(

)

,

)

( ) 0

38

Example

Solve 1 = 0

Denote ()= 1; let starting point = 0.1

()=

(

(

)

)

= 0.1

.

.

= 0.0048374

Repeat until |

| < 0.00001,

= 7.106 10

39

In Poisson Distribution, find is equivalent to

maximizing ln ()

( )

( )

define () =

() =

(

(

)

)

Given , , ,

40

Example contd

Suppose we collected a sample from Poi():

18,10,8,13,7,17,11,6,7,7,10,10,12,4,12,4,12,10,7,14,13,7

( )

( )

41

() =

-lnL(lamda).

42

The End!

Thank you!

43

- Empirical likelihood estimation of interest rate diusion modelUploaded bymirando93
- Parameter Estimation for the Lognormal Distribution.pdfUploaded byAhmad Rizalul Harbi
- The Big Problems FileUploaded byMichael Mazzeo
- Lecture 2 - Mathematical StatisticsUploaded byIda Khairina Kamaruddin
- Very Sparse Random ProjectionsUploaded byvkk106
- Econometrics Appendices B and CUploaded byReneẻ Màlizía Marandu
- Structural ReliabilityUploaded byBMWrider1453
- Bias in Statistics_ Definition, Selection Bias & Survivorship Bias - Statistics How ToUploaded byMurali Dharan
- Rouder Bayes RUploaded byrr
- Econometrics_ch4.pptUploaded byKashif Khurshid
- Small vs big dataUploaded byshalen
- CHAPTER 1 Estimation S2Uploaded bysupercarlaaa
- Maximum Likelihood Estimation in a Class of Nonregular CasesUploaded byThaís Martins
- Infocom submissionUploaded byShubhabrata Roy
- week 5 actlUploaded byJohnny Bui
- Estimasi Gamma Part 2Uploaded byEllazka D'are
- Almost Unbiased Ratio and Product Type Estimator of Finite Population Variance Using the Knowledge of Kurtosis of an Auxiliary Variable in Sample SurveysUploaded byAnonymous 0U9j6BLllB
- Samples variance estimatorUploaded byshotorbari
- Hanif LectureUploaded byRenukha Pannala
- 05696052Uploaded byDimas Benasulin
- LESSON 11Uploaded byMike Lawrence
- Computational StatisticsPart2Uploaded byaset999
- R05421204-PATTERNRECOGNITIONUploaded byvanamgoutham
- Taylor Effect PaperUploaded byRoss80
- HW9Uploaded byRichard Adhyaputra
- BayesUploaded byLeón Vw
- BiasUploaded bymarian_grajdan
- FAUploaded bybarber bob
- 0611914Uploaded byMahfudhotin
- Topic3 Multiple RegressionUploaded byZulutan

- 10 Stress Health CopingUploaded byman420
- PAKISTAN AFFAIRS MCQS BOOK.pdfUploaded byman420
- Who is Who & What is What _ General KnowledgeUploaded bymuhammad_sarwar_27
- AWDC_AnnualReport2016Uploaded byman420
- DISC 112-Computers& Problem Solving-Zehra Waheed-Shakeel Sadiq JajjaUploaded bytuahasiddiqui
- World Economic OutlookUploaded bylito77
- r CorrelateUploaded byman420
- Stata Merging 2Uploaded byman420
- 18 ReferenceUploaded byman420
- Linerar Algebra 2Uploaded byman420
- Friction Al FinanceUploaded byman420
- 2.b.1_MonteCarlo_v1.xlsUploaded byman420
- INOMICS Salary Report 2015Uploaded byman420
- Level II 2015 2016 Program ChangesUploaded bympartab3501
- Austria - University of Applied Sciences BFI ViennaUploaded byman420
- Easy Kitchen Menu Items-2Uploaded byman420
- Python for InformaticsUploaded byMukundan Sankar
- Pages From MAF Johnson ExamMLC3L StudySupplement October2012Uploaded byEsme Kurnali
- definite+matricesUploaded byman420
- Body ShopUploaded byman420
- Statistics ExercisesUploaded byman420
- GroupsUploaded byman420
- Assignment 2Uploaded byman420
- Student Solution ManualUploaded byNicolas Lefevre-Laumonier
- Research Challenge Official RulesUploaded byman420
- 320Uploaded byman420

- MHI - PRESENT STATUS AND FUTURE DEVELOPMENT OF DOUBLE HULL TANKERS.pdfUploaded byTermiteEnkay
- BYU FSAE Race Car StudyUploaded bygosculptor
- Case Presentation for Head InjuryUploaded byYdynn Parejas Gavina
- eac 551 narrativeUploaded byapi-241121673
- Transformer Design CodeUploaded bymaheshggarg
- DECE 1040-3040 SYLLABUSUploaded bynanobala15
- A Biodiversity Medplant Gobi Desert Mongolia 2018Uploaded byMelsipa
- Alternate Energy SourcesUploaded byBhavesh Pipaliya
- lab 2Uploaded byapi-300103665
- primis_be_032604Uploaded byJan Michael Vincent Garcia
- ASTM 4228 - Calificacion Aplicadores de recubrimientos.pdfUploaded byEl_memito
- [IBT] - G8 - Documentary CreditUploaded bytram_ngoc_19
- Business Ethics Ppt FinalUploaded bySreena Binu
- Juan_Carlos_Pagan__CV_.pdfUploaded byAnonymous N3pvnIQr
- Hemorrhoids Pathophysiology to TreatmentUploaded byKen Cheung
- External Connection Add-In.pdfUploaded byanon_984093298
- FANUC EnglishUploaded byhac_xa
- Using SAP Cloud Platform IntegrationUploaded byRavi
- vcoablary qUploaded byapi-245625982
- Conect OresUploaded bylorena ropero
- Floating PointUploaded byShanmugha Priya Balasubramanian
- Mil SIL DiscrepanciesUploaded byArun Kumar
- Nemirovskii NP HardnessUploaded bySilvia Truşcă
- Assignment 1 Scoping StudyUploaded byWray Tan
- 1971 Bennet,The Talofloc Decolorization ProcessUploaded bynghi
- Jackfruit ReportUploaded byCristopher Patayon
- Chapter 13 Iin 2007 Ver - GeanKoplisUploaded byHaris_Shajar
- Pomegranate de SeederUploaded byKamal Bharakhda
- Nouveau Document Microsoft WordUploaded bykenlesurvivant
- Intra-Lock International v. Dentium et. al.Uploaded byPriorSmart