© All Rights Reserved

7 views

© All Rights Reserved

- Campbell 2007 Equity Returns
- Bayes Workbook
- Random Variables
- Maxwell Boltzmann Distribution
- heteroskedasticity
- System Analysis & Contol
- Regression Estimators
- Improve CaO% in Lime
- Diversification Strategy, CEO Management Style and Firm Performance 2011
- Measuring Value Relevance in a (Possibly) Inefficient Market
- Acce Course Modules
- L2 Probabilistic Reasoning
- UNIT I - Random Variables
- Assignment of Or
- PPF eqs
- Etf3600 Lecture3 Mle Lpm 2013
- std-12-maths-2-board-question-paper-maharashtra-board.pdf
- Abele Et Spurk (2009). How Do Objective and Subjective Career Success Interrelate Over Time
- ASE Critica
- DL Total Noise

You are on page 1of 3

This note aims to give a more general definition of the likelihood function.

In class, we saw how to define the likelihood associated with a statistical

model in two cases: when the observations are continuous and when they are

discrete. In general, the data do not fall in one of these cases. For example,

in the logistic regression, the data are i.i.d. pairs (Xi , Yi ), i = 1, . . . , n, where

the Xi s may be continuous random variables, whereas the Yi s are Bernoulli

random variables. In addition, we have always assumed that the data are

i.i.d. However, sometimes, they may not be identically distributed. This is

the case, for example, in the linear regression with deterministic design:

Yi = Xi + "i , i = 1, . . . , n,

where the Xi s are fixed (deterministic) and known covariates and the noise

terms "i are i.i.d. In this case, the data are Y1 , . . . , Yn , which are independent

but not identically distributed (for instance, they do not have the same expectation). More generally, the data may not even be independent ! This is

the case in heteroscedastic linear regression: the noise terms "i are no longer

assumed to be i.i.d., but instead, the vector " = ("1 , . . . , "n ) is assumed to be

centered Gaussian with covariance matrix that is not necessarily diagonal.

In all those cases, it is still possible to define the likelihood function.

Let us set some notation. The sample is denoted by Z1 , . . . , Zn (in the

first chapters, we usually used the letter X instead of Z, but it is more

convenient to use Z in this note). For example, the Zi s could be Bernoulli

random variables, or exponential random variables, or each Zi could be a

pair (Xi , Yi ) (e.g., in the regression setup). The space of the data is denoted

by E: Zi E, i = 1, . . . , n (e.g., E = Rp R for the linear regression, E = R

for a sample of Gaussian random variables, etc...) and is the parameter

space. Recall that the parameter is what we aim to estimate (in the linear

regression, the parameter space is Rp and the parameter is ).

Then, the likelihood is defined as the following function:

Ln

En

R

(z1 , . . . , zn , )

(z1 , . . . , zn ),

1

given that the parameter is . If Z1 , . . . , Zn are assumed to be independent,

then

(z1 , . . . , zn ) is nothing but the product of the pdfs of each Zi , i =

1, . . . , n.

The probability distribution function

If Z is a discrete random variable on a discrete set E, then its pdf is

the function

Z E R

z P[Z = z].

If Z is a continuous random variable, then its pdf is its density.

If Z = (X, Y ), then its pdf (denoted by X,Y ) is also called the joint

pdf of X and Y . If X and Y are independent, then their joint pdf is

the product of the pdfs of X and Y (denoted by X and Y ):

Z (x, y) = X (x)Y (y).

If X and Y are not independent, one can use Bayes rule, that states

that:

Z (x, y) = X (x)Y X=x (y),

where Y X=x is the conditional pdf of Y given X = x.

Examples

and i.i.d. Gaussian noise terms with unknown variance 2 , when the

parameter of interest is ( , 2 ). Additional question: Compute the

MLE ( , 2 ).

Heteroscedastic linear regression with deterministic design:

Yi = Xi + "i , i = 1, . . . , n

Yi = Xi + "i , i = 1, . . . , n

where the Xi s are i.i.d. continuous random vectors with given density

f and, for all x Rp , the conditional distribution of " given X = x is

Nn (0, (x)), for some known matrix valued function (x), when the

parameter is just . Additional question: Compute the MLE .

2

Yi Ber (Xi ) , i = 1, . . . , n,

where (u) =

eu

, u R.

1 + eu

i.i.d. random pairs, where X1 is a continuous random variable in Rp

with some density g and for x Rp , the conditional distribution of Y1

given X1 = x is Bernoulli with parameter (Xi ).

- Campbell 2007 Equity ReturnsUploaded byHamid Waqas
- Bayes WorkbookUploaded byAllister Hodge
- Random VariablesUploaded bymadan@123
- Maxwell Boltzmann DistributionUploaded byDean Aidan Martinez
- heteroskedasticityUploaded byarmailgm
- Regression EstimatorsUploaded byosiccor
- Improve CaO% in LimeUploaded byemailidabhi
- Acce Course ModulesUploaded byMuhammad Jawad Vohra
- System Analysis & ContolUploaded byAbhinav Chaudhary
- Diversification Strategy, CEO Management Style and Firm Performance 2011Uploaded byLie Michael Eliantono Adiseputra
- L2 Probabilistic ReasoningUploaded bySahil Kothari
- PPF eqsUploaded bysreedeviish
- Measuring Value Relevance in a (Possibly) Inefficient MarketUploaded by1111sami
- UNIT I - Random VariablesUploaded byShubham Vishnoi
- Etf3600 Lecture3 Mle Lpm 2013Uploaded byVince Lau
- std-12-maths-2-board-question-paper-maharashtra-board.pdfUploaded bySachin Dedhia
- Assignment of OrUploaded byPalluRaju
- Abele Et Spurk (2009). How Do Objective and Subjective Career Success Interrelate Over TimeUploaded byzizkhalil
- ASE CriticaUploaded byRolando Juan Dávila Salcedo
- DL Total NoiseUploaded byAnjit Rajkarnikar
- Lecture 6Uploaded bySandra Gilbert
- Estimation of the Fscal Impact MultiplierUploaded byMarcin Pietrzak
- CT3_QP_1111.pdfUploaded byNitai Chandra Ganguly
- maji.pdfUploaded byAnees Calicut
- k_28rvgUploaded byTigabu Yaya
- Capitulo_11Uploaded bypreziprezi
- practice1_winter2012Uploaded byJustinMalin
- chapter 8 (2)Uploaded bySo-Hee Park
- 06 SINCOS 90645 Icecream ActivityUploaded byjbbowler
- 097 Real-Time Traffic Measurement From Single Loop Inductive SignaturesUploaded byRashad M. El-Etreby

- Reading a Analog in and Controlling Audio Volume With the Raspberry PiUploaded bylolo406
- Why Friendster FailedUploaded byAkbar Thobhani
- Fundamental Plc Trg ScheduleUploaded bykodandaram
- FDP BrochureUploaded byसौरभ शर्मा
- Steel design manualUploaded bymuggur
- Vulnerability Assessment Best PracticesUploaded byYaser Shaikh
- Datastage Online Training.pdfUploaded bysoftsystraining
- ASME Y14Uploaded bynagrav
- RadExPro_release2016.3Uploaded byBrahim Bettahar
- List of Deep Learning and NLP ResourcesUploaded byBartoszSowul
- Approach Enterprise Security Architecture 504Uploaded byJason DolLaz R Us
- RFIDUploaded bydebashish1135
- 522140Uploaded byvisheshmonoxide
- Windows Forms Using C# Revision 3.0 Student GuideUploaded byissam
- AIUploaded byJayanta Sinhamahapatra
- pricelistUploaded byAnil Kumar
- EDU_CAT_EN_PHS_FF_V5R19_toprint.pdfUploaded bySolidWorks
- dairy farm group case studyUploaded bybinzidd007
- Exxsol d80 - Exxon Mobil - Brenntag LaUploaded bydeyvimor
- How to Take Note at law school.docxUploaded byMasterHomer
- assessmentUploaded byapi-243749103
- Viz Shotcut Guide 1.3Uploaded bylevisjavi
- Calypso 01 KurveUploaded byDragu Stelian
- CH4 Strings RegexpUploaded byChitra Devi
- Reservoir CouplingUploaded byGigih Adi Pambudi
- PRGMNL-848A-1596A COPPER CONNECTIONS.pdfUploaded bykrishnamarajublr
- Preventing File Inclusion Attacks on WebsiteUploaded byInternational Journal Of Emerging Technology and Computer Science
- VSoft launches new data center at HyderabadUploaded byVSoft Technologies
- PocketMoney for iPhone User ManualUploaded byWaleska Paes de Barros
- Filmdatabase-ORM-UML_eng.pdfUploaded bykhai