Identification Problems in the Social Sciences and Everyday Life

Author(s): Charles F. Manski

Reviewed work(s):
Source: Southern Economic Journal, Vol. 70, No. 1 (Jul., 2003), pp. 11-21
Published by: Southern Economic Association
Stable URL: http://www.jstor.org/stable/1061629 .
Accessed: 07/11/2012 10:42

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Southern Economic Association is collaborating with JSTOR to digitize, preserve and extend access to
Southern Economic Journal.

http://www.jstor.org
SouthernEconomicJournal2003, 70(1), 11-21

AssociationLecture

Identification
Problemsin the Social
Sciences and EverydayLife
CharlesF. Manski*

Econometricians havefounditusefulto separatetheproblem ofempirical inferenceintostatistical

andidentificationcomponents.Studiesofidentification
determinetheconclusions thatcouldbe drawn
if a researcherwereable to observea datasampleof unlimited size. Statistical
inferenceseeksto
characterize how sampling affects
variability theconclusionsthatcan be drawnfromsamplesof
limited size. ThisAssociationLectureto theSouthern EconomicAssociation describesthebroad
themesofa research program on identification
thatI beganin thelate1980sandcontinue today.I
showhowthesethemes haveplayedoutinmyanalysis oftheselection problem, a fundamentaland
pervasiveidentificationproblem.I examinehow the selectionproblemmanifests itselfin the
econometric analysisofmarketdemand.

1. Introduction

The ReflectionProblem

Here is an identification
problemfromeverydaylife: Suppose thatyou observe the almost
simultaneousmovementsof a personand of his image in a mirror.Does themirrorimage cause the
person's movements,does the image reflectthe person's movements,or do the personand image
move togetherin responseto a commonexternalstimulus?Empiricalobservationsalone cannot
answerthisquestion.Even if you wereable to observeinnumerable instancesin whichpersonsand
theirmirrorimagesmove together, you wouldnotbe able to logicallydeducetheprocessat work.To
reacha conclusionrequiresthatyou understand somethingof opticsand of humanbehavior.
A like inferential
problem, which I have called thereflection
problem(Manski 1993a), arisesif
you tryto interpretthe commonobservationthatindividualsbelongingto the same grouptendto
behavesimilarly.Two hypothesesoftenadvancedto explainthisphenomenonare endogenouseffects,
whereinthe propensity of an individualto behave in some way varies withthe prevalenceof that
behaviorin thegroup;and correlatedeffects, whereinindividualsin the same grouptendto behave
similarlybecause theyface similarenvironments and have similarindividualcharacteristics.
Similarbehaviorwithingroupscould stemfromendogenouseffects(e.g., groupmemberscould
experiencepressureto conformto groupnorms)or groupsimilarities mightreflectcorrelatedeffects
(e.g., personswith similarcharacteristics
mightchoose to associate withone another).Empirical

*
Departmentof Economics and InstituteforPolicy Research,Northwestern 2003 SheridanRoad, Evanston,
University,
IL 60208; E-mail cfmanski@northwestern.edu.
This paper is based on my AssociationLecturedeliveredto the SouthernEconomic Associationat its 2002 meetingin
New Orleans.

11
12 CharlesF. Manski

observations ofthebehaviorof individualsin groups,even innumerablesuchobservations,cannotper

se distinguishbetweenthesehypotheses.To draw conclusionsrequiresthatempiricalevidencebe
combinedwithsufficiently strongmaintainedassumptionsaboutthenatureof individualbehaviorand
social interactions.
Whymightyoucarewhether ofbehavioraregeneratedbyendogenouseffects,
observedpatterns
processeshave
by correlatedeffects,or in some otherway? A good practicalreasonis thatdifferent
differing for
implications public policy. For example, understandinghow studentsinteractin class-
roomsis criticalto theevaluationof manyaspectsof educationalpolicy,fromabilitytrackingto class
size standardsto racial integration
programs.
observedpatternsof behavior,you seek the expertadvice of
Suppose that,unable to interpret
two social scientists.One, perhapsa sociologist,assertsthatpressureto conformto groupnorms
makestheindividualsin a grouptendto behave similarly.The other,perhapsan economist,asserts
thatpersonswithsimilarcharacteristics choose to associate withone another.Both assertionsare
consistentwiththe empiricalevidence.The data alone cannotreveal whetherone assertionor the
otheris correct.Perhapsbothare. This is an identification
problem.

and StatisticalInference
Identification
Identificationproblemsare problemsof deductivelogic. The conclusionsthata researchercan
logicallydraw are determined by theassumptionsand datathatarebroughtto bear.The availabledata
about humanbehaviorare typicallylimited,and the range of plausible assumptionsis wide. So
researcherswho analyze the same data underdifferent maintainedassumptionsmay,and oftendo,
reachdifferent logicallyvalid conclusions.
Empiricalresearchersoften ask econometriciansfor assistance in "solving" identification
problems.This is askingtoo much. What econometricians can usefullydo is to clarifywhatcon-
clusionscan and cannotlogicallybe drawngivenempirically relevantcombinationsof assumptions
and data.
For morethana century, methodological use
researchin thesocial scienceshas madeproductive
of probabilityand statistics.One supposes thatthe empiricalproblemis to infersome featureof
a populationdescribedby a probabilitydistribution and thatthe available data are observations
extractedfromthe populationby some samplingprocess.One combinesthe data withassumptions
about the populationand the samplingprocessto draw statisticalconclusionsabout the population
featureof interest.
Workingwithinthis familiarframework, econometricianshave found it useful to separate
components.Studiesof identification
inferenceintostatisticaland identification determinethe con-
clusionsthatcould be drawnif a researcherwere able to observea data sample of unlimitedsize.
how samplingvariability
seeks to characterize
Statisticalinference affectstheconclusionsthatcan be
drawn from samplesof limited size.
distinctforit to be fruitful
and statisticalinferenceare sufficiently
Identification to studythem
separately.The usefulnessof separatingtheidentification and statistical has
componentsof inference
long been recognized.Koopmans (1949, p. 132) putit thisway in the that
article introduced
theterm
intothe literature:
identification
In ourdiscussion we haveusedthephrase"a parameter thatcan be determinedfroma sufficient
number of observations."We shallnow definethisconceptmoresharply, and giveit thename
Insteadofreasoning,
of a parameter.
identifiability as before,from"a sufficientlylargenumberof
weshallbaseourdiscussion
observations" ona hypothetical knowledge oftheprobability
distribution
oftheobservations, morefullybelow.It is clearthatexactknowledge
as defined ofthisprobability
 AssociationLecture 13

distribution
cannotbe derivedfromanyfinite number ofobservations.
Suchknowledge is thelimit
approachable butnotattainableby extended observation.
By hypothesizingneverthelessthefull
ofsuchknowledge,
availability weobtaina clearseparation
between
problems ofstatistical
inference
arisingfromthevariability
offinite
samples,andproblems ofidentification
inwhichwe explorethe
limitsto whichinference
evenfroman infinite number ofobservations
is suspect.

My Research Program
I have been concernedwithidentification problemsthroughout my career.My earlyresearch
concernedtheproblemof inference on people's preferences
fromobservations ofthechoicesthatthey
make. Economistsare fondof sayingthatchoicebehavior"revealspreferences." In fact,observation
of theactionthata personchooses onlyrevealsthatthisactionis weaklypreferred to all otherfeasible
actions.It does notrevealhow thepersonranksnonchosenactionsrelativeto one another.Discrete
choice analysis,as it is practicedin econometrics, combinesdata on choices withassumptionsabout
thedecisionrulesthatindividualsuse whenmakingthechoicesthatresearchers observe.The concern
of myearlyresearchwas to determine whatcan be learnedaboutpreferences givendata on choices
and relativelyweak assumptionsabout the decisionrules thatpeople use. This is an identification
problem.
Over time,I have come to thinkthat,althoughstatistical
problemscontribute to thedifficulty
of
empiricalresearch, identificationis the more fundamentalproblem of the social sciences. In what
follows,I firstdescribethe broad themesof a researchprogramthatI began in the late 1980s and
continuetoday. I next show how these themeshave played out in my analysis of the selection
problem,a fundamentaland pervasiveidentification problem.I then examine how the selection
problem manifests itselfin the econometric of
analysis marketdemand.

2. Broad Themes

My 1995 book,IdentificationProblemsin theSocial Sciences (Manski 1995), putsforwardfour

broad,relatedthemes.They are as follows.

Begin with the Data Alone

The prevalentapproach to empiricalresearchin the social sciences begins by maintaining
assumptionsthatare strongenoughto identify quantitiesof interestand to yieldstatistically
precise
point estimatesof these quantities.Concernsabout the credibilityof assumptionsare commonly
addressedthroughtheperformance of specification
testsand/orsensitivity analysis.Concernsabout
credibility
may also be addressedby exploringhow estimates change and statistical
precisionfallsas
functionalformand distributionalassumptionsare weakened.
A complementary approachto empiricalinferencebeginsby askingwhatcan be learnedfrom
thedata,givenonlytheknowledgeof the samplingprocessand no otherpriorinformation. Having
determined this,one may thenask whatmore can be learnedgiven successivelystrongerformsof
priorinformation. This approach yields a series of successivelytighterbounds on quantitiesof
interest.The bound is widestwhen no assumptionsare maintained,and it narrowsas strongeras-
sumptionsare imposed.Sufficientlystrongassumptionsnarrowthebound to a point.
This approachto empiricalresearchhas particularvalue whensocial scientists
invokingdifferent
strongassumptionsfindthemselvesin disagreement about the interpretation
of empiricalevidence.
Establishingtheconclusionsthathold up underweak assumptionscan build a domainof consensus
14 CharlesF. Manski

and confinedisagreementsto those questions whose resolutionsreally do requirecontroversial

maintainedassumptions.

Pointsand Bounds
I have just made reference to boundson quantitiesof interest.
Social scientistscommonlythink
of identificationas a yes-or-noquestion:a parameteris eitheridentified or not identified.Yet iden-
generally,is nota binarystate.A researcher
tification, who does nothave richenoughpriorinforma-
tionand data to infertheexactvalue of a parameter maynevertheless be able to partiallyidentifythe
parameter-that is, to bound it.
The fixationof social scientistson point identification has inhibitedthe appreciationof the
potentialusefulnessof bounds. I use the term"fixation"because I cannotreadilyunderstandthe
scientificbasis forthe longstanding notionthata parameteris eitheridentifiedor not. Bounds on
parameters have been reported from timeto timein the methodologicalliterature. Nevertheless, in
empirical research and in the of
teaching econometrics, identificationhas generally been thought of as
pointidentification.

Coping withAmbiguity
The scientificcommunitytendsto rewardresearcherswho produce strongfindings,and the
publictendsto rewardthosewho makeunequivocalpolicyrecommendations. These incentivestempt
researchersto maintainassumptionsthatare farstronger thantheycan persuasivelydefend,to draw
strongconclusions.We need to developa greatertoleranceforambiguity. We mustface up to thefact
thatwe cannotanswerall of thequestionsthatwe ask.
has yieldeda numberof formalnegativeresults.I have reported
My researchon identification
simple"impossibility theorems"showingthatbroadclasses ofhypotheses arenotempirically testable
unlesssufficiently
strongassumptions are maintained.I have foundthatresearchersare oftenreluctant
to acknowledgethat,given the available data, theycan logicallydraw some conclusionof interest
onlyif theymaintainstrongassumptionsthatmay have limitedcredibility. Be thatas it may,I feel
stronglythatbothpositiveand negativefindings contribute
to theadvancement of thesocial sciences.

EmpiricalInferencein Life
Social science seeks to understandthe behaviorof individualhumanbeings and theirsocial
interactions.In theirday-to-daylives, ordinarypeople face problemsof empiricalinference-
problemsof both identification and statisticalinference-similarto those that confrontsocial
problemsare subjectto the same rules of logic as are social
scientists.People facinginferential
The conclusionsthatpeople can logicallydraware determined
scientists. by theassumptionsand the
data thattheybringto bear.
Social scientistsneed to keep thisconstantlyin mindas we seek to model and interprethuman
behavior.We do notknowmuchabouthow people deal withtheinferential problemsthattheyface.
Economistshave been particularly negligent.Economistsusually suppose thatpeople's empirical
inferencesare expressedin theirexpectationsforthefuture.Expectationsare a subjectiveconcept,but
economistshave long exerciseda self-imposed on theuse of subjectivedata in empirical
prohibition
analysis.Rather than seek to learnabout expectations,economistshave generallymade assumptions
aboutexpectations.
 Association
Lecture 15

The rationalexpectationassumptionscommonlymade by economistsmay be elegant and

analyticallyappealing.However,theyhave littleempiricalsupport.In manyapplications,accepting
a rationalexpectations
assumptionmeansacceptingtheidea thatordinary people somehoware able to
solve identification
problems thathave long challengedsocial scienceresearch.As I see it,ordinary
people-like social scientists-haveto cope withambiguity.

3. The SelectionProblem

Social scientistsconstantlyask "treatmenteffect"questionsof the form:What is the effect

of on ? For example,whatis the effectof welfareprogramson labor supply?What is
theeffectof schoolingon wages? Whatis theeffectof the sentencingof offenders on recidivism?
Empiricalanalysis of treatmenteffects
poses a fundamental identification
problem,which is
commonly called the selectionproblem.The researcherwants to compare the outcomesthatpeople
would experienceif theywere to receivealternative treatments.However,treatments are mutually
exclusive.At most,theresearchercan observethe outcomethateach personexperiencesunderthe
treatment thatthispersonactuallyreceives.The researchercannotobservetheoutcomesthatpeople
wouldhave experiencedunderothertreatments. These otheroutcomesare counterfactual.Hence,data
on treatments and outcomescannotby themselvesrevealtreatment effects.

The Returnsto Schooling

Ordinarypeoplewantto learntreatmenteffectsineverydaylifeand so facetheselectionproblem.

Consider,forexample,youngpeopledecidingwhether to continuetheirschoolingor to enterthelabor
market.To makegood decisions,youngpeople wantto learntheirreturns of schooling.Theymaybe
able to observetheoutcomesexperiencedby family,friends,and otherswho have madetheirownpast
schoolingdecisions.However,theylogicallycannotobservewhatoutcomesthesepeople wouldhave
experiencedhad theymade otherdecisions. Thus, young people makingschoolingdecisions in
ordinarylifeare "adolescenteconometricians"who face identification
problemssimilarto thosethat
have made it so hardforlaboreconomiststo agreeon thereturns of schooling(Manski 1993b).

RandomTreatment
Selection
of treatment
Pointidentification responserequiresassumptionsabouttheprocessof determining
treatmentselectionand outcomes.The mostlongstanding practice,and stillthemostprevalentone, is
to assumethat,amongpeople withspecifiedobservablecovariates,treatment selectionis statistically
independent of outcomes. This is
assumption variously called random, exogenous,or ignorable
treatmentselection.The specifiedcovariatesare often,misleadingly,
said to "controlfor" treatment
assignment.
The assumptionof randomtreatment selectionis appropriatein the analysis of data from
classical randomizedexperiments. Indeed, thisis thereasonwhyrandomizedexperiments are valued
so highly.The assumptionof randomtreatment selectionis usually suspectin non-experimental
settings,whereobservedtreatments maybe self-selected or otherwisechosenpurposefully. Over the
years,a varietyof alternative
assumptionshave been proposedand appliedto non-experimental data.
Indeed, the developmentby econometriciansof latent-variable models and instrumental-variable
approachesin the 1970s was initiallygreetedwithenthusiasm as "solving" theproblemof identifying
treatmenteffectsfrom non-experimental data. It soon became apparent,however, that these
16 CharlesF. Manski

approachesreplacethesuspectassumptionofrandomtreatment
selectionwithalternative
assumptions
thatare no less suspect.

ComparingTreatments
Using theEmpiricalEvidenceAlone
My researchhas moved away fromthe conventionalfocus on assumptionsthatyield point
identification of treatment response.I began by askingwhatcan be learnedabouttreatment response
fromtheempiricalevidencealone,givenno assumptionsabouttheprocessgenerating treatmentsand
outcomes(Manski 1990). I foundthatthisquestionhas a simpleanswer.The observationof realized
treatments and outcomesdoes implyrestrictions on the distributionsof outcomesunderalternative
treatments. However,thedata are necessarilyconsistentwiththehypothesisthatthereis a common
distribution of outcomesundereverytreatment. Hence, empiricalevidencealone cannotdetermine
whetherone treatment is betterthananother.
Consider,forexample,thequestionofhowjudges shouldsentenceconvictedoffenders. The two
treatments mightbe imprisonment and probation.The outcomeof interest mightbe recidivism:Does
theoffender commita subsequentcrime?In thissetting, theobjectivemightbe to learntheclassical
treatment effect: the differencebetween the recidivism rate thatwould occur if all offenders were
and
imprisoned thatwhich would occur if all offenderswere sentencedto probation.
Suppose firstthatone has no data at all. Then all one can say abouttheclassicaltreatment effect
is thatit lies between-1 and 1, an intervalof width2. The treatment effectis -1 iftheprobability
of
recidivismis 0 underthe mandatoryimprisonment policy and is 1 under the mandatoryprobation
policy.The effectis 1 if thesepolicies have theoppositeconsequence.
It can be shownthatobservationof realizedtreatments and outcomesenables one to cut the
widthofthisintervalexactlyin half,to an intervalof width1 ratherthan2. Thus,data alone solvehalf
of theidentification problem.Assumptionsare needed to solve theotherhalf.
The locationof theintervalwithinwhichthetreatment effectmustlie depends,in a simpleway,
on theempiricalpatternof thedata on treatments and outcomes.Whatevertheintervalis, it neces-
sarilycontainsthe value 0. Hence, the data alone do not sufficeto determinethe sign of the treat-
menteffect.

An EmpiricalIllustration
Manski and Nagin (1998) analyzed observationaldata on the sentencingof 13,197 juvenile
offendersin thestateofUtahand theirsubsequentrecidivism.We comparedrecidivismunderthetwo
mainsentencing optionsavailabletojudges: confinement in residential
facilities(t = 1) and sentences
thatdo not involveconfinement (t = 0).
Let theoutcometakethevalue y = 1 ifan offender is convictedof a subsequentcrimein the2-
yearperiodfollowingsentencing, and y = 0 otherwise.The empiricaldistribution of treatmentsand
outcomesamongtheobservedoffenders
was foundto be as follows:

of residentialtreatment
probability = 0.11,
of
probability recidivismconditional = 0.77,
on residentialconfinement
of recidivismconditionalon nonresidential
probability treatment
= 0.59.

The problemis to use this empiricalevidence to draw conclusionsabout the probabilitiesof

recidivismundertwo hypothetical
policies--one in whichall offenders
are confinedand theotherin
 AssociationLecture 17

whichnone is. Someone willingto assume thatjudges presentlysentenceoffenders randomlycan

concludethattheserecidivismprobabilitiesare 0.77 and 0.59, respectively. However,theempirical
evidencerevealsonlythattheyfall in theranges[0.08, 0.97] and [0.53, 0.64].
Randomsentencing does notseem a particularlycredibleassumption, so thereis a clearneed for
empiricalresearchanalyzinghow judges actuallymake sentencingdecisions.The criminologyand
sociologyliteraturesreportsome descriptive,usuallyqualitative,studiesof judicial behavior.How-
ever,these studiesdo not yieldmuch information on judges' decisionprocesses.
A criticalopen question concerns the way thatjudges make theirown inferencesabout
recidivism.Judgesneedto knowtreatment responseiftheyareto makeeffective sentencing decisions.
Thus, in theireverydayprofessionallives, judges confrontthe same identification problemas do
criminologistswho studysentencingand recidivism.

TreatmentChoice Under Ambiguity

Recently,I have begun to explore the implicationsof the selection problem and other
problemsfortreatment
identification choice (Manski2000, 2002). I supposethata social plannermust
choose a treatment rulethatassignsa treatment to each memberof a heterogeneous population.The
plannercould, forexample, be a physicianchoosing medical treatments foreach member of a pop-
of a
ulation patients, school officialmakingcourse-placement decisionsforeach memberof a pop-
ulationof students,or a judge decidingsentencesforeach memberof a populationof convicted
offenders.
I suppose thatthe plannerobservescertaincovariatesforeach person.These covariatesde-
terminetheset of treatment rulesthatare feasibleto implement. The set of feasiblerulesis theset of
all functionsmappingthe observedcovariatesintotreatments. Each memberof thepopulationhas
a responsefunction mappingtreatments intoreal-valuedoutcomes.I supposethattheplannerwantsto
choose a treatment rulethatmaximizesa utilitarian social welfarefunction.
Suppose, forsimplicity, thatall treatmentshave thesame costs.Then it is easy to show thatan
optimaltreatment rule assigns to each memberof the populationa treatment thatmaximizesmean
outcomeconditionalon theperson'sobservedcovariates.The plannerfaces a problemof treatment
choice underuncertainty if he knows the conditionalmean responsesand, consequently,can im-
an
plement optimal rule.The plannerfacesa problemof treatment choice underambiguity ifhe does
notknowenoughaboutmeanresponseto be able to implement an optimalrule.The termambiguity
dates back at least to Ellsberg(1961). Economistssometimesreferto ambiguityas Knightianun-
certainty.
The generalpresumption among economistshas been thateconomicagentsface problemsof
choice underuncertainty.I arguethatidentification
problemsmakeambiguity a fundamental
problem
of treatmentchoice in practice.Althoughempiricalevidenceon realized treatments
and outcomes
does implyinformative boundson meanresponsesunderalternative theseboundsneces-
treatments,
sarilyoverlap.Hence, observationsof realized treatments
and outcomesdo not sufficeto rankthe
feasibletreatment
rules.
The factthattheempiricalevidencedoes notenable thedetermination of theoptimaltreatment
rule does not implythata plannershould be paralyzed,unwillingand unable to choose a rule.
However,it does implythattheplannercannotassertoptimality forwhateverrulehe does choose. A
plannerwho actsas a subjectiveBayesianor who uses themaxi-minrulemaysensiblyassertthathe is
using a "reasonable" decisionrule,but he shouldnot assertthathe is usingan optimalrule.
18 CharlesF. Manski

in EverydayLife
Ambiguity
Ordinarypeople face ambiguityas do social planners. Manski (2003a) analyzes social
interactionsthatstemfromthesuccessiveendeavorsofnew cohortsofheterogeneous decisionmakers
to learnfromthe experiencesof past cohorts.A dynamicprocessof information accumulationand
decisionmakingoccursas the membersof each cohortobservethe experiencesof earlierones and
then make choices that yield experiencesobservable by futurecohorts.Decision makers face
theselectionproblemas theyseek to learnfromobservationof past actionsand outcomeswhilenot
observingthe counterfactual outcomesthatwould have occurredhad otheractionsbeen chosen.
the
Under assumption that all cohortsfacethesame outcomedistributions,I showthatsocial learning
is a processof sequentialreductionin ambiguity.The specificnatureof thisprocess,and itsterminal
state,dependcriticallyon how decisionmakersmake choices underambiguity. I use theproblemof
learning about innovations to illustrate.

ofMarketDemand
4. Identification

The Law ofDecreasing Credibility

Determiningwhat can be learnedusing the data alone providesa logical startingpoint for
empiricalanalysisbutordinarily willnotbe theendingpoint.Havingdetermined whatcan be learned
in theabsence of assumptions, we shouldthenask whatmorecan be learnedif assumptionsof dif-
ferentstrengthsand degreesof plausibilityare imposed.As a methodologist, I have not advocated
thatempiricalresearchers assumptionor another.My objective
or social plannersmakeone particular
wantedto clarifythedilemmathat
has been to providea menuof possibilities.I have particularly
researchersand plannersfaceas theydecidewhatassumptions to maintain.I have calledthisdilemma
(Manski 2003b) theLaw of Decreasing Credibility. of inferencedecreaseswiththe
The credibility
strengthof theassumptionsmaintained.

Classical EconometricAnalysisofDemand
To demonstratethe Law of Decreasing Credibility,consider the most venerable of all
problemsin econometrics:the identification
identification of marketdemandfromobservationsof
marketequilibriums.Economic analyses of marketdemandusuallysuppose thatthereis a set of
isolatedmarketsfora givenproduct.Each marketis characterized by a demandfunction,whichgives
the quantityof productthatprice-taking consumerswould purchaseif the price were set at any
specifiedlevel. In each market,theinteractionof consumersand firmsdeterminesthepriceat which
transactionsactually take place.
The classicaleconometric analysisof marketdemandachievesidentification
through twocritical
assumptions.One is thatdemandvarieslinearlywithprice,withthesame slope in everymarket.The
otheris thatdemandis mean-independentof an observedcovariate,termedan instrumental variable.
Considertheseassumptions.Economictheorydoes notsuggestthatdemandshouldbe linearin
priceor,indeed,thatdemandshouldbe anyparticularfunction ofprice.All thateconomictheorydoes
suggestis thatdemandshouldbe downwardslopingin price.Of course,even thisis not a universal
prediction;forexample,textsregularlynote thepossibilityof Giffengoods. However,the ordinary
presumption of economistsis thatdemand is downwardsloping in price. It certainlyis not that
demandis linearin price.
 AssociationLecture 19

Economic theorydoes not suggestthatdemandshouldbe mean-independent of any particular

covariate,so the credibilityof this assumptionmustbe assessed on a case-by-casebasis. The as-
sumptionoftenis suspectin practice.Empiricalresearchers
regularlydebatewhethersomecovariateis
or is not a "valid instrument."

Monotone TreatmentResponse

Concernedthatclassicaleconometric analysisis builton fragilefoundations,

I have,in a seriesof
papers,studiedwhatmaybe learnedaboutmarketdemandwhenweaker,morecredibleassumptions
are imposed.
Manski (1995, 1997) investigatedwhatmay be learnedwhenone assumesonlythatdemandis
in
downwardsloping each market.My analysisassumed nothingelse about the shape of demand
functionsand,moreover,assumednothingabouttheprocessof pricedetermination. The basic idea is
thatobservationof an equilibrium(quantity,
price)pairimpliesthatsome downwardslopingdemand
functionpasses throughthispoint.Aggregatingacross marketsyield bounds on the distribution of
demand functions.These bounds reveal what basic economic theoryimplies forthe econometric
analysisof marketdemand.
In the language of the analysis of treatment response, downwardsloping demand is an
assumption of monotone treatment response.In this language,prices are treatments,quantityde-
manded is an outcome,and the demandfunctionis a responsefunctionmappingtreatments into
outcomes.So theeconometricanalysisof marketdemandis an instanceof theanalysisof treatment
response.
The assumptionof monotonetreatment responseis crediblein many applicationsotherthan
analysisof market demand.Another important economicapplicationis to productionanalysis.There,
outputsare outcomes,and theproductionfunctionis theresponsefunction.
inputsare thetreatments,
In thiscase, theassumptionthatresponsefunctionsare monotonemeans simplythatoutputweakly
increaseswithinputs.An additionalcredibleassumptionin productionanalysismaybe thatresponse
functionsare concave,so thatincreasesin inputshave diminishingmarginalreturns. Manski (1997)
powerof thisassumption,as well as thatof monotonicity.
studiedthe identifying

Using InstrumentalVariables
Justas it may enhance credibilityto replace the assumptionof linear demand with one
of downwardslopingdemand,it may also enhancecredibility to use instrumental
variableswithout
making accompanyingassumptions about the form of treatment response.Manski (1990, 1994)
showed that,if outcomes are bounded variables,the mean-independence assumptioncommonly
assumed in demandanalysisand elsewheregenerallydoes not point-identitytreatmentresponsebut
does yield informative
bounds on mean treatment response. Manski (2003b) presentsstronger
findingswhen one imposesa statistical-independence
assumptionratherthana mean-independence
assumption.These boundsareeasy to implement. Theyare intersections, values ofthe
acrossdifferent
instrumentalvariable,of theboundsthatapplywhentheempiricalevidenceis used alone. Thus, an
empiricalresearcherstudying marketdemandcan use an instrumentalvariablewithoutalso havingto
how demandvarieswithprice.
restrict
Manski and Pepper(2000) studiedthe identifyingpower of monotoneinstrumental variables.
These weakentheequalitydefininga traditional
mean-independence assumptionto a weak inequality,
yieldinga new assumptionthatmay be crediblewhenmean independenceis not.
20 Charles F. Manski

Consider, for example, the continuingeffortby labor economiststo learn the returnsto
schooling.Many empiricalarticlesestimateregressionsof wages on schoolingand interpret the
findingsas estimatesof thereturns is appropriate
to schooling.As is well known,thisinterpretation
only if schooling is exogenous,in the sense thatwage functionsare mean-independent of chosen
levels of schooling.However,manyeconomicmodels of schoolingchoice and wage determination
predictthatpersonswithhigherabilityhave highermean wage functionsand tendto choose higher
levels of schoolingthando personswithlowerability.So theassumptionof exogenousschoolingis
suspect.However,itmaybe credibleto assumethatpeople who selecthigherlevelsof schoolinghave
weaklyhighermean wage functions thando thosewho selectlowerlevels. This assumptionmakes
level of schoolinga monotoneinstrumental variable.

Problems
5. Other Identification

I have dwelled on the selectionproblem-thatis, the unobservability of counterfactual out-

comes.This is an especiallyimportant and distinctive
identification
problem.However, it is hardlythe
only one confronted in social science and in everydaylife.
Mundaneproblemsof missingoutcomeand covariatedata ariseregularlyin empiricalresearch.
It has been commonto "solve" theproblemby assumingthatdata are missingat random,but this
assumptionis rarelyjustifiable.As whenconfronting counterfactual
outcomes,it is important to know
whatcan be learnedfromtheavailableempiricalevidencealone. Aspectsof thisproblemhave been
studiedin Manski(1989, 1994), Horowitzand Manski(1998, 2000, 2001), and Zaffalon(2002). The
relatedproblemof missingtreatment data in theanalysisof treatment
responsehas been studiedby
Molinari(2002a). Anotherrelatedproblemis inferencewithinterval-measured data (Manski and
Tamer2002; Haile and Tamer2003).
Missingdata,as problematic as theymaybe, pose less severean identificationproblemthando
contaminated data. In contaminated samplingproblems,theobservabledata are a mixtureof "good"
observationsdrawnfromthedistribution of interest
and "bad" observationsdrawnfromsome other
distribution.The researcherdoes notknow whichdata are good and whichare bad. Aspectsof this
problemhave been studiedin Horowitzand Manski (1995), Bollinger(1996), Hotz, Mullin,and
Sanders(1997), Cross and Manski(2002), Dominitzand Sherman(2002), Kreiderand Pepper(2002),
and Molinari(2002b).
Identificationproblemspervadeeveryaspectofempiricalresearchand everyattempt byordinary
to
people learn about the worldin whichtheylive. Overall,I view myselfas presenting a mixed
message.My pessimisticside would arguethatit is rarelypossibleto credibly"solve" identification
problems.The optimistin me would arguethatthemorewe understand theseproblems,thebetterwe
will be able to cope withthem.

References
Bollinger,Christopher.1996. Bounding mean regressionswhen a binaryvariable is mismeasured.Journalof Econometrics
73:387-99.
Cross, Philip,and Charles Manski. 2002. Regressions,shortand long. Econometrica70:357-68.
Dominitz,J.,and R. Sherman.2002. Nonparametric analysisofmixturemodelswithverification,
withan applicationto testscore
data. Unpublishedpaper,CarnegieMellon University.
Ellsberg,D. 1961. Risk, ambiguity,and the Savage axioms. QuarterlyJournalof Economics 75:643-69.
 AssociationLecture 21

Haile, Philip,and Elie Tamer. 2003. Inferencewithan incompletemodel of English auctions.Journalof Political Economy.
111:1-51.
Horowitz,Joel,and Charles Manski. 1995. Identificationand robustnesswithcontaminatedand corrupteddata. Econometrica
63:281-302.
Horowitz,Joel,and CharlesManski. 1998. Censoringof outcomesand regressorsdue to surveynonresponse:Identification and
estimationusing weightsand imputations.Journalof Econometrics84:37-58.
Horowitz,Joel, and Charles Manski. 2000. Nonparametricanalysis of randomizedexperimentswith missingcovariateand
outcomedata. Journalof theAmericanStatisticalAssociation95:77-84.
Horowitz,Joel,and CharlesManski. 2001. Impreciseidentificationfromincompletedata. Proceedingsof the2nd International
Symposiumon Imprecise Probabilities and Their Applications. Accessed 13 December 2002. Available http://
ippserv.rug.ac.be/--isipta01/proceedings/index.html.
Hotz, V. Joseph,Charles Mullin, and Seth Sanders. 1997. Bounding causal effectsusing data froma contaminatednatural
experiment:Analyzingthe effectsof teenagechildbearing.Reviewof Economic Studies64:575-603.
Koopmans,Tjallings. 1949. Identificationproblemsin economic model construction. Econometrica17:125-44.
Kreider,Brent,and JohnPepper.2002. Disabilityand employment:Reevaluatingtheevidence in lightof misreporting errors.
Unpublishedpaper,Universityof Virginia.
Manski,Charles. 1989. Anatomyof the selectionproblem.Journalof Human Resources 24:343-60.
Manski, Charles. 1990. Nonparametricbounds on treatment effects.AmericanEconomic Review Papers and Proceedings
80:319-23.
Manski, Charles. 1993a. Identificationof endogenous social effects:The reflectionproblem.Review of Economic Studies
60:531-42.
Manski, Charles. 1993b. Adolescenteconometricians:How do youthinferthe returnsto schooling?In Studies of supplyand
demandin highereducation,editedby CharlesClotfelter and Michael Rothschild.Chicago: Universityof Chicago Press,
pp. 43-57.
Manski,Charles. 1994. The selectionproblem.In Advancesin econometrics, SixthWorldCongress,editedby Christopher Sims.
Cambridge,UK: CambridgeUniversityPress,pp. 143-70.
Manski,Charles. 1995. Identificationproblemsin thesocial sciences. Cambridge,MA: HarvardUniversityPress.
Manski,Charles. 1997. Monotonetreatment response.Econometrica65:1311-34.
Manski, Charles. 2000. Identification
problemsand decisions underambiguity:Empiricalanalysisof treatment responseand
normativeanalysisof treatment choice. Journalof Econometrics95:415-42.
Manski,Charles.2002. Treatment choice underambiguityinducedby inferential problems.JournalofStatisticalPlanningand
Inference105:67-82.
Manski,Charles.2003a. Social learningfromprivateexperiences:The dynamicsof theselectionproblem.ReviewofEconomic
Studies.In press.
Manski,Charles. 2003b. Partial identification New York: Springer-Verlag.
ofprobabilitydistributions.
Manski, Charles,and Daniel Nagin. 1998. Boundingdisagreementsabout treatment effects:A case studyof sentencingand
recidivism.Sociological Methodology28:99-137.
Manski, Charles,and JohnPepper. 2000. Monotone instrumental variables: With an applicationto the returnsto schooling.
Econometrica68:997-1010.
Manski, Charles,and Elie Tamer. 2002. Inferenceon regressionswithintervaldata on a regressoror outcome.Econometrica
70:519-46.
Molinari,Francesca. 2002a. Missing treatments. Unpublishedpaper,Northwestern University.
Molinari,Francesca.2002b. Identificationof probability
distributionswithmisclassifieddata. Unpublishedpaper,Northwestern
University.
of missingdata. Journalof StatisticalPlanningand Inference105:105-22.
Zaffalon,Marco. 2002. Exact credal treatment