You are on page 1of 5

Issue Brief

TranslatingResearchtoPolicy

November2007/Issue12

StateHealthInsuranceCoverageEstimates:
AFreshLookatWhyStateSurveyEstimatesDifferfromCPS

Introduction
TheU.S.CensusBureauproducesstatelevelestimatesofthedistributionofhealthinsurancecoverageon
anannualbasis.1AlthoughtheCensusBureausestimatesofcoverageareoftencitedinacademicand
policycircles,analystssuspectthattheyoverstateratesofallyearuninsuranceandunderstateMedicaid
enrollment.Foravarietyofreasons,manystateanalystsprefertocollectandmakeuseoftheirown
surveydata.Over40stateshaverecentlyconductedtheirownpopulationsurveystogetstatelevel
estimatesofhealthinsurancecoverage.2

StatesurveyestimatesofuninsurancearetypicallylowerthantheestimatesproducedbytheCensus
BureausCurrentPopulationSurvey,AnnualSocialandEconomicSupplement(CPS).These
discrepanciesfuelthedebateaboutthenumberofuninsuredandmaythreatenthevalidityand
usefulnessofsurveydatatoinformpolicydecisionsaroundaccesstohealthcareandinsurance.Inthis
briefwecomparestatesurveyandCPSestimatesofuninsurance,analyzefactorswiththegreatest
potentialtoexplainthesedifferences,anddiscussthepolicyimplicationsofthispersistentdiscrepancy.

ComparisonofEstimatesDerivedfromStateSurveysandtheCPS
WecomparedCPSallyearuninsuranceestimatestoestimatesofuninsuranceatthetimeofthesurvey
(pointintime)across24states,15ofwhichalsoproduceallyearestimates.Figure1comparesstate
uninsuranceestimatesforfourgeographicallydispersedstateswithestimatesfromtheCPSthreeyear
averageuninsuranceestimates.3Forthiscomparisonweselectedstatesinwhichthesurveyprovides
bothpointintimeandallyearestimatesofuninsurance.4

Figure 1: Comparison of State Survey Estimates and Current Population Survey Estimates of Uninsurance
20%

15%

13.5%
11.2%

10%

State Survey Point-intime Estimate

13.4%
10.8%

10.2%

10.4%

10.2%

8.8%
7.4%

6.7%

6.1%
4.0%

5%

State Survey All-year


Estimate
CPS 3-year Average
(all year)

0%
Alabama

Massachusetts

Utah

Wisconsin

Source: U.S. Census Bureau and states5.

State Health Access Data Assistance Center (SHADAC) | University of Minnesota School of Public Health
612-624-4802 | fax: 612-624-1493 | www.shadac.org

WefindthatstatesurveypointintimeestimatesarealmostuniformlylowerthanCPSallyearestimates.
Table1showsthat,onaverageamongthe24states,thestatesurveysspecifyapointintimerateof
uninsurancethatis22.4percentlowerthanthethreeyearallyearCPSestimate.Whencomparing
availableallyearuninsuranceratesfromstatesurveystotheCPS,estimatesareevenfurtherapart.State
allyearuninsuredestimatesare,onaverage,46.0percentlowerthanthethreeyearCPSrateof
uninsurance.

Table 1. Comparison of Average State Survey Point-in-time and 12-month Estimates


and Current Population Survey 3-year Average Estimates of Uninsurance

Average
Average State Survey Point-in-time Uninsurance Estimate
(average of 24 state surveys*)
Average State Survey All-year Uninsurance Estimate
(average of 15 state surveys**)
CPS 3-Year Average Uninsurance Estimate
(2002-2004)

10.8%
7.5%
13.8%

Difference Between Point-in-time and CPS as a Percent


of the CPS 3-Year Estimate

-22.4%

Difference Between All-year and CPS as a


Percent of the CPS 3-Year Estimate

-46.0%

Source: U.S. Census Bureau and states6.


* States include AL, AR, CA, CO, CT, FL, GA, IL, IN, KS, ME, MA, MN, MO, NH, OK, OR, PA, SD, UT, VT, VA, WA, WI
**States include AL, CA, CO, CT, GA, IN, MA, MN, MO, OK, OR, UT, VT, VA, WI

ReasonsforDifferences
Thereareseveralmethodologicalreasonswhystatesurveysproducelowerestimatesofuninsurancethan
thoseproducedbytheCPS.Thekeyreasonsfordiscrepanciesarediscussedbelow.

PopulationCoverageandSampleDesign
Moststatesurveysusetelephonesurveyrandomdigitdialing(RDD)tosamplehouseholds
withactivetelephonelinesonly.Bycontrast,theCPSusesanareaprobabilitydesign
drawingfromaddressframesthatincludehouseholdswithandwithoutactivetelephone
lines.7Approximately2.4percentofU.S.householdsdidnothavephonesin2000and
membersofthesehouseholdsweremuchmorelikelytobeuninsured.8Relyingexclusively
ontelephonesamplingmaybiasestimatesdownward.

Anemergingpopulationcoverageissueforstatetelephonesurveysisthegrowthofexclusive
cellphoneonlyhouseholds.9In2005itwasestimatedthat6.7percentofhouseholdswere
cellphoneonlyhouseholds,asignificantincreasefrom4.5percentin2004.10Theestimate
increasedto9.6percentin200611;whenconsideringthe2.4percentofhouseholdswithout
anyphone,thisleavesapproximately13percentofthehouseholdsinaccessibleusingRDD.

TheCPSincludescellphoneonlyhouseholdsintheirareaprobabilitydesign,whereasthe
stateRDDsamplesusuallypurgecellphonenumbers.Evidencesuggeststhatpeopleincell
phoneonlyhouseholdsaresignificantlymorelikelytolackhealthinsurancecomparedto
peoplewithlandlinetelephoneservice.12TheexpectationisthatstateRDDsurveyswillhave
loweruninsuranceratesasaresult.

QuestionsMeasuringHealthInsuranceCoverage
RespondentsintheCPSareaskedabouttheirinsurancestatusinFebruary,MarchorApril
forthepriorcalendaryear(JanuarytoDecember)inordertomatchthereferentperiodfor
incomeandemploymentquestions.Thismeansrespondentsmusttrytorecallinsurance
coverageforaperiodthatbegan14to16monthspriortotheinterview.Thelongrecall
periodmaydecreasetheaccuracyofcoveragereports.13Bycontrast,statespecificsurveys
askabouttherespondentsinsuranceatthetimeofthesurvey(inaddition,someaskabout
coverageovertheprioryear),whichincreasesthelikelihoodofaccuratereporting.14

TheCPSisalaborforcesurveythataddsasupplementattheendwhichincludeshealth
insurancequestions.Bycontrast,manyofthestatesurveysfocusspecificallyonhealth
insurance,placingtheseitemsclosetothebeginning.Statesurveyshaveanadvantageover
theCPS,asthehealthrelatedcontentfocusestherespondentsattentioninawaythatmay
enhancetheaccuracyofreportsabouthealthinsuranceirrespectiveoftheirplacementinthe
overallsurvey.

NonResponseBias
TheCPScontinuestoenjoyhighresponseratesofabout84percent.15RDDsurveyshave
experiencedasignificantdeclineinresponseratesoverthepast10years.16Forexample,
recentresponseratesfortheBehaviorRiskFactorSurveillanceSystem(BRFSS)survey,a
decentralizedtelephonesurveyconductedbystates,hoveraroundthe50percentrange,
whereasratesforgeneralpopulationtelephonesurveysinthelate1980sweretypicallyinthe
vicinityof70percent.17SimilardownturnshavebeenobservedintheUniversityof
MichigansSurveyofConsumerAttitudeswheretheresponserateshavedeclined
approximately1.5percenteveryyearsince1996,sothatby2003theresponseratewas48
percent.18However,thereisnoevidencetosuggestthatlowerresponserateshaveledto
biasedhealthinsuranceestimates.
DataProcessing
Dataprocessingpriortoestimationofcoverageratesmayalsoaccountforsomeofthe
discrepancybetweenestimates.Lessthanthreepercentofcaseshavemissingdataonhealth
insuranceitemsinmoststatesurveys;therefore,thesedataareseldomimputedand
estimatesaremadefromcompletecasesonly.19Imputationisadataprocessingtechniquefor
dealingwithitemnonresponse;i.e.,asurveyresponsethatdoesnothaveallitems
answered.Mostofthetechniquesuseinformationfromthecompletedcasestoimputea
modelbasedestimatetothecaseswithmissingdata.

Bycontrast,theCPSdataarefullyimputedandeditedwith13percentofthesamplemissing
healthinsurancedata,mostlyforthosewhorespondtothemonthlyCPSsurveybutrefuseto
taketheASECsupplement.Thestatisticalmethodusedtoimputemissinghealthinsurance
datacreatesbiasinstateestimatesofcoveragebecausestateisnotoneofthevariablesin
themodel.20Inaddition,recentresearchshowsthattheCensusBureausmethodfor
imputinghealthinsuranceleadstoanundercountofpeoplewithemployersponsored
coverageandanovercountofpeoplewhoareuninsured.21

ConclusionsandPolicyImplications
Estimatesofuninsurancefromstatespecificsurveysarealmostconsistentlylowerthanratesfromthe
CPS.FundamentaldesignandadministrationfeaturesofstatesurveysandtheCPScontributeto
persistentdifferencesinstateestimatesofuninsurance.

Withregardtopopulationcoverage,sampledesignandnonresponsebias(e.g.,householdslacking
telephones,increasesincellphoneonlyhouseholds,decliningresponserates),theCPShastheadvantage
inthatitsareaprobabilitysampledesignincludespeoplelivinginhouseholdswithandwithoutactive
telephonelines.StatestypicallylacktheresourcestodoinpersonsurveysandmustrelyonRDD
telephonesurveys.

Statesurveyshavetheadvantageintheareaofmeasurementinthattheytypicallyfocusonhealth
insuranceandaccesstocare,askaboutcurrenthealthinsurancecoverage,andstatesurveyanalystshave
greaterflexibilitytoaltertheirinstrumentstoaccommodatechangesinthelocalinsuranceofferingsand
statespecificnamesofpublicprograms(e.g.MinnesotaCare,BadgerCare).Bycontrast,theprimary
purposeoftheCPSistocollectinformationaboutlaborforceparticipationandquestionsabouthealth
insurancecoveragemustadheretothetimeframe(calendaryear)inwhichthelaborforcequestionsare
asked.

Bothsourcesofstatespecificestimatesofinsurancecoveragehavemeritandlimitationsmakingthem
morerelevantforspecificpurposes.Contrastingtrendsinthenumberandcharacteristicsofthe
uninsuredacrossstatesrequiresuseoftheCPS.Ifinterestedinspecificinformationaboutsubpopulations
ofuninsuredwithinagivenstate,statesurveysarepreferredbecauseoftheirlargersamplesizeandthe
abilitytoalterthesampledesigntofocusonsubpopulationsofinterest.

AlthoughtheestimatesderivedfromtheCPSandstatesurveysareconsistentlydifferent,ongoing
researchisinvestigatingwhetherthetrendsintheestimatesaresimilarovertime,tellingacoherentstory
aboutthedistributionofinsurancecoverageandcharacteristicsoftheuninsured.Policyanalystshave
muchtogainindrawingonbothsourcesofdatatoinformdecisions.

Finally,itisimportanttorecognizethatthenumberofuninsuredwillneverbeexactlypinneddown.
Differentsurveysproducedifferentestimatesbecauseofthemanycomplexchoicesinvolvedincollecting
healthinsurancedata.Whileresearchshouldfocusonproducingbetterestimates,thenumberremains
justthat:anestimate.

Acknowledgements
PreparationofthisissuebriefwassupportedbyfundingtotheStateHealthAccessDataAssistance
CenterfromtheU.S.DepartmentofHealthandHumanServicesOfficeofTheAssistantSecretaryfor
PlanningandEvaluation,theAgencyforHealthcareResearchandQualityandtheRobertWoodJohnson
Foundation.Inquiriesmaybedirectedto6126244802orshadac@umn.edu.

ThisissuebriefisanupdateofanearlierSHADACissuebrieffrom2001,StateHealthInsurance
CoverageEstimates:WhyStateSurveyEstimatesDifferfromCPS.AllSHADACissuesbriefsare
availableonlineatwww.shadac.org.

State Health Access Data Assistance Center


Minneapolis, MN: University of Minnesota
Issue Brief #12; November 2007

Notes

C.DeNavasWalt,B.D.Proctor,andC.H.Lee,Income,Poverty,andHealthInsuranceCoverageintheUnited
States:2004,CurrentPopulationReports,P60229.(WashingtonDC:USCensusBureau,2005).

L.ABlewettetal.,MonitoringtheUninsured:AStatePolicyPerspective,JournalofHealthPolitics,PolicyandLaw
29,no.1(2004):10745;andStateHealthAccessDataAssistanceCenter(SHADAC),SummaryofHousehold
PopulationSurveysConductedbyStates,Workingpaper(MinneapolisMN:UniversityofMinnesota,2006).

TheuseofthreeyearCPSaveragesisintendedtoincreasetheprecisionofuninsuranceestimatesthatareotherwise
morevariablefromyeartoyear.R.J.MillsandS.Bhandari,HealthInsuranceCoverageintheUnitedStates:
2002,Reportno.P60223(WashingtonDC:USCensusBureau,2003).

ForthecompleteanalysisseeK.T.Call,M.Davern,L.A.Blewett,EstimatesofHealthInsuranceCoverage:
ComparingStateSurveystotheCurrentPopulationSurvey.HealthAffairs26,no.1(2007):26978.

U.S.CensusBureau,Table11:Income,PovertyandHealthInsuranceintheUnitedStates:2004.Exceptionsare
estimatesfortheunderage65populationforCA,FL,KS,NJ;SHADACanalysesofmicrodata(August2005
release).Withtheexceptionofsourceslistedhere,statespecificsurveyestimatesarefromHRSASPGreports
foundathttp://www.statecoverage.net/hrsa.htm:CaliforniaHealthInterviewSurvey/AskCHIS2.0.;Florida
HealthInsuranceStudy.InsuranceCoverageUpdates.FactSheetNo1;April2005;2004MinnesotaHealthAccess
Survey;2004MissouriHealthCareInsuranceandAccessSurvey,February2005;2004OklahomaHealthCare
InsuranceandAccessSurvey:SelectResults.OklahomaHealthCareAuthority.April2005;TheHealthInsurance
StatusofPennsylvaniasStatewideSurveyResults,May2005;2004UtahHealthStatusSurveys,UtahDepartment
ofHealth;WisconsinDepartmentofHealthandFamilyServices.WisconsinHealthInsuranceCoverage,2003,
September2004.AlsorefertoK.T.Call,M.Davern,L.A.Blewett,EstimatesofHealthInsuranceCoverage:
ComparingStateSurveystotheCurrentPopulationSurvey.HealthAffairs26,no.1(2007):26978.

Ibid.

U.S.CensusBureau,CurrentPopulationSurveyTechnicalPaper#63,Reportno.TP63RV(WashingtonDC:US
CensusBureau,2002)

M.Davernetal.,TelephoneServiceInterruptionWeightingforStateHealthInsuranceSurveys,Inquiry41,no.3
(2004):28090.

S.J.Blumberg,J.V.Luke,andM.L.Cynamon,Telephonecoverageandhealthsurveyestimates:evaluatingtheneed
forconcernaboutwirelesssubstitution,AmericanJournalofPublicHealth96,no.5(2006):92631;andC.Tucker,
M.Brick,andB.Meekins,TelephoneserviceinU.S.householdsin2004,presentationattheAmerican
AssociationforPublicOpinionResearchAnnualMeeting,PhoenixAZ,May2004.

Blumbergetal.,Telephonecoverageandhealthsurveyestimates:evaluatingtheneedforconcernaboutwireless
substitution.

10

Ibid.

11

Ibid.

12

S.Sudman,N.Bradburn,andS.Schwarz,Thinkingaboutanswers(SanFrancisco:JosseyBass,1996).

13

Ibid.

14

U.S.CensusBureau,CurrentPopulationSurveyTechnicalPaper#63,Reportno.TP63RV(WashingtonDC:US
CensusBureau,2002).

15

R.Curtin,S.Presser,andE.Singer,Changesintelephonesurveynonresponseoverthepastquartercentury,
PublicOpinionQuarterly69,no.1(2005):8798;andPewResearchCenter,Surveyexperimentshows:Pollsface
growingresistance,butstillrepresentative,(WashingtonDC:ThePewResearchCenterforthePeopleandthe
Press,2004).

16

R.M.Grovesetal.SurveyMethodology.(NewYork:Wiley,2004).

17

R.Curtin,S.Presser,andE.Singer,Changesintelephonesurveynonresponseoverthepastquartercentury.

18

K.T.Call,etal.TheMedicaidUndercountandBiastoEstimatesofUninsurance:NewEstimatesandExisting
EvidenceHealthServicesResearch,forthcoming(2008).

19

M.Davernetal.MissingtheMark?ExaminingImputationBiasintheCurrentPopulationSurveysStateIncome
andHealthInsuranceCoverageEstimates.JournalofOfficialStatisticsSep2004:20(3):51949.

20

M.Davern,DoesImputationBiasLeadtoMoreUninsuredintheCurrentPopulationSurveysEstimates?
PresentationattheAcademyHealth2005AnnualResearchMeeting,BostonMA,26June2005.

21