You are on page 1of 98

CE 451 - Urban Transportation Planning and Modeling

Iowa State University

Calibration, Adjustment and Validation
Sources:

Calibration and Adjustment of System Planning Models
Note: Date = 1990 (need to adjust for inflation, other changes)

Model Validation and Reasonableness Checking Manual

NHI course on Travel Demand Forecasting (152054A)

Objectives:

• Identify and interpret trends affecting travel demand
• Explain difference between calibration and validation
• Identify critical reasonableness checks
– socioeconomic
– travel survey
– network
– trip generation
– mode split
– trip assignment

Terminology
• Model Calibration
– Estimate parameters
– Match observations (OD, AADT)
• Model Validation Is the model
sensitive to policy
– Reasonableness checks
options?
– Sensitivity checks
• Special generators
• Screen lines, cut lines, cordons

readysetpresent. Planner responsibilities • Actively involve all participants – Modelers – Planners – Decision makers – Public • Fairly present all alternatives – Timely – Unbiased • Identify (clearly) the decision making process – Who.com . when. and how – Allows input from all interested groups • You must rely on the TDM – Therefore. must be validated – Accurate and easy to understand (documented) www.

com • Flexibility in application • Types of available outputs • Operational costs • Experience and successes to date • Public or private domain availability . How do you judge a model/recommend improvement? Scrutinize these characteristics: • Data requirements • Logic of structure and conceptual appeal • Ease of calibration • Effectiveness of the model (accuracy. sensitivity) cio.

edu.gov. transportation1. Trends Affecting Travel Demand • Planners should monitor the following trends: – Demographics – Composition of the labor force – Immigration and emigration Must be aware of – Regional economic development trends to ensure – Modal shares reasonable forecasts – Vehicle occupancy – Average trip length – Freight transport • Are trends consistent with assumptions made in the modeling process? Image sources: scu. uwex.gov.org. usda.edu.edu. mwcog. fhwa. illinois.org .

doe.pdf .gov/oiaf/aeo/pdf/trend_4.eia.doe.gov http://www.

.

g. customer • Make sure road attributes are addresses. e. Caliper Corp. traffic • Use trip chaining (tour based) and • Use hourly counts activity based trip generation • Income and auto ownership don’t • We don’t know much about fully explain travel attractions – ITE sample too small – • age. life cycle and personal do your own interest come into play • Drive the network using GPS • Use survey data • Get some data and do some statistics – visualize these data to derive your parameters – Survey can be done cheaply – Cooperation will be good if there’s a good reason for it – mayor sends letter. esp. gender. license plates correct. 3/13/04 peer review . Tips for building a good model* • Build accurate road network • Employer based surveys get good • Use aerial photos behind response (but may be biased) – Some will give home addresses. *Howard Slavin.

3/13/04 . Tips for building a good model* • Some models are completely made • Examine individual links after model up except traffic counts run – See if you really believe the counts – Where are the trips coming from and going to that use the link? • Create your OD matrix from ground – In TransCAD. what is the process used to “made up” the whole model (no surveys) show where traffic from a particular zone – TransCAD has a tool for this is going to? – If still want to use trip gen/dist. this • Familiarity with your region is helpful method can be used to determine K factors – Could also use the row and column totals as the dependent variables in your trip gen model *Howard Slavin. what is the process used to counts determine this (for a particular link)? – May be better than trip gen/dist if you – In TransCAD. Caliper Corp.

wrong • Specification variables • Data Transfer • Data aggregation . Sources of Error • Coding • Sampling • Computation (if done by hand) Improper structure of model.g. e..

Key Concepts • Not enough attention on model evaluation and reasonableness checks • Checks should be performed after each step – reduces error propagation Errors can also “cancel” .

.

Evaluation and Reasonableness Checks Overview Number and location Complete? Transportation of households and Level of Detail? employment (demand) Reasonable? system Methodology? ►Socioeconomic Data (supply) Source? ►Network Data TDF Travel ►Model Specification survey data ►Model validation and calibration Sensitive? Transportation Documentation of system Current? calibration? performance Reasonable? Valid for base year? .

or borrow structure and parameters from a "similar" area Model Application VALIDATION is checking if the model is accurately estimating traffic volumes by calculated measures (like RMSE) . Model development is sometimes called calibration or estimation as we are estimating parameters and Model Calibration constants for the particular model structure. estimating is a statistical process … want high correlation coefficients and Model Validation significant parameter values can "import" a model . CALIBRATION and VALIDATION are sometimes confused.

return to calibration if not • Validation of a previously calibrated model – Compare to a new base year. Model Validation • Validation of new model – Model applied to complete model chain – Base year model compared to observed travel – Judgment as to model suitability. with new … • SE data • Special gen. • Network • Counts “Transportation Conformity Guidelines” (Air Quality) require model validated < 10 years ago .

need a survey.(long lines.g.) .compare traffic counts across … .Validation suggestions . fine tuning) if "importing" should validate all borrowed parameters and constants . local knowledge of commute patterns helps .. e. CBD.Cordon lines (surround a major generator. verify corridor flows.Screenlines . university..Cutlines (shorter. check major flows) check trip interchange (distribution) between large sections or quadrants .Systemwide .

com .IT IS VERY IMPORTANT TO HAVE A GOOD COUNT PROGRAM DESIGNED TO SUPPORT VALIDATION! iowadotmaps.

1%) The Calibration and Adjustment manual is not intended to replace good OD data. if any cities have developed OD databases since 1980.To "calibrate" the model. This is time consuming and expensive.g. Few. . (and has some old data in it! – more recent data area available in the Barton-Ashman publication). but many have updated old ones since then using a small survey (e. need an OD database from a survey. and is intended more for small urban areas.

trips/person. parameters or variables according to chapters on: .Calibration and Adjustment Steps: 1) verify network and socioeconomic data 2) run the model 3) develop region-wide values (e. combination) 8) modify one or more equations.auto occupancy .trip generation .traffic assignment . vmt/person) 4) compare region wide values with “Appendix A” values 5) develop screenlines and cutlines 6) compare model results with ground counts for crossings 7) determine problems (system level. local.networks .trip distribution .g.

Other chapters focus on: .transit .conclusions .expected vs. required accuracy .trouble shooting . local changes .system vs.external stations .

Network Data Reasonableness Checks • Check Trees for 2-3 major attractions* • Check coded facility types – how used (BPR?)? • Verify speed and capacity look-up table (what LOS used for capacity?)* – Speed adjust (can lower the freeway speed if it is being overloaded – tweak?) • Significant transportation projects – narrative included? Still viable? • Consistency with MTP • Plot (facility types. # lanes. speeds. area types) to detect coding errors* * Items we can check in labs .

Network Errors 2.represent local streets .1 Centroid Connectors .not connected to intersections .check access (all 4 sides?) . etc.Details 2.) .make sure they are not blocked by a physical barrier (river.

Des Moines Model Capacity Look-up Table .

Des Moines Model Capacity Look-up Table (cont.) .

) .Des Moines Model Capacity Look-up Table (cont.

Des Moines Model Capacity Look-up Table (cont.) .

3 Intersection Penalties (check them!) .more important in sub-area modeling .speed volume function .can do it manually for small networks -check for circuity (correct with small turn penalties!) .account for congestion .turn penalties .2.can include delay on approach links .most congestion here .

-See TransCAD Manual B “Chapter 10: Traffic Assignment with Volume Dependent Turning Delays” .

4 Intrazonal times • increasing intrazonal trips (in distribution) decreases interzonal trips (useful if too many trips are being loaded on the network) •number of trips is a function of travel time (gravity model) -can adjust travel time on intrazonals -can adjust friction factor curve to produce more shorter trips (which intrazonals usually are) -can change definition of zones (size.2. land use) •Air quality analysis implications??? .

socioeconomic data can be a source of error .initial step is to check system trip totals.still a problem?.1 Trip generation . check production/attraction rates .3. compare w/ Table 4 and A1 and A2 (next pages) .if there is a problem. check the system number of dwelling units .

116 • Average Household Size 2.99 Colorado Springs 1996 Travel Demand Model Calibration .800 • Population per employee 1.26 • Person trips per household 10.44 • HBW productions per household 1. Trip Generation Calibration Reasonableness checks – compare to other cities.65 • HBW attractions per employee 1.345 • Households 201.81 • Person trips per person 4.465 (24%) • Service employment 101. check future trends • Population 503.697 (43%) • Military employment 42.74 • HB shopping attractions per retail employee 5.50 • Basic employment 76.795 (33%) • Retail employment 50.

.

Table A2 .

More recent data … From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

.

3.be sure you are using “real” dollars .2 Income .

borrowed. coordinate all of the above .3 P and A rates • Problems: old. but not for sub-areas • check system-wide values (see tables. but can apply occupancy factor and check against vehicle rates (ITE) • later. 3. small survey • may work OK at the system level. screen line counts can be adjusted by varying trip generation rates (post assignment) • check cutlines and cordon counts . next pages) – raise or lower trip generation rates – Person trip or vehicle trip rates used? • we usually have person trip by purpose.

.

.

.

Non- home .

More recent data … Trip Generation Calibration Typical Values • Person trips per household: 8.3 • HBO person trips per household: 3.5 to 4.7 to 2.5 • HBW person trips per household: 1.9 • HBW trips: 18% to 27% of all trips • HBO trips: 47% to 54% of all trips • NHB trips: 22% to 31% of all trips .5 to 10.8 • NHB person trips per household: 1.7 to 2.

check the whole process.g. Trip Generation Reasonableness Checks • Examine trip production and attraction models – Form? – sensitivity? – IMPORTANT: keep parameters reasonable (e.. sometimes we may use ... • Check models for … – External-through and external-local trips – Truck trips • To calibrate trip generation and trip distribution. – default values from past surveys – very limited new surveys – census journey to work data (CTPP) .) • If you think you need to use unintuitive parameters.. don't use negative coefficients in regression models just because they provide the best fit.

985 Internal-external 0.908 Scale survey for participation class is a HBW high income 0.591 Colorado Springs 1996 Travel Demand Model Calibration .991 HB university 0.936 (relative purpose! HB elementary school 0.858 NHB other 0.895 HB shopping 0.861 income HBW upper middle income 0.875 NHB work-related 0.820 Truck 0.733 participation) HB high school 1.823 each HBW middle income 0. Examine trip purposes used … Use more trip purposes? TRIP PURPOSES Scaling Factor HBW low income 0.795 Note: HBW low-middle income 0.698 HB social-recreation 0.945 HB other 0.

Travel Survey Data
Reasonableness Checks
• Determine source of travel survey data
– Types of survey conducted
– Year of survey
• Scale survey for participation
• If no survey (borrowed)
– Check source of trip rates, lengths, TLFD
– Is area similar
• Geographic area?
• pop/HH/empl. characteristics?
• Urban density and trans system?
• Compare to similar regions and to same
region in earlier times:
– Person trip rates by trip purpose
– Mean trip lengths by trip purpose
• HBW longest? HBO shortest?
– TLFDs by trip purpose

Socioeconomic Data: Check
Reasonableness
• Review source for estimates and forecasts
• Visualize (plot) trends …
– Population and household size
– Household income
– automotive availability
– distribution of employment by type (basic, retail, service)
– employees per household and per capita … rate of increase is
decreasing
• Check future household and employment changes by zone

3.4 Special generators
-e.g. universities, airports, malls, ...
-Use ITE or survey

5 trip balancing factors 4. check your PA rates and socioeconomic data • NHB is usually out of balance • Automobile occupancy – by trip purpose? – Basis? – Constant? • see table 6 and A9 (next pages … are these still good?) .1.0 Auto occupancy • initially. if not. Ps and As should balance to should be 0. 3.9 to 1.

.

1 Mean Trip Length .recall: shape of curve affects trip length distribution -See below for effect of changing friction factors F Curve trips link vols.0 Trip Distribution 5.5. internal vols. F tt more long trips more short trips .

-varying trip length has a big impact on assigned volumes -portions of a friction factor table can be adjusted (more flexible than adjusting equations) .

.

19 HBSR t = 2.12 HBSh t = 8.1 NHB t = 0.20 where p is population SR = social/recreation Sh = shopping .18 x p.98 x p.5.2 Estimate Trip Length -compare average trip lengths (in minutes) by purpose to: HBW t = 0.63 x p.

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

Source: Virginia Travel Demand Modeling Policies and Procedures Manual .

Source: Virginia Travel Demand Modeling Policies and Procedures Manual .

Source: Virginia Travel Demand Modeling Policies and Procedures Manual .

mostly) problem: match low income households with low income jobs solution #1: disaggregate trip purposes by income quartile solution #2: use k-factors (trial and error) … yuk .3 Employment Distribution Problems (large cities.5.

schools (ignore if small %?) .trucks (calibrate with externals?) -Taxi normally. distortions are insignificant . other trip purposes .5.4 Special Treatment.

Trip Distribution Reasonableness Checks Examine … • Mean trip length (increasing or decreasing?) • TLFDs • Treatment of friction factors (same?) • Treatment of terminal times (logic?) • Treatment of K factors • Comparison with JTW trip length • Comparison with JTW sector interchange volumes or percentages. .

Calibrating Friction Factors .

29 1.04 32.65 4.91 1.96 25.0 9.91 0.85 11.29 7.75 8.0 16.000 0.5 9.5 5.100 0.95 22.17 0.700 0.5 16.49 1.500 0.0 14.00 12.0 15.06 0.13 0.900 1.950 2.29 17.16 0.100 1.80 9.92 2.84 1.80 19.000 0.500 1.93 1.5 15.95 27.5 17.5 13.100 30.300 1.87 25.0 8.76 … … … … … … .100 1.900 1.850 1.0 15.88 30.100 0.26 12.70 6.50 19.98 5.95 18.19 0.5 7. Calibrating a Gravity Model Adjusting Friction Factors Travel Times Observed Trip Input Gravity Adjustment New Friction Factors Ranges from Expanded from Friction Model Trips Factor Friction Skims Surveys Factors Observed Adjustment Gravity Factor x Friction Model Factor 2.500 0.400 0.000 1.100 0.90 14.09 15.67 10.250 1.900 1.200 0.150 0.300 0.0 11.50 16.20 17.250 0.87 20.300 1.

.

Trip Distribution Calibration and Validation • Check modeled vs.48 County 30-39 17.13 County 40-59 7.00 To Other Suburban 10.49 County 15-29 41.63 Central-Suburban 7.04 Suburban-Central 15.87 Central-Central 31.70 Within Suburban 32. household survey TLFD and mean trip lengths • Get HBW area-to-area flows from JTW HBW 1990 JTW TLFD and Area-to-Area Flows for Kansas City Commute Length in Journey-to-Work Minutes Percent Flows Percent < 15 27.11 .98 County >60 3.44 Work out of area 2.81 County Mean 21.

.

com/2009/08/bright-side-of-sitting-in-traffic. http://googleblog. But leave it to Google to make it really easy .blogspot. system optimality OD data are destroyed! (privacy) .maybe too easy. OD validation Using cell phone and/or GPS location to determine travel patterns is nothing new.html Adam Shell Office of Systems Planning Iowa Department of Transportation Link POA: price of anarchy (30%?) Nash equilibrium vs.

1 All or nothing .6.adjusting link speeds will change assigned volumes .initial speeds should be set to LOS C speeds (0.0 Traffic assignment 6.87 x free flow speeds) .

volume = f(time) .6.2 capacity restraint .adjust free flow time or c (capacity) to change volumes IF… THEN… Link Speed Travel Assigned Capacity Time Volume .final volume is average of all iterations or later iterations can be weighted more heavily .

6.87c) ultimate: LOS E (1.15(v/c)4 if defined as LOS E.00c) parameters differ depending on definition of capacity … if defined as LOS C.80(v/c)4 (see HCM) .2. 0. 0.1 definition of capacity design: LOS C (0.

.

adjust socioeconomic data or make direct P/A adjustments .7.38 = 1.If not using a transit network.0 Transit Ridership . increase auto occupancy by transit percentage (e. decrease trip production or attraction rates (one of them only. then balance) … if you use productions. modify productions or attractions by zone .get data from transit company . may not have to build a transit network .g. can vary mode split by income class 3.05.05 x 1.for small/medium cities.45) if transit percentage is 38% 2. then change to 1. can use the following method (if trip generation includes transit trips): 1. if auto occupancy is 1.

gov/publicati ons/journal_of_transportation _and_statistics/volume_08_n umber_02/html/paper_05/fig ure_05_03.html .bts. Mode Split Reasonableness Checks • Mode split model? • Form? • Variables included in the utility functions? • Coefficients logical? • Value of time assumptions • Parking cost assumptions • How do mode shares change over time? • Mode share comparisons with other cities http://www.

Mode Split Calibration and Validation • Experienced planning consultant required … – Form of LOGIT model – Variables included in utility functions – Calibration of coefficients for utility function variables – Testing for IIA properties – Analysis of household survey data – Analysis of on-board transit survey data • Calibration tasks we can do: • Compare highway and transit trips • Total • By purpose • Compare Ridership by route • CBD cordon line survey (if bus service is downtown only) .

I/E treated with the gravity model -E/E .8.Ps and As are prepared by matching ground counts .compare with Table 11 below .externals have no socioeconomic data . External stations .

all zones .intrazonal travel times .centroid connectors . local checks check 1. vary 2. major movements (cutlines) (for zones affecting corridor…) 3.auto occupancy .local network configuration .9.trip generation rates if all screenlines are high or low.trip generation rates .intrazonal times .socioeconomic data .speed .auto occupancy . vary .intersection penalties .intersection penalty .trip lengths . system wide (screenlines) if corridor volumes are high or low.all zones if links are high/low. System vs. vary . links .special generators .land use .centroid locations .

Expected/Required accuracy  We are concerned about errors that would require a design change (e. number of lanes)  Note that ground counts also contain error  Perfectly calibrated models produce link estimates with 1/3 above the standard error in ground counts and 2/3 below the standard error.  Need ground counts for 65% of freeways and arterials.10.g. and a good sample from other facilities From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

next page)  VMT/household should be 40-60 for large areas. 10-16 for smaller areas (see also Table A7. Expected/Required accuracy (cont. 30-40 for smaller areas .10.)  The correlation coefficient should be greater than .88  VMT estimate (region-wide) should be within 5% (take care to compare same roads in systems)  VMT/person should be 17-24 for large areas.

.

From CTRE Employment Data Project: .

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

Source: Virginia Travel Demand Modeling Policies and Procedures Manual .

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

Source: Virginia Travel Demand Modeling Policies and Procedures Manual .

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN .

conversion factors source? (peak hour to 24-hour) • Local VMT (% assigned to intrazonals and centroid connectors . Trip Assignment Reasonableness Checks • All-or-nothing assignment • study effect of increasing capacity • Compare to Equilibrium assignment • Check volume delay equation (BPR parameters) • Compare • screen line volumes • Cut line volumes • Time-of-day assignments? • Source of factors • Peak spreading used for future? • If not.

Trip Assignment Calibration and Validation Overall VMT or VHT check • 40 to 60 miles per day per HH in large metro areas • 30 to 40 miles per day per HH in medium metro • +/.10% OK on screen lines • Sign is important .

time of day .facility type .volume group .Compute by … .transit assignments .

.

12. TROUBLE SHOOTING .

.

.

Other Factors Impacting Forecasted Travel Demand • Can be implied in travel surveys (but not explicit) – Telecommuting – Flexible work hours – HB business • How to account for … – Aging population – Internet shopping – Roadway congestion (will it affect generation in the future) – New modes .

Issues for modeling • Transferability of parameters – More research is needed • Forensic analysis – How well did the models work? • Confidence and Credibility – How to improve • “Official” versions vs. documentation. what-if models – Integrity of the model • Need more transparency. appropriateness of techniques .