© 2013 ForeSee

ACSI E-GOVERNMENT
SATISFACTION INDEX
(Q3 2013)
MOBILE NATURALIZATION IN
DIGITAL GOVERNMENT
October 2013
Commentary and Analysis by:
Larry Freed, President and CEO, ForeSee
Dave Lewan, Vice President of Public Sector
Sales and West Region Sales, ForeSee
Eric Feinberg, Senior Director Mobile, ForeSee
Research by: Julie Anderson
2
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
EXECUTIVE SUMMARY
This quarter’s report on the American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index
has three parts:
> Part 1: Introduction – Mobile Naturalization in Digital Government. Citizen satisfaction scores
have hovered around 75 for the past four years. The bad news is that citizen satisfaction scores have
not increased over this period. One major inuence impacting citizen expectations is the growing
availability of mobile as a direct line for citizens to interact with government organizations,
agencies, and departments. Thus, as mobile is the next natural step for citizens, it should be the
logical next step for the digital federal government to measure and manage. This section also
shares mobile insights across a handful of agencies, departments, and organizations that asked
citizens additional questions regarding mobile use.
> Part 2: Citizen Satisfaction. This is a quarterly update on citizen satisfaction with e-government at
the aggregate level, including individual satisfaction scores for the 105 federal government websites
participating in the Index. Key ndings include:
– Citizen satisfaction with the e-government experience remains steady in Q3 2013. The third
quarter of 2013 registered an average citizen satisfaction score of 74.9, which is insignicantly
down from 75.0 in Q2 2013. The Social Security Administration’s (SSA) Retirement Estimator, SSA’s
iClaim, and Extra Help with Medicare Prescription Drug Plan Costs each scored a 90 to lead the
Index. See a complete list of individual agency scores on page 8.
– E-government outperforms overall government in citizen satisfaction. Each year, it is
clear that citizens prefer to interact with the federal government via the Internet. Average citizen
satisfaction with e-government continues to outperform average citizen satisfaction with the
overall federal government, recording a score of 74.9 for Q3 2013 compared to 68.4 for the overall
federal government in the ACSI’s 2012 U.S. Federal Government Index.
3
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
– Why measure satisfaction? Highly satised citizens are 92% more likely to use the federal
website they evaluated as a primary resource, 96% more likely to recommend the site, 64% more
likely to put their trust behind the agency, 54% more likely to return to the site, and 47% more
likely to participate with the agency in the future. See the chart and explanation on page 22.
– Search, Online Transparency, and Navigation are top priorities for improvement. By using a
scientic approach to measure e-government experiences through the eyes of the citizen, agency
managers and government ofcials can determine which improvements will have the largest impact
on satisfaction, thereby affecting future behaviors, such as those described on page 23.
> Part 3: Appendix A – Keeping Pace with Mobile: ForeSee’s Five Tips for a Better Mobile
Experience. Eric Feinberg, senior director of mobile, ForeSee, shares ve tips to help improve the
mobile experience in this abridged reprint of his whitepaper of the same name. While the examples
used are taken from the private sector, Feinberg believes that mobile excellence transcends all
industries, public and private.
PART ONE: INTRODUCTION
THE NATURALIZATION OF MOBILE IN
THE DIGITAL GOVERNMENT
In spite of undulating citizen expectations and opinions of the federal government through the past decade,
citizen satisfaction with the web arm of the federal government has actually remained fairly steady for more
than four years, hovering around the 75 mark.
Citizen satisfaction scores, in fact, have remained at 75 or higher (on the study’s 100-point scale) for 15 of
the last 17 quarters—since Q3 2009. The good news for e-government is that there has not been a major
decline in citizen satisfaction. The bad news is that citizen satisfaction scores have not increased over this
period either. Ideally, government ofcials need to address this problem by continuing to improve the
experiences they offer in order to meet the challenge of evolving citizen expectations.
4
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
One major inuence impacting expectations is the growing availability of mobile as a direct line for citizens to
interact with government organizations, agencies, and departments.
We are seeing mobile becoming a prime catalyst for interaction. Across all industries, ForeSee clients see
an average of 10% to 20% of overall requests to a given company’s digital experiences coming from mobile
devices (including mobile web and apps), according to Keeping Pace with Mobile: ForeSee’s Five Tips for
a Better Mobile Experience. This means for a company with one million unique monthly visitors, 100,000
to 200,000 individuals are inuenced by a mobile experience with that brand—and that percentage is
continuously increasing.
Furthermore, according to a June 2013 Nielsen report, three out of ve (61%) American adults owned
smartphones during the three-month period of March to May 2013, up more than 10% since early 2012.
Pew Research shows that in June 2013, one-third (34%) of U.S. adults owned tablets, which is almost two
times as many as a year ago (18%).
While mobile accessibility is a major focus for the federal government in its efforts to be more citizen-centric
as set forth by the Digital Government: Building a 21st Century Platform to Better Serve the American
People report (May 2012), this sudden advancement of mobile can create certain challenges—much like
what we see in the private sector. Most notably, consumers do not distinguish between the different
channels an organization offers. Although today’s customers (or in this case citizens) are busy traversing
across multiple touch points with multiple devices, they see the experience as singular and expect it to be
seamless. If citizens are unable to complete their tasks via a mobile device (whether it is a transaction or
informational search or something else) as easily as they can through a web interaction, their satisfaction
with the organization as a whole could suffer.
5
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Mobile implementation is a huge undertaking, as is ensuring that the experiences meet or exceed citizen
expectations. We continue to receive numerous inquiries from both concerned clients and curious prospects
regarding the best practices for mobile accessibility. In Appendix A of this report (page 27), Eric Feinberg,
senior director of mobile, ForeSee, shares ve tips for a better mobile experience.
As mobile is becoming the next natural step for citizens to take when engaging with federal government, it
should be the logical next step for government leaders to measure and manage.
MOBILE IN GOVERNMENT UPDATE
To get a better idea of how mobile use measures up in the digital government, ForeSee examined 19 federal
Index participants who added questions to their surveys regarding citizen mobile experiences. These addition-
al survey questions received more than 20,000 responses.
On average across all the participating agencies, more than half (55%) of government website visitors
reported ever using a mobile phone or tablet to access the Internet, showing a small increase from 52% last
quarter and from 48% when rst measured in Q4 2012 as the use of mobile devices for web access becomes
more common.
More than one-third (37%) of visitors (also a benchmark average) reported having accessed any federal
government website using a mobile phone or tablet, compared to 34% in Q2 2013. Furthermore, 13% of
visitors reported they had not used a mobile device to access federal government websites but planned to do
so; 36% of visitors reported they had not used a mobile device to access a federal government website but
may do so in the future; and 14% of visitors reported they had not used a mobile device to access a federal
government website and had no plans of doing so in the future.
6
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
On average, 31% of visitors to federal government websites participating in the survey reported that they
had used a mobile phone or tablet to access the specic website they were on; 21% of visitors reported they
had not used a mobile phone or tablet to access the website but planned to do so; 37% of visitors reported
they had not used a mobile phone or tablet to access the website but may do so in the future; and 11% of
visitors reported they had not used a mobile phone or tablet to access the website and had no plans of doing
so in the future.
PART TWO
SATISFACTION WITH E-GOVERNMENT REMAINS STEADY
Satisfaction with federal government websites continues to outperform satisfaction with the federal
government overall. In fact, e-government recorded a satisfaction score of 74.9, whereas the overall
government scored a 68.4 in the ACSI 2012 U.S. Federal Government Report released in February 2013.
Federal government websites have long lagged behind their private-sector counterparts in customer
satisfaction. However, a recent slide for the e-business sector broke this trend by scoring 71.8 in the ACSI
annual report on the sector in July 2013. These scores indicate that people are actually more satised with
federal government websites than they are with comparable private-sector e-business websites (such as
portals, search engines, news and information websites, and social media sites).
Government agencies have an obligation to the public to be scally responsible by using the federal budget
as wisely as possible. There are denite cost savings associated with offering a highly satisfying website
experience. Therefore, government agencies need to maintain the highest online standards by meeting or
exceeding citizens’ expectations. Federal websites that are already successfully meeting users’ needs should
take note of their achievements without becoming complacent. If done right, improving the citizen
experience will increase users’ likelihood to participate with government in the future, use the government
website as a primary resource (rather than costlier channels such as branch locations and contact centers),
and recommend the government site to others.
7
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
On the study’s 100-point scale, a satisfaction score of 80 or higher is typically considered the threshold for
excellence and can be achieved only if the organization is doing an outstanding job of meeting and
exceeding citizen expectations. This quarter, 30% (32 websites) of the federal participants in the Index
scored 80 or above. A score in this range shows that public-sector websites can live up to, and in some cases
exceed, expectations that the private sector sets in large part.
The ACSI E-Government Satisfaction Index is a comprehensive reection of the citizen experience with
federal government websites, and it serves as a critical checkpoint for evaluating the success and
performance of the federal government’s online initiatives. More than 270,000 responses were collected
across 105 federal government websites for the 2013 third-quarter Index. This demonstrates that
citizens are willing to share their voices to help agencies and departments improve. The use of ForeSee
technology and ACSI methodology then enables agency leaders to determine which website
improvements will have the greatest impact on future usage and recommendations.
S
a
t
i
s
f
a
c
t
i
o
n

S
c
o
r
e
(
1
0
0
-
p
o
i
n
t

s
c
a
l
e
)
Q3
03
Q4
03
Q1
04
Q2
04
Q3
04
Q4
04
Q1
05
Q2
05
Q3
05
Q4
05
Q1
06
Q2
06
Q3
06
Q4
06
Q1
07
Q2
07
Q3
07
Q4
07
Q1
08
Q2
08
Q3
08
Q4
08
Q1
09
Q2
09
Q3
09
Q1
10
Q2
10
Q3
10
Q4
09
70 69 70 70 71 72 71 72 73 73 73 74 73 73 73 73 73 72 72 72 73 74 73 73 75 75 74 75 75
ACSI E-GOVERNMENT INDEX
Satisfaction by Quarter 2003 - 2013
Satisfaction Q4
10
75
Q1
11
Q2
11
Q3
11
75 75 75
Q4
11
75
Q1
12
75
74
Q2
12
Q3
12
75 75
Q4
12
75
78
77
76
75
74
73
72
71
70
69
68
Q1
13
75
Q2
13
75
Q3
13
74
8
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Three websites from the Social Security Administration (SSA)—Retirement Estimator, iClaim, and Extra
Help with Medicare Prescription Drug Plan Costs—each scored a 90 to lead the top overall scores in the
E-Government Satisfaction Index. These three SSA sites also match or outperform some of the top measured
private sites to date in 2013, such as Mercedes-Benz (88), Apple (87), FedEX (85), and Amazon (85 in 2012).
The following table displays scores for all 105 participating federal websites in the E-Government Satisfaction
Index. The highlighted agencies represent those that have reached an excellent score of 80 or higher.
Later pages of the commentary show scores by category for more specic benchmarking purposes.
Q3 2013 ACSI E-Government Satisfaction Index
Department Website Satisfaction
Average   74.9
SSA Extra Help with Medicare Prescription Drug Plan Costs -- socialsecurity.gov/i1020 90
SSA SSA iClaim -- socialsecurity.gov/applyonline 90
SSA SSA Retirement Estimator -- ssa.gov/estimator 90
HHS MedlinePlus -- medlineplus.gov 88
SSA SSA Electronic Access/Online Statement -- ssa.gov/mystatement 88
DHS
U.S. Citizenship and Immigration Services Resource Center --
uscis.gov/portal/site/uscis/citizenship
88
HHS MedlinePlus en español -- medlineplus.gov/esp 87
HHS National Women’s Health Information Center (NWHIC) main website -- womenshealth.gov 85
SSA Social Security Business Services Online -- ssa.gov/bso/bsowelcome.htm 85
SSA Social Security Internet Disability Report -- ssa.gov/applyfordisability 85
HHS National Library of Medicine AIDS Information -- aidsinfo.nih.gov 84
HHS NIH - Senior Health -- nihseniorhealth.gov 84
HHS HHS Healthy People -- Healthypeople.gov 83
HHS National Cancer Institute main website -- cancer.gov 83
HHS National Cancer Institute Site en Español -- cancer.gov/espanol 83
HHS National Institute of Diabetes and Digestive & Kidney Diseases -- www2.niddk.nih.gov 83
CIA Recruitment website -- cia.gov/careers 83
DHS U.S. Citizenship and Immigration Services Español -- uscis.gov/portal/site/uscis-es 83
SEC U.S. Securities and Exchange Commission -- investor.gov 83
HHS CDC main website -- cdc.gov 82
Agencies
Scoring
80
+

9
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Q3 2013 ACSI E-Government Satisfaction Index (continued from page 8)
Department Website Satisfaction
DHS Federal Emergency Management Agency Ready Campaign -- ready.gov 82
DOD DOD Pentagon Channel -- pentagonchannel.mil 81
DOJ National Institute of Justice -- nij.gov 81
HHS NIAMS public website -- niams.nih.gov 81
DOS
U.S. Department of State Bureau of Educational and Cultural Affairs alumni website --
https://alumni.state.gov
81
Treasury U.S. Mint Online Catalog and main website -- usmint.gov 81
Boards, Commissions,
and Committees
American Battle Monuments Commission -- abmc.gov 80
DOD DoD Navy -- navy.mil 80
FTC FTC OnGuardOnline -- onguardonline.gov 80
NASA NASA main website -- nasa.gov 80
DOC
National Geodetic Society, National Oceanic and Atmospheric Administration website --
ngs.noaa.gov
80
HHS National Institute of Dental and Craniofacial Research -- nidcr.nih.gov 80
DOL Department of Labor Job Listings -- doors.dol.gov 79
DOJ FBI main website -- fbi.gov 79
DOI National Park Service main website -- nps.gov 79
DOS Recruitment website -- careers.state.gov 79
HHS SAMHSA Store -- store.samhsa.gov 79
NIH
The National Center for Complementary and Alternative Medicine (NCCAM) -- nccam.nih.
gov
79
NRC U.S. Nuclear Regulatory Commission website -- nrc.gov 79
PBGC MyPAA -- https://egov.pbgc.gov/mypaa 78
PBGC MyPBA -- https://egov.pbgc.gov/mypba 78
HHS National Library of Medicine main website -- nlm.nih.gov 78
DOS Bureau of Consular Affairs -- travel.state.gov 77
DOL Bureau of Labor Statistics -- bls.gov 77
HHS National Institute of Allergy and Infectious Diseases -- www3.niaid.nih.gov 77
HHS National Library of Medicine AIDS information -- aidsinfo.nih.gov 77
SBA SBA main website -- sba.gov 77
HHS U.S. Food and Drug Administration main website -- fda.gov 77
DOI U.S. Geological Survey -- usgs.gov 77
HHS Agency for Healthcare Research and Quality -- ahrq.gov 76

Agencies
Scoring
80
+

10
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Q3 2013 ACSI E-Government Satisfaction Index (continued from page 9)
Department Website Satisfaction
DOD Department of Defense portal -- defense.gov 76
FTC FTC main website -- ftc.gov 76
GAO GAO main public website -- gao.gov 76
USDA Recreation One-Stop -- recreation.gov 76
SSA SSA iAppeals - Disability Appeal -- ssa.gov 76
DHS U.S. Citizenship and Immigration Services -- uscis.gov/portal/site/uscis 76
USDA ERS main website -- ers.usda.gov 75
HHS HHS National Health Information Center -- Healthnder.gov 75
DOJ National Criminal Justice Reference Service -- www.ncjrs.gov 75
NIST National Institute for Standards and Technology main website -- nist.gov 75
DHS U.S. Citizenship and Immigration Services -- uscis.gov/e-verify 75
DOD DoD Air Force -- af.mil 74
DOT Federal Aviation Administration -- faa.gov 74
FTC FTC Complaint Assistant website -- ftccomplaintassistant.gov 74
HHS National Library of Medicine Clinical Trials website -- clinicaltrials.gov 74
GSA GSA main website -- gsa.gov 73
HHS Health Resources and Services Administration main website -- hrsa.gov 73
Treasury Making Home Affordable -- makinghomeaffordable.gov 73
OPM Recruitment website -- usajobs.gov 73
DOT U.S. Department of Transportation -- fhwa.dot.gov 73
PBGC U.S. PBGC main website -- pbgc.gov 73
DOD DoD Marines -- marines.mil 72
FDIC FDIC main website -- fdic.gov 72
Treasury Financial Stability -- nancialstability.gov 72
GSA GSA Auctions -- gsaauctions.gov 71
HHS SAMHSA website -- samhsa.gov 71
SSA Social Security Online: Frequently Asked Questions -- ssa-custhelp.ssa.gov 71
ITC U.S. International Trade Commission main website -- usitc.gov 71
FDIC FDIC Applications -- www2.fdic.gov 69
NARA NARA main public website -- archives.gov 69
Treasury Treasury main website -- treasury.gov 69
DOS Department of State main website -- state.gov 68
DOC U.S. Census Bureau main website -- census.gov 68
11
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Q3 2013 ACSI E-Government Satisfaction Index (continued from page 10)
Department Website Satisfaction
EPA U.S. Environmental Protection Agency -- epa.gov 68
SEC U.S. Securities and Exchange Commission main website -- sec.gov 68
DOC BEA main website -- bea.gov 67
DHS Department of Homeland Security main website -- dhs.gov 67
SSA Social Security Online main website -- socialsecurity.gov 67
HHS Girls Health -- girlshealth.gov 66
VA VA main website -- va.gov and myhealthva.gov 66
DOT DOT Research and Innovative Technology Administration website -- rita.dot.gov 65
Treasury Department of the Treasury, Alcohol and Tobacco Tax and Trade Bureau website -- ttb.gov 65
DOT Federal Railroad Administration main website -- fra.dot.gov 64
DOT Federal Motor Carrier Safety Administration main website -- fmcsa.dot.gov 63
USDA Forest Service main website -- fs.usda.gov 63
USDA NRCS website -- nrcs.usda.gov 63
GSA Ofcial Site to Buy U.S. Government Property -- govsales.gov 63
DOE U.S. Department of Education -- ed.gov 63
HHS HHS -- grants.gov 62
DOL Disability -- Disability.gov 59
Treasury IRS main website -- irs.gov 59
Treasury TreasuryDirect -- treasurydirect.gov 59
USDA FSIS main website -- fsis.usda.gov 58
DOD TRICARE -- tricare.mil 58
DHS
Federal Emergency Management Agency main
website -- fema.gov
55
12
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
TOP GAINERS
Any time a federal government agency, program, or department website shows signicant improvement in
satisfaction (of three points or more), it should be noted as a success, as it is sometimes difcult to keep pace
with the ever-changing citizen expectations. These entities show that they are denitely doing something right
by increasing citizen satisfaction, and others, whether in the same category or not, should take note of how
they are achieving this success.
The following chart shows the eight websites that demonstrated signicant increases (three points or more)
in citizen satisfaction from Q2 2013 to Q3 2013.
E-Gov Top Gainers (Quarter to Quarter)
Department Website
Satisfaction
Gain
HHS Girls Health -- girlshealth.gov 6
PBGC U.S. PBGC main website -- pbgc.gov 5
USDA ERS main website -- ers.usda.gov 4
GSA GSA Auctions -- gsaauctions.gov 4
DOD Pentagon Channel -- pentagonchannel.mil 4
DOC U.S. Census Bureau main website -- census.gov 4
NRC U.S. Nuclear Regulatory Commission website -- nrc.gov 4
Treasury Treasury main website -- treasury.gov 3
SATISFACTION BY FUNCTIONAL CATEGORY
In this report, federal government websites are organized by both functional category and organizational
structure to allow for benchmarking against peers. The functional website categories include: News
and Information; Portals and Department Main Websites; E-commerce and Transactional; and Career
and Recruitment.
13
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Since missions can vary greatly by category, it is useful for a government website to benchmark its score
against other sites in the same category in addition to comparing against the overall aggregate average.
To provide the most accurate and precise data, the ForeSee standard requires that a category consist of
at least ve websites before an average is calculated. Only four websites are measured in the Career and
Recruitment category; therefore, this category does not include an overall average. However, we do list the
scores of the individual websites within the Career and Recruitment category at the end of this section.
Federal E-Commerce and Transactional Websites
The following chart shows the citizen satisfaction scores for all 16 federal government E-Commerce and
Transactional websites measured in this category.
Federal E-Commerce and Transactional Websites
Department Website Satisfaction
E-Commerce Aggregate 79
SSA Extra Help with Medicare Prescription Drug Plan Costs -- socialsecurity.gov/i1020 90
SSA SSA iClaim -- socialsecurity.gov/applyonline 90
SSA SSA Retirement Estimator -- ssa.gov/estimator 90
SSA SSA Electronic Access/Online Statement -- ssa.gov/mystatement 88
SSA Social Security Business Services Online -- ssa.gov/bso/bsowelcome.htm 85
SSA Social Security Internet Disability Report -- ssa.gov/applyfordisability 85
Treasury U.S. Mint Online Catalog and main website -- usmint.gov 81
HHS SAMHSA Store -- store.samhsa.gov 79
PBGC MyPAA -- https://egov.pbgc.gov/mypaa 78
PBGC MyPBA -- https://egov.pbgc.gov/mypba 78
USDA Recreation One-Stop -- recreation.gov 76
SSA SSA iAppeals - Disability Appeal -- ssa.gov 76
FTC FTC Complaint Assistant website -- ftccomplaintassistant.gov 74
GSA GSA Auctions -- gsaauctions.gov 71
GSA Ofcial Site to Buy U.S. Government Property -- govsales.gov 63
Treasury TreasuryDirect -- treasurydirect.gov 59
14
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
E-commerce and Transactional websites remain the top-scoring category in this Index, recording a score
of 79 in Q3 2013 (matching the category score from Q2 2013), with the scores of websites in this category
ranging from 59 to 90. However, this category still trails behind private sector e-commerce websites, which
scored 81.1 in the ACSI E-Commerce annual report published in February 2013.
The good news is that the aggregate average for this category has steadily improved over the last year,
increasing from 77 in Q4 2012 to 78 in Q1 2013, and now 79 in both Q2 and Q3 2013. This is welcomed
stability to a category that has been erratic at best in its performance through the years, ranging from 75
in 2007, to 82 in 2009, to 77 in 2012.
Federal government E-commerce and Transactional websites could be on the way to stability. Only time will
tell, however. As long as they use a scientic, reliable, and accurate measurement system to monitor citizen
experiences, agency leaders should see improvements categorically and individually.
The Social Security Administration continues to set the bar high with six websites leading this category whose
scores range from 85 to 90. Three of the websites score a 90, outdoing every top-performing private-sector
e-commerce website measured by the ACSI, including Amazon.
The GSA Auctions site was the only site in the category to show a marked improvement (three points or
more) in Q3 2013, increasing their citizen satisfaction score four points to 71.
Federal News and Information Websites
The following chart shows the citizen satisfaction scores for all 53 of the federal government News and
Information websites.
15
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Federal News and Information Websites
Department Website Satisfaction
News/Information
Aggregate
75
HHS MedlinePlus -- medlineplus.gov 88
DHS
U.S. Citizenship and Immigration Services Resource Center --
uscis.gov/portal/site/uscis/citizenship
88
HHS MedlinePlus en Español -- medlineplus.gov/esp 87
HHS National Women’s Health Information Center (NWHIC) main website -- womenshealth.gov 85
HHS National Library of Medicine AIDS information -- aidsinfo.nih.gov 84
HHS NIH - Senior Health -- nihseniorhealth.gov 84
HHS HHS Healthy People -- Healthypeople.gov 83
HHS National Cancer Institute Site en Español -- cancer.gov/espanol 83
HHS National Institute of Diabetes and Digestive & Kidney Diseases -- www2.niddk.nih.gov 83
SEC U.S. Securities and Exchange Commission -- investor.gov 83
DHS Federal Emergency Management Agency Ready Campaign -- ready.gov 82
DOD DOD Pentagon Channel -- pentagonchannel.mil 81
DOJ National Institute of Justice -- nij.gov 81
DOS
U.S. Department of State Bureau of Educational and Cultural Affairs alumni website --
https://alumni.state.gov
81
Boards, Commissions,
and Committees
American Battle Monuments Commission -- abmc.gov 80
DOD DoD Navy -- navy.mil 80
FTC FTC OnGuardOnline -- onguardonline.gov 80
DOC
National Geodetic Society, National Oceanic and Atmospheric Administration website --
ngs.noaa.gov
80
NIH
The National Center for Complementary and Alternative Medicine (NCCAM) -- nccam.nih.
gov
79
NRC U.S. Nuclear Regulatory Commission website -- nrc.gov 79
DOS Bureau of Consular Affairs -- travel.state.gov 77
DOL Bureau of Labor Statistics -- bls.gov 77
HHS National Institute of Allergy and Infectious Diseases -- www3.niaid.nih.gov 77
HHS National Library of Medicine AIDS information -- aidsinfo.nih.gov 77
DOI U.S. Geological Survey -- usgs.gov 77
HHS Agency for Healthcare Research and Quality -- ahrq.gov 76
USDA ERS main website -- ers.usda.gov 75
HHS HHS National Health Information Center -- Healthnder.gov 75
DOJ National Criminal Justice Reference Service 75
DHS U.S. Citizenship and Immigration Services -- uscis.gov/e-verify 75
16
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Federal News and Information Websites (continued from page 15)
Department Website Satisfaction
DOD DoD Air Force -- af.mil 74
DOT Federal Aviation Administration -- faa.gov 74
HHS National Library of Medicine Clinical Trials website -- clinicaltrials.gov 74
DOJ U.S. Department of Justice, Ofce of Justice Programs -- www.ojp.usdoj.gov 74
HHS Health Resources and Services Administration main website -- hrsa.gov 73
Treasury Making Home Affordable -- makinghomeaffordable.gov 73
DOT U.S. Department of Transportation -- fhwa.dot.gov 73
DOD DoD Marines -- marines.mil 72
Treasury Financial Stability -- nancialstability.gov 72
SSA Social Security Online: Frequently Asked Questions -- ssa-custhelp.ssa.gov 71
FDIC FDIC Applications -- www2.fdic.gov 69
DOC U.S. Census Bureau main website -- census.gov 68
SEC U.S. Securities and Exchange main website -- sec.gov 68
DOC BEA main website -- bea.gov 67
HHS Girls Health -- girlshealth.gov 66
DOT DOT Research and Innovative Technology Administration website -- rita.dot.gov 65
Treasury USTTB website -- ttb.gov 65
DOT Federal Motor Carrier Safety Administration main website -- fmcsa.dot.gov 63
USDA Forest Service main website -- fs.usda.gov 63
USDA NRCS website -- nrcs.usda.gov 63
HHS HHS -- grants.gov 62
USDA FSIS main website -- fsis.usda.gov 58
DOD TRICARE -- tricare.mil 58
The News and Information aggregate score remained at 75 for Q3 2013, with scores for individual
websites ranging from 58 to 88. The aggregate score for Federal News and Information Websites
was higher than that of private sector news and information websites, which scored 73 in the most
recent ACSI E-Business report.
17
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Eighteen websites (34%) in this category are considered top performers (scoring 80 or higher). The U.S.
Citizenship and Immigration Services Resource Center and the English version of the MedlinePlus site led this
category, each with a score of 88. The Spanish version of the MedlinePlus site scored 87, followed by the
National Women’s Health Information Center main website with an 85.
This category also accounted for 63% of the websites that experienced a signicant increase of three points
or more in citizen satisfaction since Q2 2013 (see chart on page 12). The Health and Human Services Girl’s
Health website saw the largest score increase for this category and index, jumping six points to 66.
Also in this category, the Department of Defense’s Pentagon Channel site jumped into the top performer
category with a four-point increase in satisfaction to 81. The U.S. Nuclear Regulatory Commission website
(79), the U.S. Department of Agriculture’s Economic Research Service main website (75), and the U.S. Census
Bureau main site (68) all increased by four points from last quarter.
Looking ahead, the challenge—as it is with any business or government agency that experiences an increase
in satisfaction—will be not just to maintain, but to grow and improve. If these federal government websites
continue to measure the expectations and experiences of their visitors, they should be able to make the
improvements necessary for continued success.
In spite of the number of top performers and the signicant satisfaction increases that some websites
experienced this quarter, the aggregate category score did not improve and has not improved much over
time. In fact, this category has recorded a citizen satisfaction score of about 75 for the past 10 quarters.
This stagnation is a sign that underperforming websites are not gaining and, more likely, are losing ground.
In this quarter, 15 websites witnessed declines in citizen satisfaction.
18
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Federal Portals and Department Main Websites
The following chart shows the citizen satisfaction scores for all 33 of the Federal Portals and Department
Main websites in this category.
Federal Portals and Department Main Websites
Department Website Satisfaction
Portal and Department
Main Sites Aggregate
72
HHS National Cancer Institute main website -- cancer.gov 83
DHS U.S. Citizenship and Immigration Services Español -- uscis.gov/portal/site/uscis-es 83
HHS CDC main website -- cdc.gov 82
HHS NIAMS public website -- niams.nih.gov 81
NASA NASA main website -- nasa.gov 80
HHS National Institute of Dental and Craniofacial Research -- nidcr.nih.gov 80
DOJ FBI main website -- fbi.gov 79
DOI National Park Service main website -- nps.gov 79
HHS National Library of Medicine main website -- nlm.nih.gov 78
SBA SBA main website -- sba.gov 77
HHS U.S. Food and Drug Administration main website -- fda.gov 77
DOD Department of Defense portal -- defense.gov 76
FTC FTC main website -- ftc.gov 76
GAO GAO main public website -- gao.gov 76
DHS U.S. Citizenship and Immigration Services -- uscis.gov/portal/site/uscis 76
NIST National Institute for Standards and Technology main website -- nist.gov 75
GSA GSA main website -- gsa.gov 73
PBGC U.S. PBGC main website -- pbgc.gov 73
FDIC FDIC main website -- fdic.gov 72
HHS SAMHSA website -- samhsa.gov 71
ITC U.S. International Trade Commission main website -- usitc.gov 71
NARA NARA main public website -- archives.gov 69
Treasury Treasury main website -- treasury.gov 69
DOS Department of State main website -- state.gov 68
EPA U.S. Environmental Protection Agency -- epa.gov 68
19
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Federal Portals and Department Main Websites (continued from page 18)
Department Website Satisfaction
DHS Department of Homeland Security main website -- dhs.gov 67
SSA Social Security Online main website -- socialsecurity.gov 67
VA VA Main website -- va.gov and myhealthva.gov 66
DOT Federal Railroad Administration main website -- fra.dot.gov 64
DOE U.S. Department of Education -- ed.gov 63
DOL Disability -- Disability.gov 59
Treasury IRS main website -- irs.gov 59
DHS Federal Emergency Management Agency main website -- fema.gov 55
The Portals and Department Main Websites category scored 72 for the third quarter of 2013, which
is one point off from Q2 2013. The websites in this category had scores ranging from 55 to 83. While
federal e-government Portals and Department Main Websites are the lowest-scoring category in this report,
they also lag behind the private-sector measure of Portals and Search Engines, which scored 76 in the
July 2013 ACSI E-Business Report.
The National Cancer Institute main website and the Spanish version of the U.S. Citizenship and Immigration
Services site lead the category, with both sites scoring an 83.
Two sites from this category landed on the top gainers chart (see page 12). The U.S. Pension Benet Guaranty
Corp main website increased satisfaction with the citizen experience by ve points (the second highest in the
index this quarter) for a 73. After gaining three points in Q2 2013, the Treasury’s main site improved another
three points in Q3 to 69.
As with the News and Information Websites category, some Portals and Department Main Websites saw
improvement, while others experienced signicant decreases that, in turn, lowered the overall category score.
Again, websites that fail to meet the expectations of citizens may not see a return on investment (ROI) if they
do not begin or continue to measure the experiences that citizens have with their websites.
20
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
ForeSee recently conducted an analysis in the private sector to quantify the impact of customer satisfaction
on future revenue among 91 of the top 100 e-retailers. Predicting year-end 2011 revenue with spring 2011
customer satisfaction, we found that a one-point increase in customer satisfaction translated to a 10.6 %
increase in revenues, which corresponded to roughly $72 million. This relationship between customer
satisfaction and revenues is based on data from the largest 100 e-retailers as designated by Internet Retailer.
This excludes Amazon because Amazon is atypical, even among the top 100. Individual companies may nd
that this relationship differs somewhat when modeling their own data; however, it is clear that there is a
relationship between customer satisfaction, as measured with the ACSI, and actual revenue.
While increasing satisfaction impacts organizations in the public sector differently (such as cost savings
versus revenue) than it does the private sector, the above ndings make a strong case for the critical
importance of providing an excellent customer experience and having tools in place that help quantify ROI.
If measured and managed properly, federal agency and government leaders can make a difference in the
e-government landscape.
Federal Career and Recruitment Websites
Federal Career and Recruitment Websites
Department Website Satisfaction
CIA Recruitment website -- cia.gov/careers 83
DOL Department of Labor Job Listings -- doors.dol.gov 79
DOS Recruitment website -- careers.state.gov 79
OPM Recruitment website -- usajobs.gov 73
21
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
We have not seen many changes in the Career and Recruitment category, with only four websites currently
measuring citizen satisfaction. None have signicant increases or decreases in satisfaction this quarter.
However, there is a 10-point score range among the participating entities, which means the websites trailing
in this category have ample opportunity to improve considerably. Those who lead should continue to meet
and exceed citizen expectations.
Career and recruitment websites are dependent upon citizens returning to, participating with, and
recommending them in order to be benecial to the agencies maintaining these websites. Therefore, it is
critical for the leaders of these agencies to keep in mind future behavior scores (see Why Satisfaction
Matters below) and the elements that drive satisfaction and impact behavior (see Common Elements of Satis-
faction on page 23).
Why Satisfaction Matters
Satisfaction has been shown to have a direct impact on behavior. If federal government agencies focus on
improving their websites’ priority areas, citizen satisfaction with their websites should also improve.
Every quarter, this Index compares less-satised website visitors (with satisfaction scores of 69 or less) to
highly satised website visitors (with satisfaction scores of 80 or higher) to produce likelihood scores of what
citizens will do in the future (i.e., participate again, use the website as a primary resource, recommend the
website to others, return to the website, and trust the agency). The following page contains a graph that
shows the range of satisfaction for each measured future behavior.
Based on likelihood scores, citizens who are highly satised with a federal government website rate their trust
in the agency 64% higher. Highly satised citizens also report being 47% more likely than those who are less
satised to participate with the government by expressing their thoughts to the agency.
Satisfaction also increases the likelihood that the citizen will return to the website again (54%), use it as a
primary resource (92%) as opposed to more costly channels, or recommend the site to others (96%).
22
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Why Sansfacnon Mauers
 
Highly
Satised
Citizens (80+)
Dissatised
Citizens (< 70)
Difference
in Scores
The Impact of Higher Website Satisfaction
Future
Participation
63 43 47%
Citizens are more likely to participate with and express their thoughts
to their government, which strengthens the democratic process and
may provide useful feedback.
Return to Site 97 63 54%
Government departments and agencies have an ongoing channel to
provide information and services to citizens efciently and relatively
inexpensively.
Recommend
Site
96 49 96%
Use of government websites will grow as citizens recommend them to
their friends, family, and colleagues.
Use Site
as Primary
Resource
92 48 92%
Cost-savings for departments and agencies can result as citizens
are right-channeled to web; citizens get information from a credible
government source, rather than another online/ofine source (in cases
where options exist, e.g., health-related information).
Trust 90 55 64%
Citizens believe the agency is trustworthy and acting in their best
interests, which fosters faith in the democratic process.
These gures illustrate how vital it is to measure citizen experience and demonstrate that the results the
ForeSee methodology produces are more than just numbers. The information here demonstrates that
technology-driven customer satisfaction analytics, when done right, can predict website visitors’ future
behaviors. Furthermore, the data-driven ndings can guide agencies to make improvements that will increase
desired citizen behaviors.
High customer satisfaction is important for federal websites, but it can be increased.
The federal websites that are using ForeSee to measure citizen satisfaction also measure a number of website
elements, or drivers of satisfaction. Although there are variations in the set of elements that are relevant to
each website, the most common elements are: Search, Functionality, Online Transparency, Navigation,
Look and Feel, Content, and Website Performance.
23
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Common Elements of the Website Experience
Element What It Measures
Priority for
Improvement
Search
The relevance, organization, and quality of search results available on
the site. (Although this element is not applicable universally, it is often
extremely impactful for sites where it is relevant.)
Priority 1 = Top Priority
Functionality
The usefulness, convenience, and variety of online features and tools
available on the website.
Priority 1 = Top Priority
Online Transparency
How thoroughly, quickly, and accessibly the website discloses information
about what the agency is doing.
Priority 1 = Top Priority
Navigation The organization of the site and options for navigation. Priority 2
Look and Feel The visual appeal of the site and its consistency throughout the site. Priority 3
Content
The accuracy, quality, and freshness of news, information, and content
on the website.
Priority 3
Site Performance The speed, consistency, and reliability of loading pages on the website. Priority 3
By measuring these elements, federal agencies can pinpoint and prioritize areas of improvement from
the citizen’s perspective, which leads to increased satisfaction. If federal agencies fail to scientically
measure and analyze the results, they will have difculty making the changes that will enhance their value
and usefulness to citizens in a cost-effective manner.
24
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
ABOUT THE AUTHORS
Dave Lewan manages ForeSee’s sales organization focused on the public sector, including government
departments and agencies, non-prot organizations, and higher education institutions. He leads ForeSee’s
continuing expansion in these markets to help organizations measure and manage satisfaction of
their website visitors. Dave brings more than 20 years’ experience in sales, leadership, online strategy,
technology, and consulting. Most recently, he served as Vice President of Product Marketing and
Management at Gevity HR, where he acted as the primary leadership and planning force for Gevity’s
product and solutions strategy. Dave graduated from the University of Minnesota with a degree in
speech communication.
As president and CEO of ForeSee, Larry Freed is responsible for managing the company’s strategy and
signicant growth since the company’s founding in 2001. Larry brings more than 20 years of experience
in senior management and in directing ForeSee’s e-commerce and technology initiatives. An expert on
the cross-channel customer experience and author of Innovating Analytics – Word of Mouth Index:
How the Next Generation of Net Promoter Can Increase Sales and Drive Business Results and Managing
Forward: How to Move from Measuring the Past to Managing the Future, Larry speaks extensively on the
topic at private and public sector industry events. He has also been quoted in numerous publications and
media outlets, including CNN, The Wall Street Journal, The Washington Post, Investor’s Business Weekly,
Internet Retailer, Multichannel Merchant, DM News, Computerworld, Federal Computer Week, and
Government Executive, among many others.
Senior Director of Mobile, Media & Entertainment Eric Feinberg provides leadership to ForeSee’s
mobile solutions as well as solutions related to the media and entertainment industries. He is responsible
for working with product, delivery, sales, and marketing teams to ensure that ForeSee brings innovation
and operational excellence to its mobile offerings. Eric brings 15 years of customer-focused experience to
the team and is currently on the Board of Directors for the Digital Analytics Association (DAA), formerly the
Web Analytics Association.
25
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
ABOUT THE RESEARCH TEAM
Julie Anderson, research analyst at ForeSee, has over 13 years of consulting and research experience
in public and private organizations, with a focus on large-scale data analytics. Through her work as a
satisfaction research analyst at ForeSee, she has extensive experience with customer experience and
customer satisfaction methodologies, and in providing voice-of-customer analytics with a special focus
on digital analytics.  
ABOUT THE ACSI E-GOVERNMENT SATISFACTION INDEX
The ACSI E-Government Satisfaction Index is a special quarterly report of the American Customer
Satisfaction Index (ACSI) in partnership with customer experience analytics rm, ForeSee.
The ACSI, created at the University of Michigan, is the only uniform, national, cross-industry measure of
satisfaction with the quality of goods and services available in the United States, both in the private and
public sectors. In 1999, the federal government selected the ACSI to be a standard metric for measuring
citizen satisfaction. More than 100 federal government agencies have used the ACSI to measure
citizen satisfaction with more than 200 services and programs and more than 100 websites. The report
on ofine federal government services is released annually in December. The E-Government Index is
released quarterly.
ForeSee collects and analyzes the data for the e-government websites included in the report. The
e-government scores were calculated based on data gathered from voluntary online surveys of randomly
selected site visitors. Each government website was rated by its visitors on various components of overall
satisfaction. The ratings were converted to a score on a 100-point scale, using the ACSI methodology.
26
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
The ACSI methodology identies key drivers of online satisfaction (such as navigation, look and feel, search,
site functionality, etc.) and quanties their relationship to overall citizen satisfaction. This cause-and-effect
methodology demonstrates the impact of website enhancements in these areas on overall customer
satisfaction. In turn, customer satisfaction, measured in this way, has been proven to predict how citizens
will behave in the future. Improvements to customer satisfaction will make citizens more likely to choose
to interact with an agency online (the more cost-effective channel), return to the site, and recommend it to
others. Monitoring and improving customer satisfaction has a tangible impact on citizen usage of the web
channel and on the bottom line.
ABOUT THE ACSI
The American Customer Satisfaction Index (ACSI) is a national economic indicator of customer
satisfaction with the quality of products and services available to U.S. consumers. It is updated quarterly
with new measures for differing sectors of the economy, building on the previous year’s data. The overall
ACSI score for a given quarter factors in scores from more than 200 companies in 44 industries, and from
government agencies over the previous four quarters. The Index was founded at the University of
Michigan’s Ross School of Business and is produced by ACSI, LLC.
ABOUT FORESEE
As a pioneer in customer experience analytics, ForeSee continuously measures satisfaction across
customer touch points and delivers critical insights on where to prioritize improvements for maximum
impact. Because ForeSee’s superior technology and proven methodology connect the customer experience
to the bottom line, executives and managers are able to drive future success by condently optimizing
the efforts that will achieve business and brand objectives. The result is better business for companies
and a better experience for consumers. Visit www.foresee.com for customer experience solutions and
original research.
27
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
APPENDIX A
KEEPING PACE WITH MOBILE: FORESEE’S FIVE TIPS FOR A
BETTER MOBILE EXPERIENCE
WHY MEASURE MOBILE
There’s a saying at ForeSee: You have to know what you know, and you have to know what
you don’t know.
What most people know in the mobile space right now relates to the quantity of things: the number
of app downloads, the number of mobile website visitors, and a menagerie of other behavioral data.
This is good information to have; however, there’s a bit of a challenge there. Since the popularity of
mobile is increasing, and the number of people using mobile is increasing, the basic metrics around the
experience are increasing, regardless if a company is doing an excellent job or not. This could lead to
misinformation and misguided decisions.
What people need to know is what they don’t know—the quality of things and what is motivating
someone to take action. As mobile use continues to grow, companies need to benchmark against
themselves, their peers, and the market. They also need to understand how mobile is being used. What
makes mobile so perplexing is that it has an impressive duality where it is both a stand-alone channel where
people can engage with it alone and a companion channel where it supports other experiences they might
be having with the brand. This duality makes for a complex, sometimes hard-to-navigate environment.
Knowing exactly how you are doing and where you can improve, whether you are a mobile veteran or
newcomer, starts with listening to your customers. The voice of the customer is louder than any other voice
out there and should be listened to rst and foremost. Start by measuring it.
28
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
While the following ve tips include examples primarily from the private sector, they should not be taken
lightly, overlooked, or discarded out of fear they do not apply to the federal government space. Mobile
excellence transcends all industries, private and public, as the channel is becoming a dominant force in
driving satisfaction with the overall citizen and customer experience.
TIP 1: WELCOME CUSTOMERS TO YOUR MOBILE HOME
When someone visits a traditional website, companies do their best to
customize the customer’s experience based on his or her recent visits,
referring URL, browser, and so on. The same needs to be done in mobile.
Company leaders should “welcome visitors to their mobile homes” the way
they would welcome people intoa virtual and real home. Here’s how:
> Say hello. Repeat visitors are becoming more and more
commonplace. Therefore, if a company is taking the time to
collect visitor information through cookies, it should take the
time to acknowledge these loyal customers personally and say
hello. Amazon.com does a great job of this.
> Acknowledge them. While most mobile visitors are anonymous,
there are many, many things to know about them, such as: their
device’s operating system, screen size, and referring URL. The
IMDb.com mobile site acknowledges individuals even if they
have never been there before by recognizing the type of phone
and offering a link to download the app or the option to go to
the full site.
IMDb recognizes the operating system being
used by the visitor and asks if s/he wants to
download from the App Store, which is Apple.
29
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
> Be their guide. Many mobile experience executives think that
if they don’t have the best mobile site possible, then they
shouldn’t put anything mobile-optimized out there. That’s not
entirely true and, in some cases, is just plain wrong. Customers
are smart, and they appreciate honesty and altruistic intent.
Discount Tire does an admirable job of guiding visitors to the
three main things people want to do on mobile: nd a store,
see promotions, or look up product information. After viewing
this initial interstitial landing page for mobile handset visitors,
visitors still have to go to the full site to accomplish most tasks,
but these few mobile-optimized interstitial pages show that the company understands what the
customer wants and caters to their needs and expectations. This is a good idea of what a company
can do as an interim solution while in the process of instituting a more comprehensive mobile site.
> All of the above. Executives should strive to do all of the above. Even if it’s just a single
mobile-optimized page, it will show the multitude of visitors that the company is trying to meet
their new mobile needs.
30
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
TIP 2: CHALLENGE CONVENTION
Mobile is ripe for creativity, and more business leaders need to take more chances. Those who do only the
bare minimum will be left behind as others innovate and excel. In addition to assuming that a company
can and should deliver everything customers have come to expect from the more traditional website,
the amazingly smaller form and fantastic new technologies within touchscreen capabilities factor into
customers’ rising expectations and wants in mobile. Here are some technologies available in the mobile
space to be creative and take chances with:
> Geolocation. By using location-based services like Meijer’s Find-it
app, users can track down anything they need while right
inside the store. Geolocation can help further customize the
mobile experience. All retailers should be providing a different
user experience for three groups of people: mobile visitors
inside their stores, mobile visitors nearby their stores (or
prepping for a store visit), and everyone else. Executives should
strive to develop the fourth user experience aimed at those in
a competitor’s store.
31
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
> Use HTML5 or similar.
Like m.ESPN.com and
TripAdvisor, using smart
layering techniques allows
for seemingly magical
navigation across and
through content.
> Cut or Curate. While one faction of the dueling schools of
mobile navigation says to cut content to make it t in this
smaller handset experience, the other side says to simply curate
content beautifully and render every piece of content within this
new paradigm. Anyone who says that the form factor is restrictive
is unimaginative. The people at m.footballfanatics.com did a
brilliant job of challenging convention on their retail
conrmation page. They put every single thing a visitor
would get on the traditional web into a perfectly designed
conrmation page.
32
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
> Familiar things in unfamiliar
places. By using Crackle’s
and ABC’s mobile phone
experiences, visitors can view
content on their own time.
In some cases, they can view
broadcast content before it
even airs. Companies should
constantly think about how
to deliver valuable content
and shopping experiences in ways that delight and satisfy consumers.
> Simple navigation is not better navigation. Just because it is
easy to splash identical navigation on mobile sites and apps
doesn’t mean it’s the best for visitors. Navigational structures
should be specic and creative. If a company is just getting in
to the game by developing a mobile experience, simple might
be better. If it comes down to the choice between getting in
the game and doing nothing—get in the game.
33
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
> Off-canvas navigation solves much. When building a proper
mobile experience from scratch or evolving an existing, full
mobile experience, a navigational schema such as “off-canvas”
navigation should be a consideration. Digital product leader
and design expert Luke Wroblewski speaks eloquently about this in
“Multi-Device Layout Patterns” at www.LukeW.com. This type of
design can really wow your visitors without them being mobile experts.
These layouts use the space outside a browser’s view to hide secondary
elements until people need them. Facebook is the most well-known
example of this approach. As a visitor taps the navigation queue
in the upper left, the full navigation is revealed as the content slides over to make room.
TIP 3: MEASURE, MEASURE, MEASURE
In the old days, before mobile app and sites and even the traditional website, “location, location, location” was
the typical business mantra. Those were the days when a business would live or die by how physically visible its
presence was.
In today’s ever-expanding mobile and online world, companies—out of need rather than want—are changing
the way they do business, and if they haven’t yet, then they need to soon before they become extinct. In this
day and age, it’s more about measurement than location, since consumers, through web and mobile devices,
can shop anywhere at any time. Measurement gives executives insights into the who, what, where, when, and
why of their customer and prospective customer base.
34
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
In analytics, mobile analytics especially, it’s all about measurement, measurement, measurement. Here are four
things that will help mobile initiatives succeed:
The most important thing is to do these things continuously. This continuous measurement is vital as mobile
initiatives are being designed, getting implemented, changing, and moving forward. Not only are companies
changing over time, but so are their customers: their needs should be met and their expectations exceeded.
FORESEE SATISFACTION ANALYTICS FOR MOBILE
Measure effectiveness and success
> Apply science to continuous customer experience measurement across mobile-optimized sites,
tablets, and apps
> Measure satisfaction and drivers of satisfaction and impact on future behaviors
>
Gain intelliegence
>
– Who is your mobile audience: personas, segments, demographics
– Where are your consumers using mobile and when in the lifecycle
– Why do they use mobile
> Diagnose strengths, weaknesses and opportunities
Prioritize improvements
> Predict impact of change on satisfaction and behavior
> Provide actionable insights
Benchmark your performance
> Compare against peers, competitors, best-in-class, yourself over time
Measure how mobile inuences and is inuenced by other channels
Prole your mobile users
35
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
TIP 4: PERFORMANCE KILLS
When mobile-optimized websites or apps work, all is well. When customers
request information from their mobile device, the wireless carrier signal
goes, nds it, and delivers the data (the content) to the phone. They are
satised because their expectations were met, and they were able to accom-
plish their goal.
Satisfaction, however, isn’t always high in the mobile content consumption
space. ForeSee benchmark data shows that mobile satisfaction lags behind
traditional website satisfaction. As most mobile users have experienced
at one point or another, somewhere along the line the signal fails, and the page is not delivered or an error
message is rendered. If people try to engage a mobile site for content and can’t nd what they want, who are
they going to blame? Recently there has been a great distinction over who is to ultimately blame for a failed
mobile experience. The fact is that customers will blame the company for failures and credit them for successes,
regardless of whose responsibility it really is. It’s the company that loses out because performance killed the
experience. The truth of the matter is that it might be the carrier’s fault, but
the customer doesn’t think so—perception reigns here.
The solution: For every error message that is rendered, there should be a
certain level of customization alongside a dose of personality. This allows
for the ultimate one-to-one connection that companies should have with
customers but is often forgotten about in the digital world. Something as
simple as a creatively worded error message can change the customer’s
perception of the experience. Companies should take notice of all the error
messages associated with their mobile sites and apps, and be clear with
the empathy associated with the experience. An apology can also go a long way in creating a more positive
customer experience that will garner customers that are more loyal, more likely to make a purchase, more likely
to return to the mobile site, and more likely to recommend the company, site, or app to others.
36
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
TIP 5: CONNECT THE DOTS
Connecting the dots is a simple exercise of linking a seemingly random group of points in a way that, when
done, results in a complete picture. Connecting the dots with deep linkages to inventory is powerful—it’s where
everything is headed. This is a technical hurdle to help sew the systems together. However, once achieved there’s
still a need to provide a fantastic customer experience for the mobile visitor.
Since the mobile experience is becoming more and more fragmented, using a consistent and powerful
technology to measure across the board will help connect those pieces and bring it all together to make a
complete picture—one representing the best multichannel customer experience possible.
Connecting the dots is all about recognizing that mobile inuences other channels. It might inuence a store
experience, a call experience, a web experience, a social media experience, and/or an email experience. We also
need to understand that mobile is inuenced by these other channels, creating this wonderful bidirectional ow.
As people are engaging at a store and want to look something up, they reach for their mobile device, creating
an extraordinary and quality nexus point to measure in.
When mobile is measured thoroughly with a scientic measurement, executives will get a clear picture of not
just who is coming to the site and why, but also the inuence back to these other channels. Tip 3 talked about
knowing what you know (the quantity of things) and knowing what you don’t know (the quality of things).
These together deliver a complete picture of the behaviors and the attitudes of what people are engaging in.
37
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Attitudes will drive what people do (purchase, purchase again, return to the site, remain loyal, and recommend
to others), and that’s an important component when connecting the dots.
The mobile experience continues to get more complex when looking at the impact that mobile has on other
channels. What happens when customers go from the web to mobile to a store, or they go to a store, then
to their mobile device, then to the web, then to social media to ask friends, then back to mobile because they
were using that for their social experience, and then make the nal purchase on the web? How can company
leaders get the most reliable data from a multiple-device experience like that? Better yet, how can they predict
what a multi-device consumer will do next? With the right technology—one that is accurate, precise, and
sensitive—they can measure across all touch points, see where the different experiences intersect, and actually
predict what the consumer will do next.
BEHAVIORS ATTITUDES
FUTURE
BEHAVIORS
Impact on
Future Behaviors
RECOMMEND MOBILE SITE
PURCHASE
2.5
2.9
3.4
70
72
78
RETURN TO MOBILE SITE
Dashboard Dashboard
17,311 Visits
1.74 Visits
69.61% Bounce Rate
Dashboard
Visitors
Traffic Sources
Content
Goals
About the Report
Beta Feedback
Report Finder
Common
Questions
38
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
> It doesn’t have to be pretty. At right is an example of a basic
mobile site from an automotive company. While it won’t win
any mobile customer experience design awards for this, the
‘call’ to action is clear. This is a simple example of connecting the
dots—simple yet effective.

> From another channel to mobile. Some great examples of this
include HSN’s constant awareness of what’s being broadcasted so
that users who are co-browsing while watching TV can easily get
in the funnel.
> Mobile to retail store. Lowe’s has a fantastic interconnected
mobile site and app experience. With great ease, a visitor can
prepare for a store visit via mobile device. It’s what the customer
wants, and it’s built beautifully.
39
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Mobile lives before, during, and after key experiences. So when measured well—the behaviors and attitudes
of what people are doing—the full picture of the customer experience begins to emerge, lending a sense not
only of mobile attribution but of mobile contribution.
CONCLUSION
The mobile experience is becoming more challenging every day because there’s so much more to navigate.
There are phones operating across mobile sites and apps; there are different-sized screens and diverse
functions among the various smartphones; and there is an increasing array of operating systems. This makes
mobile a hard and confusing landscape for companies to navigate.
ACQUISITION SOURCES & INFLUENCERS
CUSTOMER
STORES
MOBILE WEB
SOCIAL
MEDIA
CONTACT
CENTER
EMAIL
SOCIAL
MEDIA
EMAIL ADVERTISING BRAND SEARCH
40
ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)
Tracking and counting through traditional metrics can be a little misleading when trying to traverse through
this new multichannel landscape. Just because a company’s analytics or information technology teams
measure more people coming to their sites doesn’t necessarily mean the company is doing a good job of
meeting the expectations of its visitors. It could mean that more people are using mobile experiences, but it
doesn’t mean they are satised with the experiences provided. It doesn’t mean they’re going to buy anything,
or subscribe, or come back again.
Mobile is cool and exciting, but executives and decision makers rst have to grow out of the mindset that,
just because they’re developing a mobile experience, it will serve the customer admirably. It might; it might
not. The customer is the key here. Until the customer experience is measured, the dots remain just dots. Only
when they are connected, through measuring both behavioral and attitudinal, can business leaders create the
experiences that customers are looking for and deserve.
As far as mobile has come already, it is still a new and nascent industry full of possibilities and open to
innovation. Instincts are important when developing mobile experiences and should always be trusted.
If there is an adventurous and exciting mobile idea on the boardroom table that everyone believes will
work, it probably will.
Mobile is the one channel where there’s still the possibility of piquing people’s interest, changing the ways they
engage with companies, and offering them the ability to do things that are easier, faster, and more convenient
than ever imagined. There’s a certain exibility in mobile right now that offers companies an opportunity to
do what they want and to do it in the best interest of the consumer to positively inuence the other channel
experiences that they have control over.
Before implementing any of these tips, though, companies need to rst talk with their customers and nd out
what they want, need, and expect, by measuring their experiences. Then, from their perspective, companies
can design a mobile experience that works for the customer and for the company.
In the end, everyone wins.

Sign up to vote on this title
UsefulNot useful