Research Article

TRRJOURNAL OF THE TRANSPORTATION RESEARCH BOARD

Transportation Research Record

0(0) 1–10
Establishing Face and Content Validity ! National Academy of Sciences:
Transportation Research Board 2020
of a Survey to Assess Users’ Perceptions Article reuse guidelines:
sagepub.com/journals-permissions
of Automated Vehicles DOI: 10.1177/0361198120930225
journals.sagepub.com/home/trr

Justin Mason1, Sherrilene Classen1, James Wersal1,

and Virginia P. Sisiopiku2

Abstract
Fully automated vehicles hold promise for providing numerous societal benefits, including drastically reducing road fatalities.
However, we know little about the adoption practices of individuals related to automated vehicles. To assess transportation
users’ perceptions of automated vehicles, a 40-item survey was designed using guidance from several technology acceptance
models. A focus group was used to assess face validity to ensure the items appeared credible and understandable to the
layperson. Seven subject-matter experts rated items for their relevance to provide a content validity index for each item and
for the overall survey. The final scale had a scale content validity index rating of 1.00, with 32 of 32 items rated greater than
or equal to 0.86 and a scale content validity index of 0.96 (mean content validity index of all items), indicating acceptable
content validity. The approach adopted in this study ensured the face and content validity of the survey and enhanced the
items’ relevance, concision, and clarity. Future validation is required to assess scale reliability and validity. The paper provides
an overview of models used for determining acceptance and adoption of technology and describes in detail the methodology
used to establish face and content validity of the questionnaire survey developed for assessing adults’ perceptions of auto-
mated vehicles.

Traffic crashes account for 1.25 million fatalities each
year throughout the world, with an additional 50 million
people injured (1). An estimated 93% of all traffic
crashes result from human errors including any or all
of the following: driving while distracted, speeding, alco-
hol, drug involvement, and fatigue (2,3). Fully automat-
ed vehicles (AVs), in which the autonomous driving
technology system performs all driving tasks, may dras-
tically reduce road fatalities (4,5). In addition to increas-
ing road safety, automated driving is predicted to reduce
traffic congestion, fuel consumption, and gas emissions
(6). AV technology may catalyze changes to infrastruc-
ture, policies, traffic flow, and public transit. However,
all such potential societal benefits will not be achieved
unless these vehicles are widely accepted and adopted by
the general public. Thus, as we prepare for the potential
introduction of AVs into the market, it is important to
understand individuals’ perceptions and attitudes toward
the use of AVs. The purpose of this study was to report
on the item development and initial validity of a new
survey to assess adults’ perceptions of AVs.
Automated Vehicles
The Society of Automotive Engineers (5) defined six
levels of AVs, ranging from no automation (Level 0) to
full automation (Level 5). The focus of this paper is full
driving automation (SAE Level 5), since it is likely to
provide the most societal benefits. A variety of advan-
tages and disadvantages may arise from the emergence of
AVs. Major potential benefits include improved mobility
1
Department of Occupational Therapy, University of Florida, Gainesville, FL
2
Department of Civil, Construction, and Environmental Engineering,
University of Alabama at Birmingham, Birmingham, AL
Corresponding Author:
Justin Mason, JustinMason@phhp.ufl.edu
2 Transportation Research Record 0(0)
for the elderly and disabled (7) and the liberating of
parking spaces for other land uses (2). However, adop-
tion of AVs may cause increased traffic congestion,
resulting from increased travel overall owing to
enhanced availability and lower costs, along with
empty cars traveling the roads when collecting their
users or returning from trips (8). The potential disadvan-
tages also include concerns relating to privacy, security,
insurance, and liability, as well as job losses (9). The
extent to which these positive and negative outcomes
eventuate will be highly dependent on whether users
accept and adopt this emerging technology.
A multitude of automotive manufacturers, technology
companies, and institutions are developing innovative
technology to address transportation safety and equity
for users across the lifespan and mobility spectrums.
These developers must create technologies that are safe
and efficient, while also being acceptable and adoptable
by the intended users. Recent studies have suggested that
AVs should be safer than human drivers for transporta-
tion users to adopt and accept this technology (10,11).
Thus, the public will be less likely to embrace AVs if they
have the same risk level as human driving. Furthermore,
individuals increase their demand for safety when they
entrust their safety to an external factor such as an AV
(11). Specifically, Liu and colleagues (12) found that AVs
should be four to five times as safe (i.e., 75% to 80%
reduction in traffic fatalities) as human drivers, if they
are to be tolerated and accepted. Although safety is a
critical predictor, several other factors influence user per-
ceptions and behavioral intentions (e.g., perceived use-
fulness, perceived ease of use, and trust).
To understand travelers’ perceptions and attitudes
toward AV technology adoption, the AV User
Perception Survey was constructed to quantify transpor-
tation users’ perceptions of AVs. Conducting an exten-
sive literature review was an essential prerequisite to
generating survey items and assessing face and content
validity (13). The following section provides an overview
of models used for determining acceptance and adoption
of technology.
Models for Acceptance and Adoption of
Technology
Technology Acceptance Model (TAM)
The technology acceptance model (TAM) and extended
versions of the model are explained in relation to its core
assumptions, strengths, and limitations. The TAM (14)
postulates that the use of an information system is deter-
mined by the behavioral intention of a user, which is
mediated by perceived usefulness and perceived ease of
use. Perceived usefulness indicates the extent to which
individuals believe the use of a system would improve
their job performance, whereas perceived ease of use
indicates to what extent individuals believe the use of a
system will be free of effort. Technology that is perceived
to be easier to use as well as useful is more likely to be
accepted by users (14). TAM consistently explains about
40% of the variance in individuals’ intention to use
versus the actual usage of informational technology
(15,16). Although the TAM provides a conceptual
framework for determining user ease and usefulness, it
has been criticized for its lack of predictive power and
overlooked constructs: cost, cultural differences, and
social aspects of decision making (17).
Unified Theory of Acceptance and Use of Technology
(UTAUT)
Venkatesh and colleagues integrated eight different
models of acceptance into the unified theory of accep-
tance and use of technology (UTAUT), designed to cap-
ture all the factors affecting intention to use a particular
technology (16). The UTAUT posits that performance
expectancy, effort expectancy, social influence, and con-
ditions that facilitate technology acceptance are critical
constructs. Although the UTAUT is a comprehensive
synthesis of prior technology acceptance research, it
presents a model with 41 independent variables for pre-
diction of intentions and at least 8 independent variables
for predicting behavior (17). However, these constructs
may be influenced by gender, age, experience, and vol-
untariness of use (18). Madigan and colleagues found
that the UTAUT framework accounted for over half of
the variance in intention to use automated road trans-
portation systems (18). Further models have been devel-
oped as an extension of the TAM and UTAUT,
specifically focusing on individuals’ adoption of
advanced driver assistance systems (ADAS; i.e., lane
keeping assist) or in-vehicle information systems (IVIS;
i.e., lane departure warning) (15,19).
Car Technology Acceptance Model (CTAM)
The car technology acceptance model (CTAM) (15) was
developed by integrating the TAM and UTAUT. The
modeling approach supports decision processes in rela-
tion to IVIS implementation in the automotive industry.
IVIS provide contextual information (i.e., driving speed
or position of the car) to deliver driving-related informa-
tion and support to the driver. The CTAM incorporates
the UTAUT model with the addition of perceived stress,
safety, and anxiety. As such, the benefit of this model is
the consideration of safety and anxiety, as these percep-
tions may be affected by adding stimuli (i.e., information
from IVIS) to driving, an already complex instrumental
activity of daily living. The primary disadvantage to this
Mason et al.
model is the lack of empirical support, as CTAM was
developed in 2012 with 21 subjects completing their ques-
tionnaire. To our knowledge, no further studies have
utilized this model or the questionnaire generated.
4P Acceptance Model
The 4P (i.e., pod-like vehicles) Acceptance Model (20)
was influenced by UTAUT and the pleasure-arousal-
dominance framework (21). The 4P acceptance model
guided survey construction designed to elucidate user
acceptance of SAE Level 4 vehicles or driverless pod-
like vehicles without a steering wheel that operated
within the constraints of dedicated infrastructure. The
purpose of the 4P Acceptance Model is to prevent gen-
eralizability to AVs that will operate outside of a closed-
loop. The advantage of the 4P Acceptance Model is the
holistic and comprehensive view of user acceptance (22
components) that goes beyond the attributes of AVs. For
instance, components include vehicle characteristics,
contextual characteristics, mobility characteristics,
socio-demographics, willingness to pay, and arousal.
Nordhoff and colleagues conducted interviews and sub-
sequently developed a 94-item survey that still requires
validation (22). The international survey was designed to
examine cross-national effects, both an advantage and
disadvantage because of cultural and societal factors.
Safety Critical Technology Acceptance Model (SCTAM)
The safety critical technology acceptance model
(SCTAM) (19) was developed from the TAM (14) but
proposes additional focus on the psychology of control,
acceptance, and trust that influence the use of a safety
critical technology. Preliminary results suggest that
authority (i.e., approval from a governing body) is the
single greatest factor in the addition of the SCTAM (19).
Likewise, the TAM-extended framework (23), also devel-
oped from the TAM, indicates that four constructs (i.e.,
perceived usefulness, perceived ease of use, trust, and
social influence) affect users’ intention to use AVs.
Perceived usefulness was the strongest predictor of user
behavioral intention, suggesting that AVs must provide
functional benefits (i.e., freeing up users’ time and sim-
plifying their lives) if users are to adopt this technology.
Self-Driving Car Acceptance Scale (SCAS)
The findings from the self-driving car acceptance scale
(SCAS) (24) suggest that drivers must establish realistic
expectations about the performance of automation
before interacting with AVs to facilitate long-term accep-
tance. The 24-item scale was utilized in an experiment
that found that people were more accepting of AVs after
reading a vignette featuring an idealized portrayal of
3
perfect automation in self-driving cars as compared to
a scenario that described a more realistic situation in
which the human driver played a monitoring role and
occasionally intervened during vehicle automation.
Pitfalls of Previous Models
Numerous researchers have surveyed consumer percep-
tions about vehicle automation (25) using a variety of
surveys (6,15,24,26). However, the majority of these
efforts were focused on potential consumers who may
purchase this technology, which may not occur given
the potential financial and ecological constraints of
owning and maintaining an AV (27). Moreover, dynamic
ride-sharing schemes involving AVs may be implemented,
resulting in reduced private car ownership (2). Findings
from previous research showed that the determinants of
user perception on acceptance of AVs are largely
unknown. Furthermore, numerous researchers (28) have
focused on ADAS, IVIS or both systems (15) rather than
AVs with full automation (SAE Levels 4 and 5). In the
reviewed studies, researchers developed empirically based
models and generated survey items but did not report
their method for assessing face or content validity.
The extended models have succumbed to similar pit-
falls to the models they are based on, that is, UTAUT or
TAM. The UTAUT is criticized for being holistic (i.e.,
too many dimensions), whereas the TAM is criticized for
not considering important variables that may influence
users’ acceptance and adoption of technology. Although
the TAM and UTAUT have overlapping constructs,
most of the AV-related acceptance models (i.e., SCAS,
SCTAM, 4P Acceptance Model) are influenced by
UTAUT or TAM rather than both models. Similar to
CTAM, future models should integrate UTAUT and
TAM, since both models have been utilized to predict
users’ behavioral intentions with a variety of technology.
The primary objective of this study was to construct
the AV User Perception Survey derived from the social-
psychological models discussed, to assess perceptions of
user acceptance of AV technology. Therefore, the study
had two aims: a) develop a conceptual framework to
inform item generation for a survey to determine user
perception of acceptance of AVs; and b) determine the
face and content validity of the AV User Perception
Survey. Several constructs from the 4P Acceptance
Model were included in our conceptual model, including
external variables (i.e., social influence), trust, perceived
ease of use (i.e., effort expectancy), perceived usefulness,
and control.
It is unclear whether AV technologies will follow tra-
ditional vehicle ownership trends (i.e., private AV),
public transportation, shared or pooled AV. This is dif-
ficult to predict owing to policy, economic concerns,
infrastructure, and perspective. Taking into account
4 Transportation Research Record 0(0)
Figure 1. Conceptual model for survey development.
numerous benefits to travelers and assuming that the
price for using a self-driving vehicle does not restrict
use, the main remaining determinant of adoption is the
users’ perception of acceptance. However, no valid and
reliable self-report measure of individuals’ perception of
acceptance to fully AVs exists that is relevant or appro-
priate for a broad population across the lifespan.
Information gathered from this measure will not only
form the foundation for future validation work, but
may be used to inform policy makers, engineers,
and manufacturers to promote acceptance and
adoption of AVs.
To successfully deploy AV systems, users’ perception
of acceptance will need to be quantified in an attempt to
understand their apprehension, trust, and acceptance of
this revolutionary technology. The purpose of this study
was to report on the item development and validity of a
new survey to assess adults’ perception of acceptance to
AVs. Specifically, the survey in this study was developed
to quantify users’ perception of acceptance to fully AVs
(SAE Levels 4 and 5).
Methods
The Institutional Review Board approved the study as
exempt (IRB201802574). This study was a subsection of
the parent study that investigated drivers’ perception of
acceptance of AVs.
Process for Item Development and Design
The initial stage of instrument development was per-
formed in three steps: identifying the content domain,
generating sample items, and constructing the instru-
ment (29). A conceptual model (Figure 1) was developed
from seven acceptance models: 1) TAM (14), 2) SCTAM
(19), 3) CTAM (15), 4) UTAUT (16), 5) TAM-extended
framework (23), 6) SCAS (24), and 7) 4P Acceptance
Model (20). The conceptual model integrated variables
(i.e., constructs) from the seven aforementioned models
(Figure 1). The numbers (i.e., 1–7 above) correspond
with the numbers in the conceptual model. The arrows
represent expected relationships between the dimensions,
though these have not yet been tested. Survey items were
written to reflect eight potential sub-dimensions: (a)
intention to use, (b) perceived ease of use, (c) perceived
usefulness, (d) safety, (e) trust and reliability, (f) experi-
ence, (g) control and driving-efficacy, and (h) external
variables (i.e., media, governing authority, social influ-
ence, and cost). Twenty-one of 35 items were generated
by the authors whereas 14 items were modified from pre-
vious surveys (14,24,30–32). Self-generated items and
their potential domains were chosen to align with
TAM, UTUAT, and extended models while also inte-
grating additional themes that arose during subsequent
qualitative studies (33,34). Dimensions, items, and their
respective sources are detailed in the Analysis section.
AV technologies refer to a variety of terms and defini-
tions that explain the spectrum of vehicle automation.
However, it is not clear whether potential users fully
understand the terms, capabilities, and various levels of
automation. Recently, several models have been proposed
to predict and explain users’ adoption of ADAS/IVIS
(15,19,20,24). The survey was designed to elicit users’ per-
ception of acceptance to fully AVs (SAE Level 4 or 5).
Participants were prompted by the statement:
An automated vehicle (i.e., self-driving vehicle, driverless
car, self-driving shuttle) is a vehicle that is capable of
sensing its environment and navigating without human
input. Full-time automation of all driving tasks on any
road, under any conditions, and does not require a driver
nor a steering wheel.
Visual Analogue Scale
In this study, 35 visual analogue scale (VAS) items were
developed with verbal anchors, ranging from disagree to
agree. VAS questionnaires have been used for the last
Mason et al.
century to measure subjective phenomena owing to their
brevity, universality, ease of use, and scoring (35).
Researchers can utilize a VAS with pencil/paper, elec-
tronic, or online data collection. The conventional pre-
test–posttest self-report (i.e., Likert scale) is limited in its
assessment of change, whereas the VAS allows for finer
distinctions (36). The VAS is a continuous scale, typical-
ly with two descriptors (i.e., verbal anchors) on the
extremes of a 100-mm horizontal line (37).
Respondents rate their perceptions by making a mark
(i.e., vertical slash) corresponding to their level of agree-
ment/disagreement. The distance between the marked
point and the origin of the line is measured to quantify
the magnitude of the response. In addition, four open-
ended items were developed to allow individuals to con-
sider and provide their own ideas, thoughts, and
feelings (38).
Face and Content Validity
It is imperative that researchers establish face and content
validity before administering the questionnaire and assess-
ing construct-, criterion validity, or both (39). Face valid-
ity is an initial judgment of whether a tool assesses the
concept it purports to measure (40) and refers to how
items are to be interpreted by the intended audience
(i.e., layperson) who will complete the survey (41).
Laypersons, rather than subject-matter experts, should
assess face validity. The 40-item survey was presented to
the Institute for Mobility, Activity, and Participation
(I-MAP) team (two quantitative researchers, one qualita-
tive researcher, one clinician, five rehabilitation science
or civil engineering doctoral students, and two under-
graduate students) who provided input and recommenda-
tions on the wording, clarity, and comprehension of the
items.
Content validity, like face validity, is subjective and
refers to the extent to which a measurement reflects a
specific domain of content (42). It also measures the
degree to which elements of the measurement instrument
are comprehensive, relevant, and representative of the
construct for a particular assessment purpose (43).
Three or more raters are needed to provide a rigorous
rating (44) and raters should have expertise in the con-
tent area under investigation (45). Content validity index
(CVI) results are used to refine the items and the CVI
process is repeated until an acceptance level is reached
(i.e., scale CVI > .90; [46]).
Following the guidelines of Lynn, seven subject-
matter experts rated the content validity of the survey
(44). Subject-matter experts were selected to represent
relevant domains: one expert in cognitive psychology
(20 years of experience), one expert in measurement
and survey design (45 years of experience), three experts
in transportation engineering (a combined 70 years of
5
experience), and two experts in human factors (a com-
bined 45 years of experience). The experts, who repre-
sented a mix between academia and industry, provided
their feedback via a Qualtrics survey by rating the rele-
vance of each item on a four-point Likert scale (1 ¼ not
relevant, 2 ¼ relevant with major revisions, 3 ¼ relevant
with minor revisions, and 4 ¼ very relevant). They also
provided qualitative feedback on item accuracy, organi-
zation, clarity, appearance, purpose, understandability,
and adequacy (45).
Analysis
To establish face validity, the I-MAP team provided
feedback on items’ order, clarity, and how the items
were to be interpreted by the intended audience who
would complete the survey (41). Items were discussed
and revised during a meeting to incorporate feedback.
Content validity included calculation of both an item-
level CVI (I-CVI) and the scale CVI (S-CVI). Using
CVI procedures (44), rater scores were collapsed, with
an item-level score of 3 or 4 indicating acceptable item
relevance, and a score of 1 or 2 indicating the need for a
major revision or low item-relevance. After exporting all
ratings and comments to RStudio (RStudio, Boston,
MA), we calculated item-level CVI scores (i.e., the pro-
portion of the seven raters who scored the item as rele-
vant, a rating of 3 or 4) and the scale average CVI. Item-
level CVIs of 0.86 or 1.00 were acceptable (0.86 ¼ the
item was rated as relevant by six raters; 1.00 ¼ the item
was rated as relevant by seven raters), whereas scores of
0.71 or below (0.71 ¼ the item was rated as relevant by
five raters) were unacceptable. We used the statistical
package R, version 3.5.2 (47) for collating data and com-
puting the CVI. After the analysis, items with a low item
CVI (¼ 0.71) were revised by the research team, whereas
items with scores 0.57 or below (0.57 ¼ the item was
rated as relevant by four raters) were removed from
the survey. The CVI process and item refinement were
repeated until an acceptable level of content validity was
reached (average CVI  0.80; [48]). Items were ordered
thematically in relation to the domain they were intended
to represent, as this has been shown to enhance internal
consistency reliability (49,50).
Results
Face Validity
The I-MAP team received a paper copy of the survey and
was instructed to critique the survey and provide sugges-
tions to make the survey understandable to a layperson
across the lifespan. Afterwards, each item was read by a
researcher and the I-MAP team provided their feedback
and discussed potential edits. Feedback from the I-MAP
6 Transportation Research Record 0(0)

Table 1. Items and Expected Dimensions

Source for
Dimension Item # modification

Experience with I use technology to make tasks easier for me 1 Nees, 2016
technology
I use technology in my vehicle to make tasks easier for me 2 Self-developed
I have had bad experiences when I try to use new technology instead of 3 Nees, 2016
doing things “the old-fashioned way”
Intention to use I am open to the idea of using automated vehicles 4 Self-developed
I would use an automated vehicle on a daily basis 15 Davis, 1989
I would rarely use an automated vehicle 16 Self-developed
Trust/reliability I am suspicious of automated vehicles 5 Gold et al., 2015
I can trust automated vehicles 6 Choi and Ji, 2015
I will engage in other tasks while riding in an automated vehicle 7 Gold et al., 2015
I feel hesitant about using an automated vehicle 28 Cho et al., 2017
Perceived usefulness Automated vehicles will reduce traffic congestion 8 Self-developed
Automated vehicles will assist with parking 9 Self-developed
Automated vehicles will allow me to stay active 10 Self-developed
Automated vehicles will allow me to stay involved in my community 11 Self-developed
Automated vehicles will enhance my quality of life/well-being 12 Self-developed
Perceived ease of use Automated vehicles will be easy to use 13 Nees, 2016
A lot of effort is required to figure out how to use an automated vehicle 14 Nees, 2016
Safety When I’m riding in an automated vehicle, other road users will be safe 25 Self-developed
Automated vehicles will increase the number of crashes 26 Gold et al., 2015
I feel safe riding in an automated vehicle 27 Nees, 2016
Control/driving-efficacy Even if I had access to an automated vehicle, I would still want to drive 17 Nees, 2016
myself
I prefer the option to drive myself by turning off the automated system 18 Nees, 2016
My driving abilities will decline because of relying on an automated 19 Self-developed
vehicle
External variables
Cost I will be willing to pay more for an automated vehicle compared to what I 20 Nees, 2016
would pay for a traditional car
If cost was not an issue, I would use an automated vehicle 21 Self-developed
Authority I would use an automated vehicle if National Highway Traffic Safety 22 Self-developed
Administration (NHTSA) deems them as being safe
Media Media portrays automated vehicles in a positive way 23 Self-developed
Social influence My family and friends will encourage/support me when I use an auto- 24 Self-developed
mated vehicle
Open-ended Describe influences that may promote your willingness to use automated 29 Self-developed
vehicles
Describe influences that may deter you from using automated vehicles 30 Self-developed
Describe potential benefits of automated vehicles 31 Self-developed
Describe potential disadvantages of automated vehicles 32 Self-developed

team was used to assess face validity of the survey. Out
of the 40 items generated from the literature review, 30
(75%) items were approved without edits, 9 (22.5%)
items were revised and subsequently accepted, and 1
item (2.5%) was removed. These items were rephrased
to avoid leading questions, reduce ambiguity, and be rel-
evant to adults of all ages. For example, “utilize” was
changed to “use” and “drivers and pedestrians” was
changed to “road users.” One item was dropped because
of the lack of relevance to the target age group (i.e., it
was focused on older adults’ perceptions). Lastly, an
item was rephrased to assess users’ perceived usefulness
rather than users’ intention to use. The suggested
changes were discussed by the I-MAP team and the
members agreed unanimously to accept the revised items.
Content Validity Index: First Round
After establishing face validity, seven subject-matter
experts provided relevance ratings and extensive feed-
back for the survey. The first round of content validity
consisted of 39 items with an overall scale CVI of 84%
Mason et al. 7

Table 2. Dimensions and Content Validity Index (CVI) after Team Revisions and Expert Ratings

Dimension Items Item-level CVI range Item-level CVI mean

Experience with technology 3 1.00 1.00

Intention to use 3 0.86–1.00 0.95
Trust/reliability 4 0.86–1.00 0.97
Perceived usefulness 5 0.86–1.00 0.97
Perceived ease of use 2 0.86–1.00 0.93
Safety 3 0.86–1.00 0.91
Control/driving-efficacy 3 0.86–1.00 0.95
External variables
Cost 2 1.00 1.00
Authority 1 0.86 0.86
Media 1 0.86 0.86
Social influence 1 1.00 1.00
Open-ended 4 1.00 1.00
Total scale 32 0.86–1.00 0.96

(mean relevance for all items), with 28 of 39 (71.8%)
items rated greater than or equal to 0.86 (six of seven
experts rated the item as relevant). Six items were
removed from the survey with an item rating of 43%
(three of seven experts rated the item as relevant). Five
items with an item rating of 71% (five of seven experts
rated the item as relevant) were amended and sent back
to the experts for further evaluation. One expert matched
each survey item with the intended domain/construct on
the conceptual model. This was in alignment with our
intended item design and domain representation. As sug-
gested by four reviewers, an item designed to assess
users’ trust, was moved to the end of the VAS items in
an attempt to elicit users’ open-ended responses.
Content Validity Index: Second Round
All seven experts provided relevance ratings and feedback
for the five revised items. In the second round of content
validity, four of five items had a CVI greater than or equal
to 0.86 (at least six of seven experts rated the item as rel-
evant) and were thus accepted without changes. The other
item was removed from the survey as it had a CVI below
70%. The final survey (Table 1) consisted of 32 items (28
VAS and 4 open-ended), with a scale CVI of 96% (Mean
CVI of all items) and 32 out of 32 items (100%) rated
greater than or equal to 0.86. Both scale CVI values
(Table 2) indicate acceptable content validity (51).
Discussion
The purpose of this study was to report on item devel-
opment and validity of a new survey to assess adults’
perception of acceptance to AVs. Specifically, the
survey in this study was developed to quantify users’
perception of acceptance to fully AVs (SAE Level 5).
Item generation and development was guided by the con-
ceptual model, which was empirically founded from
seven acceptance models (Figure 1). The findings
revealed that a multi-staged and iterative approach in
survey development was essential. These findings were
consistent with other measures that describe survey con-
struction (52,53). Using a mixed-methods approach (i.e.,
CVI and open-ended responses) was advantageous in
selecting, rephrasing, and removing items based on feed-
back from the I-MAP team and subject-matter experts.
The main contribution of this study was in elucidating
the methods to develop the survey via conceptualization,
item generation, and face and content validation.
Previous related studies have not adequately reported
their methods to establish face or content validity of
their survey. For instance, Panagiotopoulos and
Demitrakopoulos claimed content validity by modifying
items from published studies in which those researchers
had selected items from extant literature that also failed
to report face or content validity procedures (23). A
more rigorous approach must be utilized and described
in future studies before using inferential validation tech-
niques (i.e., construct-, criterion validity, or both).
Establishing face and content validity is an essential
first step for survey validation, although more work is
required to determine this instrument’s reliability and
validity.
The acceptance of owning or purchasing an AV has
received some attention (26,54), but only two studies
have investigated behavioral intentions to use automated
forms of public transport (18,55). The conceptual model
and survey in this study were developed to assess road
users’ intent to use rather than intent to own as seen in
previous studies (23,26,54,56). Both private ownership
and intent to use (e.g., willingness to ride [57]) are impor-
tant aspects to consider in relation to future
8 Transportation Research Record 0(0)
transportation trends. However, these constructs should
be carefully separated as they are likely affected by a
plethora of different variables (i.e., socioeconomic
status, environmental factors, area of residence, etc.).
Separation is a logical next step to occur via factor anal-
ysis (in development).
Previous acceptance studies focused on automation
levels lower than SAE Level 4, typically sampling indi-
viduals who had little to no exposure to AVs. Exposure
and utilization of lower-level (SAE Levels 1 and 2) vehi-
cle automation may play an important role in the suc-
cessful transition of road users toward vehicles with
higher levels of automation (58). Without appropriate
exposure, individuals will have limited experience in
which to ground their expectations. Clearly, targeted
inquiry may further reveal factors that will influence
both short- and long-term acceptance of fully AVs.
The survey will be used to quantify users’ perception of
acceptance to AVs and be utilized in an experimental,
repeated-measures crossover design before and after
users are exposed to an AV (SAE Level 4) and driving
simulator in automated mode (SAE Level 4). The next
steps are to discern reliability and construct validity of
the survey, post data collection.
Conclusions
The approach adopted in this study ensured that the
survey instrument design was effective and included
items that were relevant, concise, and clear.
Specifically, the conceptual model guided item genera-
tion from extant literature, followed by an assessment
to determine face validity, and two rounds of reviews
from subject-matter experts to establish content validity.
The survey may be utilized to assess road users’ accep-
tance of AVs and potentially predict their intent to use
this innovative technology. However, researchers should
be cautious when drawing conclusions from data collect-
ed via the AV User Perception Survey, until it has been
sufficiently validated.
Acknowledgments
The Institute for Mobility, Activity, and Participation provided
infrastructure and support for this study. We are very appre-
ciative to the seven experts who provided relevance ratings and
extensive feedback during the development of the survey.
Author Contributions
The authors confirm contribution to the paper as follows: study
conception and design: SC, VS, JM; data collection: JM, SC;
analysis and interpretation of results: JM, SC; draft manuscript
preparation: JM, VS, SC, JW. All authors reviewed the results
and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) disclosed receipt of the following financial support
for the research, authorship, and/or publication of this article:
This work was sponsored by the United States Department of
Transportation Office of the Assistant Secretary for Research
and Technology (OST-R) through the Southeastern Transpor-
tation Research, Innovation, Development, and Education
Center (Project D2).
References
1. World Health Organization. Global Health Observatory
(GHO) Data. https://www.who.int/gho/road_safety/en/.
Accessed July 10, 2019.
2. Fagnant, D. J., and K. Kockelman. Preparing a Nation for
Autonomous Vehicles: Opportunities, Barriers and Policy
Recommendations. Transportation Research Part A: Policy
and Practice, Vol. 77, 2015, pp. 167–181.
3. Piao, J., M. Mcdonald, N. Hounsell, M. Graindorge, and
N. Malhene. Public Views towards Implementation of
Automated Vehicles in Urban Areas. Transportation
Research Procedia, Vol. 14, 2016, pp. 2168–2177.
4. U.S. Department of Transportation. Federal Automated
Vehicles Policy: Accelerating the Next Revolution in
Roadway Safety. U.S. Department of Transportation,
Washington, D.C., 2016. https://www.transportation.gov/
AV/federal-automated-vehicles-policy-september-2016.
Accessed July 10, 2019.
5. SAE International. Taxonomy and Definitions for Terms
Related to On-Road Motor Vehicle Automated Driving
Systems: J 3016_201806. SAE International, Warrendale,
PA, 2016.
6. Payre, W., J. Cestac, and P. Delhomme. Intention to Use a
Fully Automated Car: Attitudes and A Priori
Acceptability. Transportation Research Part F: Traffic
Psychology and Behaviour, Vol. 27, 2014, pp. 252–263.
7. Yang, J., and J. F. Coughlin. In-Vehicle Technology for
Self-Driving Cars: Advantages and Challenges for Aging
Drivers. International Journal of Automotive Technology,
Vol. 15, No. 2, 2014, pp. 333–340.
8. Pettigrew, S., L. Fritschi, and R. Norman. The Potential
Implications of Autonomous Vehicles in and Around the
Workplace. International Journal of Environmental
Research and Public Health, Vol. 15, No. 9, 2018, pp. 1–10.
9. Taeihagh, A., and H. Si Min Lim. Governing Autonomous
Vehicles: Emerging Responses for Safety, Liability,
Privacy, Cybersecurity, and Industry Risks. Transport
Reviews, Vol. 39, No. 1, 2019, pp. 103–128.
10. Shladover, S. E., and C. Nowakowski. Regulatory
Challenges for Road Vehicle Automation: Lessons from
the California Experience. Transportation Research Part
A: Policy and Practice Vol. 122, 2017, pp. 125–133.
Mason et al.
11. Waycaster, G. C., T. Matsumura, V. Bilotkach, R. T.
Haftka, and N. H. Kim. Review of Regulatory Emphasis
on Transportation Safety in The United States, 2002–2009:
Public Versus Private Modes. Risk Analysis, Vol. 38, No. 5,
2018, pp. 1085–1101.
12. Liu, P., R. Yang, and Z. Xu. How Safe is Safe Enough for
Self-Driving Vehicles? Risk Analysis, Vol. 39, No. 2, 2019,
pp. 315–325.
13. Tenenbaum, G., R. C. Eklund, and A. Kamata.
Measurement in Sport and Exercise Psychology. 1st ed.
Human Kinetics, Champaign, IL, 2012.
14. Davis, F. D. Perceived Usefulness, Perceived Ease of Use,
and User Acceptance of Information Technology. Manage-
ment Information Systems Quarterly, Vol. 13, No. 3, 1989,
pp. 319–340.
15. Osswald, S., D. Wurhofer, S. Tr€ osterer, E. Beck, and M.
Tscheligi. Predicting Information Technology Usage in the
Car: Towards a Car Technology Acceptance Model. Proc.,
AutomotiveUI ’12, 4th International Conference on
Automotive User Interfaces and Interactive Vehicular
Applications, Portsmouth, NH, 2012, pp. 51–358.
16. Venkatesh, V., M. G. Morris, M. Hall, G. B. Davis, F. D.
Davis, and S. M. Walton. User Acceptance of Information
Technology: Toward a Unified View. Management
Information Systems Quarterly, Vol. 27, No. 3, 2003, pp.
425–478.
17. Bagozzi, R. P. The Legacy of the Technology Acceptance
Model and a Proposal for a Paradigm Shift. Journal of the
Association for Information Systems, Vol. 8, No. 4, 2007,
pp. 244–254.
18. Madigan, R., T. Louw, M. Wilbrink, A. Schieben, and N.
Merat. What Influences the Decision to Use Automated
Public Transport? Using UTAUT to Understand Public
Acceptance of Automated Road Transport Systems.
Transportation Research Part F: Traffic Psychology and
Behaviour, Vol. 50, 2017, pp. 55–64.
19. Hutchins, N., and L. Hook. Technology Acceptance
Model for Safety Critical Autonomous Transportation
Systems. Proc., 2017 AIAA/IEEE 36th Digital Avionics
Systems Conference, St. Petersburg, FL, 2017, pp. 1–5.
20. Nordhoff, S., B. van Arem, and R. Happee. Conceptual
Model to Explain, Predict, and Improve User Acceptance
of Driverless Podlike Vehicles. Transportation Research
Record: Journal of the Transportation Research Board,
2016. 2602: 60–67.
21. Mehrabian, A. Pleasure-Arousal-Dominance: A General
Framework for Describing and Measuring Individual
Differences in Temperament. Current Psychology, Vol.
14, No. 4, 2007, pp. 261–292.
22. Nordhoff, S., J. de Winter, M. Kyriakidis, B. Van Arem,
and R. Happee. Acceptance of Driverless Vehicles: Results
from a Large Cross-National Questionnaire Study. Journal
of Advanced Transportation, Vol. 2018, 2018.
23. Panagiotopoulos, I., and G. Dimitrakopoulos. An
Empirical Investigation on Consumers’ Intentions
Towards Autonomous Driving. Transportation Research
Part C: Emerging Technologies, Vol. 95, 2018, pp. 773–784.
9
24. Nees, M. A. Acceptance of Self-Driving Cars. Proceedings
of the Human Factors and Ergonomics Society Annual
Meeting, Vol. 60, No. 1, 2016. pp. 1449–1453.
25. Becker, F., and K. W. Axhausen. Literature Review on
Surveys Investigating the Acceptance of Automated
Vehicles. Transportation, Vol. 44, No. 6, 2017,
pp. 1293–1306.
26. Kyriakidis, M., R. Happee, and J. C. F. de Winter. Public
Opinion on Automated Driving: Results of an
International Questionnaire Among 5000 Respondents.
Transportation Research Part F: Traffic Psychology and
Behaviour, Vol. 32, 2015, pp. 127–140. Available from:
http://dx.doi.org/10.1016/j.trf.2015.04.014.
27. Fagnant, D. J., and K. M. Kockelman. The Travel and
Environmental Implications of Shared Autonomous
Vehicles, using Agent-Based Model Scenarios. Transpor-
tation Research Part C: Emerging Technologies, Vol. 40,
2014, pp. 1–13.
28. Classen, S., M. Jeghers, J. Morgan-Daniel, S. Winter, L.
King, and L. Struckmeyer. Smart In-Vehicle Technologies
and Older Drivers: A Scoping Review. OTJR Occupation
Participation and Health, Vol. 39, No. 2, 2019, pp. 97–107.
29. Zamanzadeh, V., A. Ghahramanian, M. Rassouli, A.
Abbaszadeh, H. Alavi-Majd, and A-R. Nikanfar. Design
and Implementation Content Validity Study: Development
of an Instrument for Measuring Patient-Centered
Communication. Journal of Caring Sciences, Vol. 4, No.
2, 2015, pp. 165–178.
30. Gold, C., M. K€ orber, C. Hohenberger, D. Lechner, and K.
Bengler. Trust in Automation – Before and after the
Experience of Take-Over Scenarios in a Highly
Automated Vehicle. Procedia Manufacturing, Vol. 3,
2015, pp. 3025–3032.
31. Choi, J. K., and Y. G. Ji. Investigating the Importance of
Trust on Adopting an Autonomous Vehicle. International
Journal of Human–Computer Interaction, Vol. 31, No. 10,
2015, pp. 692–702.
32. Cho, Y., J. Park, S. Park, and E. S. Jung. Technology
Acceptance Modeling Based on User Experience for
Autonomous Vehicles. Journal of the Ergonomics Society
of Korea, Vol. 36, No. 2, 2017, pp. 87–108.
33. Buckley, L., S. A. Kaye, and A. K. Pradhan. A Qualitative
Examination of Drivers’ Responses to Partially Automated
Vehicles. Transportation Research Part F: Traffic
Psychology and Behaviour, Vol. 56, 2018, pp. 167–175.
34. Nordhoff, S., J. de Winter, W. Payre, B. van Arem, and R.
Happee. What Impressions Do Users Have after a Ride in
an Automated Shuttle? An Interview Study. Transportation
Research Part F: Traffic Psychology and Behaviour, Vol. 63,
2019, pp. 252–269.
35. Wewers, M. E., and N. K. Lowe. A Critical Review of
Visual Analogue Scales in the Measurement of Clinical
Phenomena. Research in Nursing and Health, Vol. 13,
No. 4, 1990, 227–236.
36. Rausch, M., and M. Zehetleitner. A Comparison between a
Visual Analogue Scale and a Four Point Scale as Measures
10
of Conscious Experience of Motion. Consciousness and
Cognition, Vol. 28, 2014, pp. 126–140.
37. Jensen, M. P., C. Chen, and A. M. Brugger. Interpretation
of Visual Analog Scale Ratings and Change Scores: A
Reanalysis of Two Clinical Trials of Postoperative Pain.
The Journal of Pain, Vol. 4, No. 7, 2003, pp. 407–414.
38. Creswell, J. W., and V. L. P. Clark. Designing and
Conducting Mixed Methods Research. 2nd ed. Thousand
Oaks, CA, Sage, 2011.
39. Shrotryia, V. K., and U. Dhanda. Content Validity of
Assessment Instrument for Employee Engagement. SAGE
Open, 2019, pp. 1–7.
40. Gravetter, F. J., and Forzano LB. Research Methods for
the Behavioral Sciences. 4th ed. Wadsworth, Belmont, CA,
2012.
41. Streiner, D. L., and G. R. Norman. Health Measurement
Scales: A Practical Guide to Their Development and Use.
Oxford University Press, Oxford, 1989.
42. Lawshe, C. A Quantitative Approach to Content Validity.
Personnel Psychology, Vol. 28, No. 4, 1975, pp. 563–575.
43. Haynes, S., D. Richard, and E. Kubany. Content Validity
in Psychological Assessment: A Functional Approach to
Concept and Methods. Psychological Assessment, Vol. 7,
No. 3, 1995, pp. 238–247.
44. Lynn, M. R. Determination and Quantification of Content
Validity. Nursing Research, Vol. 35, No. 6, 1986, pp. 382–385.
45. Grant, J. S., and L. L. Davis. Selection and Use of Content
Experts for Instrument Development. Research in Nursing
and Health, Vol. 20, No. 3, 1997, pp. 269–274.
46. Waltz, C. F., O. L. Strickland, and E. R. Lenz.
Measurement in Nursing and Health Research. 4th ed.
Springer Publishing Company, New York, NY, 2010.
47. RStudio Team. RStudio: Integrated Development for R.
RStudio, Inc., Boston, MA. 2019.
48. House, A. E., B. J. House, and M. B. Campbell. Measures
of Interobserver Agreement: Calculation Formulas and
Distribution Effects. Journal of Behavioral Assessment,
Vol. 3, No. 1, 1981, pp. 37–57.
49. Lam T. C. M., K. E. Green, and C. Bordignon. Effects of
Item Grouping and Position of the “Don’t Know” Option
on Questionnaire Response. Field Methods, Vol. 14, No. 4,
2002, pp. 418–432.
Transportation Research Record 0(0)
50. Melnick, S. A. The Effects of Item Grouping on the
Reliability and Scale Scores of an Affective Measure.
Educational and Psychological Measurement, Vol. 53, No.
1, 1993, pp. 211–216.
51. Polit, D. F., and C. T. Beck. The Content Validity Index:
Are You Sure You Know What’s Being Reported? Critique
and Recommendations. Research in Nursing and Health,
Vol. 29, No. 5, 2006, pp. 489–497.
52. Bonomi, A. E., D. L. Patrick, D. M. Bushnell, and M.
Martin. Validation of the United States’ Version of the
World Health Organization Quality of Life (WHOQoL)
Instrument. Journal of Clinical Epidemiology, Vol. 53,
No. 1, 2000, pp. 1–12.
53. Patrick, D. L., T. C. Edwards, and T. D. Topolski.
Adolescent Quality of Life, Part II: Initial Validation of a
New Instrument. Journal of Adolescence, Vol. 25, No. 3,
2002, pp. 287–300.
54. Schoettle, B., and M. Sivak. A Survey of Public Opinion
about Autonomous and Self-Driving Vehicles in the U.S.,
the U.K., and Australia. University of Michigan Transpor-
tation Research Institute, Ann Arbor, MI, 2014.
55. Madigan, R., T. Louw, M. Dziennus, T. Graindorge, E.
Ortega, M. Graindorge, and N. Merat. Acceptance of
Automated Road Transport Systems (ARTS): An
Adaptation of the UTAUT. Transportation Research
Procedia, Vol. 14, 2016, pp. 2217–2226.
56. Abraham, H., C. Lee, S. Brady, C. Fitzgerald, B. Mehler,
B. Reimer, and J. F. Coughlin. Autonomous Vehicles,
Trust, and Driving Alternatives: A Survey of Consumer
Preferences. Massachusetts Institute of Technology
AgeLab White Paper 2016-6, Cambridge, MA, 2016. pp.
1–16.
57. Anania, E. C., S. Rice, N. W. Walters, M. Pierce, S. R.
Winter, and M. N. Milner. The Effects of Positive and
Negative Information on Consumers’ Willingness to Ride
in a Driverless Vehicle. Transport Policy, Vol. 72, 2018, pp.
218–224.
58. Reimer, B. Driver Assistance Systems and the Transition to
Automated Vehicles: A Path to Increase Older Adult
Safety and Mobility? Public Policy and Aging Report,
Vol. 24, No. 1, 2014, pp. 27–31.