Professional Documents
Culture Documents
net/publication/336613232
CITATIONS READS
0 846
1 author:
Liza Dixon
Hochschule Rhein-Waal
2 PUBLICATIONS 0 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Liza Dixon on 18 October 2019.
Liza Dixon
Kamp-Lintfort, Germany
email: lizadixon@gmail.com
Abstract—The presence of automation is growing in the vehicle automation, a great irony exists in the way that the
domains of our homes, workplaces and roadways. Vehicle technology is being promoted.
automation in particular is raising critical human factors issues Unfortunately, like the greenwashing of the sustainability
which directly impact human-machine interaction and road movement [4], the capabilities of vehicle automation are
safety. It is especially important that users of partial and semi- commonly inflated. A corporation—eager to profit in the
autonomous systems in safety-critical contexts understand the short-term—might exaggerate the capabilities of a product’s
limitations of the technology, in order to ensure appropriate automation in marketing verbiage. This is a substantial risk,
reliance. Studies indicate that the language used to describe
as users’ ideas about automated systems are formulated long
vehicle automation in marketing and in the media effect user
before their first contact with a system, and these ideas
perceptions of the system’s capabilities, and later their
interaction with the system. Much like “greenwashing”, the influence how they later interact with the system [5]. A user
capabilities of automation are often overstated. The lack of that believes a system is more capable i.e. more autonomous
public awareness of this issue is one of the most critical problems than it really is, is more likely to overtrust and misuse the
impacting trust calibration and the safe use of vehicle system [6], increasing their risk for an accident. News reports
automation. Yet, it has gone unnamed and continues to affect of human fatality and severe injury in association with
the public understanding of the technology. Hence, the case for partially automated driving systems condensate a dark cloud
the use of the term “autonowashing” to describe the gap in the over the technology, increasing customer wariness about its
presentation of automation and the actual system capabilities is reliability. In the long-term, this may hinder acceptance and
put forth. This paper presents case studies and discusses key “…can reduce or even nullify the economic or other benefits
issues in autonowashing, a term/concept that influences or that automation can provide” [7].
relates to public perceptions of vehicle automation. Therefore, using transparent language in the marketing
and promotion vehicle automation which clearly describes its
Keywords—Autonowashing; Trust; Automation; Human- abilities and the limitations is critical to safe use, acceptance,
Machine Interaction; Autonomous Vehicles. and scaled adoption of vehicle automation [8].
In this paper, the concept/term of autonowashing is
I. INTRODUCTION proposed to describe the gap between the way automation
capabilities are described to users and the system’s actual
The promises of automated technologies — from robotic technical capabilities. The key concepts of greenwashing, the
assistants to self-driving cars — to improve our safety and current state of vehicle autonomy, and trust in automation are
quality of life are seemingly boundless. Fantasies of the presented. In order to better understand what autonowashing
future, in which road deaths are virtually eradicated and looks like in practice, media headlines and automakers’
mundane tasks are removed from human concern, freeing us approach marketing vehicle automation are examined as case
to enjoy greater independence and personal autonomy, are not studies. The five signs of autonowashing, the consequences of
guaranteed. To make these visions a reality, automated autonowashing and what might be done to alleviate its effects
systems must not only be functional, reliable, and trustworthy; are also presented. The following questions are addressed:
they must be mindfully introduced to the humans they intend
to support. Q1: What is autonowashing and why does it occur?
However, terms like “automation”, “intelligent systems”, Q2: When, where, and how does autonowashing occur?
and “artificial intelligence” are used loosely to describe Q3: What are the effects of autonowashing and how might
everything from a smartphone’s autocorrect feature to a they be mitigated?
safety-critical application such as an Advanced Driver
Assistance System (ADAS). As marketing buzzwords, their A. Greenwashing
meaning is becoming increasingly diluted. The Over the past few decades, the need for more sustainable
misinterpretation of AI and automation in the media is lifestyle choices and business practices has been brought into
confusing the public and is considered to be an economic and public awareness. With this awareness, so came the
humanitarian issue [1]. Economic, because the corporations opportunity for businesses to adopt a “socially responsible”
investing billions of dollars into the development of image, aligned with the concerns of their customers, hence,
automated systems [2] are counting on the return on corporate social responsibility (CSR) was born. CSR is “the
investment, and humanitarian because the misuse of idea that a company should be interested in willing to help
automated systems can be deadly [3]. Yet, in the case of society and the environment as well as be concerned about the
products and profits it makes” [9]. CSR has proven to be so
effective, that claims of social responsibility have become 71% of those surveyed believed it was possible to purchase a
increasingly stretched and exaggerated. Corporations—freely “self-driving car” today [14].
using “green” terms and labels as needed—have elegantly Issues surrounding the language used to describe vehicle
mislead customers into thinking they were purchasing more automation are noted in scientific literature [8] and in the
environmentally friendly goods than they actually were; media [15]. A study by Beller, Heesen, & Vollrath [16]
essentially, preying upon the uniformed. This practice of confirms that a user with a false understanding of system
exaggerating the “naturalness” or “eco-friendliness” of infallibility, “can lead to severe consequences in the case of
products and services grew so widespread, was given a name: automation failure.”
greenwashing.
Greenwashing was coined by environmentalist Jay C. Trust in Automation
Westerveld in 1986 [10] and is defined as “the practice of When automation (autopilot) was first introduced to the
making an unsubstantiated or misleading claim about the aviation industry, it helped pilots evade many common
environmental benefits of a product, service, technology or accident scenarios; simultaneously, new kinds of accidents
company practice” [4]. Greenwashing illuminates the emerged. It was not until pilot education about the automation
disconnect between a marketed image of corporate social and "the concept of the human-automation team” was
responsibility (CSR) and the reality of a corporation, product introduced, that the benefits of the autopilot system were fully
or service’s contribution to the sustainability movement. Over realized, supporting the aviation industry with the extremely
the years, greenwashing has become more sophisticated; low accident rates of today [17]. In truth, “…one cannot
spawning the development of frameworks for identifying remove human error from the system simply by removing the
greenwashing in practice (see the “Six Sins of Greenwashing” human operator” [7].
[11]). A key component in the acceptance of automation is an
attitude of trust in the system [6]. Trust in automation (in the
B. State of Commercial Vehicle Automation & Marketing context of automated driving systems) is defined by Körber
Presently, the highest level of vehicle automation in as, “The attitude of a user to be willing to be vulnerable to the
production vehicles on-road today is Level 2 automation [12]. actions of an automated system based on the expectation that
The levels of vehicle automation, as defined by the Society of it will perform a particular action important to the user,
Automotive Engineers (SAE), extend from Level 0 or “no irrespective of the ability to monitor or to intervene” [18].
automation” to Level 5 or “full automation” [13]. Level 1 Trust is not only critical for acceptance, but also for safety.
automation describes traditional cruise control systems (speed Both an overtrust in automation as well as a distrust in
maintenance) and Level 2 describes “partial automation” or automation create problems in human-automation interaction.
Adaptive Cruise Control Systems (ACC) in which the vehicle The goal then, is calibrated trust [19], or a level of user trust
is able to maintain speed, accelerate, decelerate and in some in the system which matches the automation capabilities of the
cases light steering for lane maintenance. However, Level 2 system in use [6] (see Figure 1).
systems require full driver supervision at all times. Level 3 Trust in partially automated vehicles begins “long before
systems are described as “semi-autonomous” or “conditional a driver’s first experience with the system, and continues long
automation”; the driver is not required to supervise the system thereafter” [5]. Multiple studies have confirmed that the
in specific scenarios (e.g. in a traffic jam) but may be called branded terms (e.g. Tesla’s “Autopilot”, Audi’s “Traffic Jam
upon by the system to take back control of the vehicle should Pilot”) used to describe vehicle technology influence
conditions change. Level 4 and Level 5 automation (“full perceptions of the technology and “name alone is not enough
automation”, “self-driving”, “autonomous”) are similar, with to appropriately orient drivers to system limitations” [8].
the exception that Level 4 automation is geofenced i.e. limited Stories about automation in the media, in advertising, and
to specific geographic areas, while Level 5 automation is those heard by word of mouth have an effect on trust [5], [8].
theoretically fully autonomous, requiring no human Further, unrealistic expectations of assistive and self-driving
supervision or presence and can operate in all conditions. technologies could be a barrier to acceptance [20]. Initial
According to a recent study, “automated driving hype is acceptance may be increased if the driver’s expectations of the
dangerously confusing customers”, and further, “Some system are unrealistically high. However, after practical
carmakers are designing and marketing vehicles in such a way experience with the automation which reveals its
that drivers believe they can relinquish control” [14]. Because shortcomings, “trust and acceptance may be irreparably
there is no regulating body overseeing the language used to harmed” [20]. Therefore, supporting drivers with realistic
describe assistive systems, automakers, also known as original expectations regarding the capabilities of automation is
equipment manufacturers (OEMs) have been unchecked in important for acceptance in the long-term [21].
their use of branded terms.
OEMs offering driver assistance systems as options in II. AUTONOWASHING
their vehicles (i.e. Audi, Ford, Tesla), use a wide vocabulary Adapted for automation [4], autonowashing is defined as
to describe these options and their abilities. The motivations the practice of making unverified or misleading claims which
for this are clear: “Carmakers want to gain competitive edge misrepresent the appropriate level of human supervision
by referring to ‘self-driving’ or ‘semi-autonomous’ capability required by a partially or semi-autonomous product, service,
in their marketing…” [14]. As a result, a recent survey of or technology. Autonowashing may also be extended to fully
1,567 car owners across seven different countries found that autonomous systems, in cases where system capabilities are
exaggerated beyond what can be performed reliably. [15], [22]–[24] is discussed in numerous studies [8], [17], [20]
Autonowashing makes something appear to be more and the subject of multiple lawsuits [25]–[27].
autonomous than it really is.
The objective of autonowashing is to differentiate and/or B. Case Study: Headlines
offer a competitive advantage to an entity, through the use of Over the past decade terms such as “autonomous”,
superficial verbiage meant to convey a level of system “driverless”, and “self-driving” have made increasing
competence that is misaligned with the technical appearances in media headlines. These buzzwords are often
specifications of the system. Autonowashing may also occur used by outlets and publications to describe all levels of
inadvertently, when one unknowingly repeats erroneous vehicle automation, baiting interest, sales and “driving traffic”
information about the capabilities of an automated system to to their respective sites. It is not uncommon to come across an
another. Autonowashing is a form of disinformation; it is, in a article discussing Level 2 automation as “autonomous” or a
sense, viral. testing vehicle as “driverless”, even though there is a human
safety driver monitoring the vehicle and the environment at all
A. Effects of Autonowashing times (see Table 1).
The results of autonowashing are, but not limited to:
misuse of a system due to inappropriate reliance, leading to TABLE I. AUTONOWASHED HEADLINES & CONTRADICTIONS
disuse of a system due to performance concerns. Headline Contradiction
Autonowashing does not support calibrated trust in “Joshua Brown, Who Died in Self- Not “self-driving”—this Tesla had
automation and increases the likelihood that a user will Driving Accident, Tested Limits Autopilot (Level 2, ADAS) which
overtrust a system (see Figure 1) [6]. of his Tesla” – The New York requires full driver supervison
Those who have been autonowashed believe that an Times [28] [29].
“Volvo puts 100 British families in Not “driverless”—all vehicles
automated system is more capable than it really is, and hence driverless cars” – Financial Times have professional safety drivers,
may be confused about how much supervision the system monitoring the the driving
requires from a driver. They may refer to an ADAS as “self- environment, ready to intervene
driving” or “autonomous” and be more inclined to engage in [30].
risky misuse, such as removing their hands from the wheel or “Tesla Has Begun Making All Its Not “self-driving”—Tesla
looking away from the road ahead, increasing the risk of New Cars Self-Driving” – NPR reportedly upgraded the hardware
[31] systems in their vehicles, which
accident. (according to Tesla) could one day
make the vehicles “fully self-
driving” [32].
“Elon Musk Defends Tesla Not “self-driving”—Tesla has
Following Latest Self-Driving Autopilot (Level 2, ADAS) which
Accident” – Adweek [33] requires full driver supervison
[29].
“Fully driverless cars are on Not “fully driverless”—modified
public roads in Texas: [subhead] vehicle with professional safety
Drive.ai is the second company to driver in passenger seat,
remove the safety driver from its monitoring the driving
autonomous vehicles” – The Verge environment, ready to intervene
[34].
“Shocking moment a Tesla driver Not a “self-driving” vehicle—this
is filmed ASLEEP behind the Tesla has Autopilot (Level 2,
wheel as his self-driving car ADAS) which is being
travels at high speeds on the “successfully” (without incident)
California interstate” misused as it requires full driver
supervison [35].
Table 1. Examples of autonowashed headlines; verbiage
which gives the reader the impression that the vehicle
being described is more autonomous than it really is. The
Figure 1. Illustration of the relationship between trust, technical contradiction to the headline is presented.
automation capability, overtrust, distrust, calibrated trust
and autonowashing [adapted from 6]. Autonowashing
affects trust, resulting in a tendency to overtrust, increasing C. Case Study: Tesla Autopilot & Full Self-Driving
the risk of system misuse. A recoverable or “safe” margin of In 2014, Tesla introduced its first iteration of a Level 2,
error (light gray) in trust calibration is to be expected in Advanced Driver Assistance System called “Autopilot” [36].
use. However, autonowashing may push the user beyond The term autopilot as defined by the Cambridge English
this margin, into a situation where an accident is more Dictionary [37] is, “a device that keeps aircraft, spacecraft,
likely. and ships moving in a particular direction without human
involvement.” The choice to name an ADAS “Autopilot”,
Although the term is not directly mentioned, the concept which requires constant human supervision has been criticized
of autonowashing has been receiving media attention [12], by experts, several organizations and government entities. In
a letter to Tesla, the German government (Federal Motor
Transport Authority) wrote: “In order to prevent Tesla’s CEO Elon Musk has promoted “Full Self-Driving
misunderstanding and incorrect customers’ expectations, we Capability” on his personal Twitter account, in one case
demand that the misleading term Autopilot is no longer used stating “Tesla drives itself (no human input at all) thru urban
in advertising the system.” [22]. streets to highway to streets, then finds a parking spot”
Since its initial release, Autopilot has received hardware without clarifying that this feature is not yet enabled [43].
updates in newer vehicle models and continues to receive Further, Musk has been seen in multiple TV interviews [23],
software updates over the air [29]. These updates have [44], [45], removing his hands from the wheel with Autopilot
improved the reliability of the system under certain conditions active. In one of these examples, he did so and stated, “See?
but have not yet made the system “more autonomous”— It’s on full Autopilot right now. No hands, no feet, nothing,”
Autopilot remains a Level 2 system as the same level of driver as he demonstrates the system to the interviewer, who is
attention is required to operate the system safely. It is sitting in the passenger seat (Figure 3) [45]. This behavior is
explicitly stated on the Tesla website and in the vehicle at odds with appropriate use, and is explicitly warned against
owner’s manual in multiple instances that the driver must keep in the Tesla Owner’s Manual [29].
their hands on the wheel and their attention on the road ahead
[29], [36]. Despite these statements, Tesla is the only OEM
currently marketing Level 2, ADAS equipped vehicles as
“self-driving” [38].
In October 2016, Tesla announced that “all Tesla vehicles
produced in our factory…will have the hardware needed for
full self-driving capability at a safety level substantially
greater than that of a human driver” [32] (see Figure 2). This
announcement also came with the sale of a new Autopilot
option called “Full Self-Driving Capability” (FSD). Tesla
stated that customers who purchased the FSD upgrade would
not experience any new features initially but that in the future,
this upgrade would enable the vehicle to be “fully self-
driving” [39]. This option was later removed, but then
subsequently reintroduced for sale in February of 2019.