2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

Self-driving cars need to give

passengers a better experience. Here’s
how.
The adoption of new technology hinges on user experience. Self-driving
cars are no different. Below, Ericsson tech hero Alvin Jude explores how
edge services will be critical to improving the self-driving car passenger
experience, for example by collecting nearby data and visualizing in 3D.
Read on to learn more.

JAN 13, 2020

|
 5 min.

Alvin Jude
Senior Researcher, Human-Computer Interaction

CATEGORY

Connected vehicles Edge computing Research

HASHTAGS

#Autonomouscars #Selfdrivingcars #EricssonResearch

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 1/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

“He doesn’t even trust my driving, do you think he’ll trust a machine?” My friend was half-
joking, but his father’s long ‘nooooooooo’ when we mentioned self-driving cars gave us a clue.
I realized one crucial barrier to the uptake of connected vehicles: most people are anxious to
enter one to begin with.

My friend’s dad isn’t alone in his sentiment. In a 2018 survey, 52 percent of Americans claimed
they would never want to use a driverless car, and only 9 percent said they would use one as
soon as possible. Older users are also far less willing to use one. This does not seem right. As I
grow old, I want to benefit from autonomous vehicles, especially if I lose my mobility or ability
to drive. I want self-driving taxis to take me to parks, connect me with my family, or take me to
medical appointments.

So, why the hesitation? Why the long, emphatic ‘nooooooo’s?

I recall my mum teaching me how to drive. I remember her sitting in the front seat and
slamming on an invisible brake pedal. I recall my passengers grabbing onto roof handles
whenever they sensed the slightest of danger. I can’t really complain. I always feel a bit of
anxiety when I’m in a taxi, or when I’m driven around by somebody whose driving styles don’t
match mine. Most of all, I remember the many people who would tell me to ‘watch that bike,’
‘pay attention to that pedestrian,’ and ‘look out for that red van.’ In most cases, I would have
already noticed the hazard, but  I still needed to say “yup, I see her” as calmly as possible with
minimal annoyance.

Now, how could we do that with a computerised driver?

Improving the self-driving car experience

Our philosophy for creating good software is to build apps that people want to use, not ones
that people have to use. This philosophy can also be applied to self-driving cars. How do we

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 2/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

create self-driving cars that people want to get into, regardless of whether or not they need to?
That’s easy. We simply need to design them with the passenger experience in mind.

In his book ‘The Design of Everyday Things’, Don Norman laid down a number of core UX
design principles. One is the ability to to provide feedback. Let’s think about this in the context
of self-driving cars: the car needs to give the passenger feedback on what actions it will take.
It should also explain why.

The motivation here is simple. The developer in me who understands what goes on under the
hood knows that the onboard computer has seen all the hazards, is calculating the best course
of action in real-time, has my safety and comfort optimised, and is constantly working to
ensure the safety of everyone around me. The passenger in me still wants to tell it to “look out
for that red van.”

Constant feedback is the key to ensuring passenger experience, specifically one that that
alleviates anxiety. It is the car telling the passengers ‘yes, I see that red van.’ So, how do we do
that? Voice is an option of course, but that would quickly become annoying. Visualization, on
the other hand, would work superbly.

Since the vehicle is already collecting a tremendous amount of sensor information, it should be
possible to reconstruct this data. The key difference is that the sensors use machine vision and
the data is intended for machine consumption, not human consumption. Through a series of
design exercises, we identified the gaps between visions intended for machines against those
intended for humans, backed by theory on human factors, our knowledge of sensors, and
observation of current trends in intelligent vehicles.

Visualization of vehicle intent

Our hypothesis was that continuous, instant, on-demand communication between the vehicle
and the passenger would alleviate anxiety in passengers, improving the passenger experience.
We intended to provide this communication through visualisation only, allowing the
passengers to glance at the screen any time they wanted. Voice and/or audio could be used
but only when absolutely necessary, thereby reducing sensory overload which would only
heighten anxiety.

The first thing to note is that people tend to reconstruct objects in their head. Take this cup for
example:

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 3/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

Figure 1: Can you imagine what this cup looks like from the back?

You’ve probably never seen this exact cup before, and you can only see it from one angle. But
you instantly know what it looks like from the back. Most people are also able to do mental
rotation of objects despite only seeing it from one angle or through photographs. Our ability to
fill in the blanks is why we see two large triangles here.

Figure 2: How many triangles do you see in this picture?

Did you notice that there are in fact no triangles at all? We fill in the blanks and draw

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 4/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

connections, which allows us to imagine the back of the cup as well as see triangles when they
don’t exist (if you’re interested in this, look up Gestalt Psychology). Machines don’t typically do
that – because they don’t have to – but they can be trained to reconstruct images the same
way people would.

Egocentric vs. allocentric

That brings us to the next part of our design: visualization. Simply put, there are two ways to
view the world: egocentric or allocentric.

Egocentric is basically a ‘first person’ view, while allocentric is a ‘third person’ view. Allocentric
views seem to be better in this case for a few reasons: it provides a wider view of the world, it
allows a better understanding of the world, and it can show items occluded from the
passenger, but not from the car (due to the position of sensors on the vehicle).

Figure 3: Egocentric view - here’s what passenger would see from the back seat.

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 5/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

Figure 4: Allocentric view of the same scene. Notice the oncoming vehicle which was previously occluded.

Now all we need to do is take this allocentric reconstructed view of the world, and place it
within view of the user.

Figure 5: Now the passenger has an allocentric, reconstructed visualization of the world.

This is just the first step. There are many more questions that we need to ask. For example, do
we need to provide photorealistic visualizations? Or would it suffice to show approximations,

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 6/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

such as an icon to represent other cars? Do passengers need to know that the car ahead is a
green 1995 Toyota Corolla? Or is it sufficient that they see it’s a green Sedan? Perhaps if
everything were monochromatic then it would be easier to inform them of more pertinent
information, such as the car’s planned path, and the hazards it sees. There is a balance to be
established, and there are many more research questions and design choices that we look
forward to investigating.

How we can overcome occlusion

An important question we ask ourselves at Ericsson is how can we further improve this
experience? For this, we need to fully understand the pain I face every morning as I drive out
of my house. Every morning there will be a car parked close to my exit, blocking my onboard
sensors from the moving vehicles behind it. 

Figure 6: All the onboard sensors will not be able to capture that moving vehicle.

Every morning, I inch slowly forward until I’m certain I won’t be t-boned. Sometimes
oncoming vehicles will see my bumper and provide me with the first (and usually only)
annoyed honk of the day. This problem could be solved by putting mirrors on the road, but that
won’t really help improve with the passenger experience of self-driving cars.

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 7/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

Figure 7: Road sensors could signal oncoming vehicles.

If we can tap into sensors, then we could fuse all this information together and show the
passengers in self-driving cars the full picture. Vehicles and hazards that are occluded from
the vehicle could now be…un-occluded. This is my vision of the world: one that is completely
free of occlusion.

The journey ahead

There is, of course, a lot more work to be done here. Reconstructing the world in real-time is no
simple task. Low-latency, high bandwidth connectivity between environment sensors and
vehicles is essential. An edge cloud which collects data from multiple heterogeneous sensors,
consolidates it into one coherent view of the world, then sends the relevant parts to those who
want it is a key ingredient in improving the passenger experience of autonomous vehicles.

I have no doubt that an unocclusionator will be available by the time I am too old to drive. I’m
working hard to make sure my friend’s dad will also benefit from it.

Read more
In a recent Ericsson Technology Review, we explore the extent to which 5G could enable
emerging road transport and automotive applications.

Visit our connected vehicles page to learn more about our ongoing partnerships and use cases
in this area.

RELATED CONTENT
https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 8/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

FEB 18, 2021

| Wolfgang John, Per Persson, Azimeh Sefidcon

Why the world needs a pervasive Network Compute Fabric

Edge computing, Research

FEB 05, 2021

| Per Fröjdh

MPEG file format honored with prestigious Emmy® Award

Research

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 9/10
2022-01-19 10:07 How to improve the self-driving car passenger experience - Ericsson

FEB 01, 2021

| Kun Wang, Joachim Sachs, Janos Farkas

How 5G integrates with TSN-based industrial communication

systems
Industry 4.0, Private networks, Research

THE ERICSSON BLOG

Like what you’re reading? Please sign up for email updates on your favorite topics.

Subscribe now

At the Ericsson Blog, we provide insight to make complex ideas on technology, innovation and
business simple.

https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience 10/10