1

HMI Design of Autonomous Driving Car

Group Challengers, Chalmers Ixd 2013 Eric Zhang, Er Luo, Annika Ridington, Zhe Liu

1. Project backgrounds
Autonomous driving is a new trend in vehicle industry. Human beings have been engaged in driving activities for a long time. Peoples driving habit as well as the transportation methods are also changing all the time. Whenever a new technology emerges, it will go through a time of exploratory research and experience summary, that ends up as a new standard. So it is important for the car industries to follow the technology innovation. Looking back to the history, seat belt is invented by English engineer George Cayley in early 19th century, though Edward J. Claghorn of New York was granted the first patent. American car manufacturers Nash (in 1949) and Ford (in 1955) offered seat belts as options, while Swedish Saab first introduced seat belts as standard in 1958. After the Saab GT 750 was introduced at the New York Motor Show in 1958 with safety belts fitted as standard, the practice became commonplace. However, the first modern three point seat belt (the so-called CIR-Griswold restraint) used in most consumer vehicles today was patented in 1955 (US Patent 2,710,649) by the Americans Roger W. Griswold and Hugh DeHaven, and developed to its modern form by Swedish inventor Nils Bohlin for Swedish manufacturer Volvowho introduced it in 1959 as standard equipment1. Since then, Volvo has been in the forefront of safety engineering, even though they are neither the first one who invented seat belt nor who introduced seat belt in car. Volvo succeed in finding a solution which fits human best. Thanks to their contribution, seat belt became part of the car, and saving a lot of lives. People got used to fastening seat belt, and it is required by law in most countries. As designers, who are changing the world, we should use our knowledge to find the best solution, and shape human behaviour. However, autonomous driving is completely new to us, although there is application in other area such as aviation. So there are many problem and challenges that we are facing, from the physical installation to software visualization, from recreation to safety issue. But we took on the challenge and named ourselves the Challengers with the ambition to bring out solutions for the new car, designed for humans. As this is a student project in cooperation with Chalmers University of Technology and Volvo Car Corporation, our work have a focus on following tasks according to Volvos requirement. - Inspiring, valuable and productive functionality that makes the AD periods valuable and enjoyable for the driver - Strategy and method for keeping the drivers ability to take over the control of the vehicle on short notice - Supporting drivers non-pronounced expectations and concerns for the capability and reliability of the AD system

2. Literature review
Before starting the project, we need to know what is HMI. According to Wikipedia, HMI (human-machine interaction) could be defined as the design of the interaction between human and machine2.The HMI vehicles could then be seen as the design that using the system inside the vehicle to aid the drivers in different aspects. Things such as analyzing data, providing entertainment or improving safety, comfort and so on from the inside system should be able to implemented during the driving process3. From the project provided by Volvo, this HMI used in the self-driving vehicles (Autonomous drive, AD) would even change its mission in the future so that the HMI applied in the vehicles would be more user-friendly and have more functionality. The HMI is not going to be just a guide of the driving. Instead, it could be anything that could bring any benefits to the users. Entertainment tool, guide map, warning system... These are all the functions that this HMI might have in order to please the drivers during the boring driving process3. In order to design for a good HMI system for this AD car, we tried to pick up some papers that could be used in this project. Some important topics then came out from these papers. The first important topic is about the peoples trust of the HMI or AD technology. According to Stephanie and Daniel4, they analyzed the trust from different aspects and did an X-ray test for their project in order to reach some conclusions about peoples trust to the AD machine. But for our project, the interesting point is that they found trust is a quite complex thing to value as it varies from time, it differs from individuals, it is effected by the user perceptions of machine characteristics and it also influenced by the actual characteristics of the machine and etc4. This point is quite meaningful in the design of the project because one task in this project is to Supporting drivers non-pronounced expectations and concerns for the capability and reliability of the AD system3. Trust can not be ignored if the system is going to be reliable for a driver. We also scanned the The role of trust in automation reliance from Mary T. D etc5. As this paper emphasized that human operators often underutilize (disuse) and overly rely on (misuse) automated aid, we think this is also a reasonable problem that could be solved in our project if we want to make our project more reliable. The second important part of this project is about the display of the information. We have checked some cars and found that normally, the display screens are somehow similar with each other. We checked the book Smart Things: Ubiquitous Computing User Experience Design from Mike K6. This book generally describes framework and methodologies for the design. But one issue he pointed out is that it takes a long time and huge amount of money for pushing products to get them adopted by users. Thus, if we take the practical issue into consideration and follow the Less is more: product complexity and product cost need to be kept as low as possible guidelines suggested by Volvo, we could not make very big changes of the cars basic system if we did not want to the risk of losing time and money in creating

very different HMI system for the car.

3. Methods
We used several different methods during this project. Some of these methods were just used in one part of the design process,while some others were used in two or more stages of the design processes. At the beginning of the requirement study, we used scenarios of use to model how the normal users would use of the different parts of this AD car. As Ian A. and Neil M. said in their book7, the scenarios could broaden the view of designers and they could also help the designers understand how the system work. We also found it was quite useful as with the help of scenarios, the general impression about how the AD car should work as well as where some potential problems could exist in the driving process became apparent to us. Thus we could consider these problems and developed the frame of the car in a consistent way throughout the process. Along with the scenarios of use method, existing system/competitor analysis was also used in the requirement study process. As we were going to develop a concept AD car that based on the current AD car, we thought analyzing the current AD car system is quite necessary for us. At least we needed to know the benefits and drawbacks that exists in these cars to figure out potential requirements that should be noticed in our later design. Role-playing was also an useful method to be used in the requirement process. One of our group member was trying to be an businessman when she was driving and thus came out with some requirements such as the need of doing work in the car, protecting his business work and making it quite safety so that nobody could steal it as well as even being able to have a meetings in the car. All these requirements were useful to us since we had decided that our target group would be business people with deep pockets that value their time. For business men all these activities would be quite normal to do in his daily life. Thus in the latter design process, all such things were taken into consideration and had great influence on our final design. Storyboarding and different kinds of sketching were used a lot during our design process. As we needed to transfer our ideas to each other, storyboarding and the sketching could help us understand each other much better than if we only expressed our ideas by words. For example, one conflict in our design process is about the non-destination drive in AD situation. This concept was firstly transferred in words by one of our group member. But at that time this concept was too conceptual to understand by others. However, when this member changed to use storyboarding and paper sketching to explain the concept for us, we realized what he meant and then even began to argue about whether it is necessary for the design. A mini heuristic or expert evaluation method was used in the middle evaluation process. Due to the time limitation and because the design is only in concept, we could not find people to try our real product. Instead, we presented our design concept to our professor

and another responsible people of this project in our middle presentation. It was a very quick presentation and we got some feedback from these persons. We thought we could see it as a heuristic or expert evaluation method as these two persons are professional in interaction design and AD cars. Whats more, they did give some important suggestions for us to consider in the latter of our design. Typically, brainstorming and open discussions were used through all the processes. As every group member was trying to come up with some ideas for this design and gave suggestions to other peoples ideas. We thought these two methods might be the best ones as they help a lot in improving and combining the ideas created by individuals during the whole of our design process. We also had a meeting in a car with ourselves as potential users to try to figure out how we would like things to work in a real life situation. By doing this we could simulate where we would like different things to be and came up with several important features to our concept.

4. Results
4.1 Requirement study
(whatwereadandgot,theexerciseswedidincourseandwhatwegot) In the requirement study, we formulated some basic functionality and requirements to achieve. First we decided on target group of users and the context of use. We decided our target group would be a businessman/woman with deep pockets that value their time and the context of use would be Sweden, with valid cultural aspects in conjunction. The project instruction mentioned that the most important thing for the drivers when they are driving these AD car is the safety2. Thus the accidents/ unexpected things happened. But how we should design this warning was then become a big problem. Sound warning and pop-up window on the screen could be useful in some cases, but they are still not so reliable after our discussion. We found if the driver is focusing on some special tasks such as listening to music and so on, it might be possible that the drivers could not hear the warning sound. Pop-up window might face the same problem if the driver is doing some other things that not based on the screens according to our discussion. Here we proposed to give the drivers some other notification besides visual and audio. Whatever tasks the driver is doing in a car, they keep sitting on the seat, so if we add some vibrating notification on the seat, the driver would never miss it. Moreover, even these warnings could be noticed by the drivers all the time, whether the drivers would trust this warning system was still unpredicted for us. As we found in the literature study, humans trust to the AD machine is quite complex and varies due to different reasons, it is too difficult to design a warning system that all the drivers would feel it reliable and confident enough to believe in this system.

According to the above reason, we concluded that we need to set up some basic principle for the warning system as shown below. These sound warning and pop-up window warning could be kept in our AD car, but they wont be the most valuable thing for the warning system in our design. 1. The drivers have full control of when they want to take over the car. The AD system can never turn into manual driving automatically. 2. There would always be some decisions such as slowing down, stopping ir pushing out airbag when the accidents happen in the AD driving process. The first principle means whether the car should be in AD model or MD model is decided by the driver rather than the system. We made this principle because drivers might not believe in the automatic system but they would believe in themselves, thus this principle might help to make them feel more reliable when they are driving the car. This decision also result in the comprehensive functionally of AD-car. Once the car is running in Ad mode, it should be able to run safely all the time to the destination and park itself. The second principle for this design is that there would always be some decisions such as slowing down, stopping and so on when the accidents occur in the AD driving process. This principle was set because we thought there should always be solutions to these accidents when the driver giving no reactions to the warning system.

4.2 Conceptualization
After the requirement study, we tried to implement some designs for both steering wheels and information provided by the screens. We decided to make some changes with the steering wheel to achieve the model changing between the AD system and MD system as steering wheel is the object would be most used and easiest to be fetched during driving process. In our design concept, the steering wheel could be pushed in and pulled out as figure 1 shows.

figure 1: Steering wheel design

The left picture shows the situation when the steering wheel is pulled out and the right picture shows how it

looks when it is pushed in.

When the wheel was pushed in, the system would be changed into AD model. And when the wheel was pulled out, the system would be changed to MD model. Besides the design for steering wheel, we also considered about the display of the information. During the first ideation process we wanted to have the normal three screens for the drivers according to the Less is more guideline. We wanted to put these screens just as they are placed now: one on the middle of the steering wheel, one behind the steering wheel and one on the right hand in the car. The information placed on them could be seen in figure 2. Typically, the head-up screen was given up in our design because we found it was ambiguous and could easily be confused to use during the driving process.

Manual Cluster Drivinginfo (speed,RPM,autodrivingindicator) HeadupGPSinfoandnavigation indicator(forrouteplanning)

Auto Autodrivinginfo (speed,time,distance) Navigation (zoomableandroute changeable) Carcontrol+iPad (entertainment,business)

Steering Wheel

CenterSta

Carcontrol (airconditioning,radio,light)

figure 2: Steering wheel design

The information should be shown on the different screens under the two different models in middle-term design.

4.3 Evaluation and Refinement

After we have formulated our first concepts, we had a chance to talk to Claes Edgren, vehicle HMI expert at Volvo Car Group. We presented him our concept, and got some valuable feedback. Our idea of the pull-push steering wheel was very much appreciated. Furthermore, the principles we proposed were also acceptable. After the mid-presentation, the design of this display was changed. As Claes mentioned that drivers in elderly age might have difficulties in looking at the steering wheel screen because of the hyperopia problem, it was then not so good to display the GPS information on the steering wheel for the drivers. Claes then suggested us to use the Cluster screen to show the information as he thought this distance is a perfect distance for the people like him to read the information display. We carried out a meeting in Annikas car, to verify the experts concern, and this is also a good time for us to carry out a self-evaluation on our ideas. New ideas pop up much

efficiently when we are in the context of user we are designing for. In the car we did a hypothetical simulation of being in a fully working AD-car trying out distances and placement of our design suggestions. We realized that comfort is important and since driving long distances, sitting in the same position can be really painful, we decided that having leg space and being able to change the position of the car seat would be useful in our concept. When we went through the scenario with The businessmans rushed morning routine we also realized that he needed a place to put his breakfast and notes, while he was having a video meeting with clients. Trying out to reach for the center stack made us realize that it would be more comfortable if we could maneuver the center stack to be closer to us, without cluttering the space in front of the steering wheel which could be dangerous. We found a lot of detailed things as mentioned above was ignored when we were not in the car. Thus we tried to mix all the detailed things together with our previous design in order to make our project more user-friendly. Things such as making the car possible for video meeting through the screens, moving the hand-brake in order to offer more space for drivers and so on were carried out at that time.

4.4 Design and Prototype

We finalized our ideas and make some prototypes to visualize some of them, in order to illustrate our idea more clearly in the industrial presentation. (1)Generalview

This is a general glimpse of our cockpit design. Regarding the human machine interface, we provides two output interfaces (cluster screen and centerstack screen) and three major input interfaces (steering wheel touchpad and control, cneterstack touch control, and drawer keyboard). There placement are quite similar to current vehicles, while they work

they work is going to change a lot as described in below. (2) Cluster Screen Cluster Screen works differently in manual mode and auto-driving mode. It keeps similar to current vehicles in manual driving mode, in order to avoid resentment for drivers. However, to achieve a minimalist design, we kept the speedometer but removed the RPM. Note that we also put a navigation preview in middle of the speedometer. On one hand it shows the current location of the car, on the other hand it also shows the prospective route if changed to auto-driving.

Here we have a new concept of Smart driving. We assumed the user should set a destination before starting the car. Then the autonomous driving system could know where to go. When user is setting the route, not only the destination, but also the route can be edited in advance. Even though, the driver is still able to change the route on the journey, and the AD-system would reload a new route dynamically. But what if the driver didnt set any destination before he start. Should the car allow him to switch to Auto-driving? Our answer is an YES. Smart driving mode allows the driver to enter auto-driving mode, and then the driver can set the destination or let the car go straight and take over later.

Then there are two modes of cluster screen in Auto-driving. They are working mode and

navigation mode. Working mode uses the cluster screen as a normal display, can play video, play game, work on video meeting, etc. This is because it is located in front of the driver, so turns out to be most comfortable display. Actually same reason as current vehicles putting cluster screen there. Driver dont need to turn around for long time when watching something. So the cluster screen has higher priority in heavy visual tasks (tasks requires driver to watch for long time or high attention).

In the navigation mode, it shows the gps information and driving plans. It is similar to current navigation systems. It has extensions such as a route manager, which saves drivers frequent route or frequent destinations, either saved by driver or retrieved from driving history via machine learning technology. One more thing we need to mention is that switching between two can be controlled from some button on the steering wheel. The the switching only swap the main display and side display. The driving information such as speed is always on the other side. (3) Centerstack We have a customisable centerstack. It has an arm which supports movement in horizontal along the the axis (green line shown in image below). Additionally, it can all rotate around the vertie axis (range from green arc in image below). It can also be flipped into a horizontal screen. This customization allows it turn to any passenger in the car without increasing any safety issue. Moreover, this is physically feasible. Also if using it to watch a movie or play a game you can increase the screen area by flipping the centerstack to a horizontal position.

10

The default content on this centerstack is the car control. Such as radio, air-conditioning, shift, phone and so on. This default mode is also the only mode in manual driving. However, when it turns to auto-driving, the centerstack becomes much more powerful. It can almost work as a tablet. (4) Drawer board As auto-driving makes it possible to do much more tasks in the car, so such a board with two faces can make many task easier. It is a drawer board locates under the steering wheel. The drawer board is only available in auto-driving, because it will block the steering wheel in manual driving. It can turn between to sides.

By providing the user with both a digital and a physical keyboard we are giving him the option to work in a manner that is most convenient for him at the time. Sometimes it is easier to just take some notes with a digital keyboard, like the ones that are used in todays

11

smartphones. If you are writing longer documents or replies to emails it might be faster and more efficient to use a traditional or external keyboard. By implementing it in our already existing tray, we can ease its use, by not having the user have to bring an external one, and also incorporate it into a sleek and logical design choice. The keyboard is of course on essential part of turning your car into an ambulating office, in conjunction with having a videoconference with clients or colleagues on the screen.

Office car- concept Turning the car into an office, by having a business meeting while in AD mode. The digital cluster is the screen and the center stack serves as a note pad.

Or using the digital cluster as a computer to read emails and the center stack to reply to them.

12

(5) The Spacious seat Auto-driving releases driver from dedicated in driving tasks. If we have a spacious cockpit such like shown in image below. The driver can stay more comfortably.

In order to achieve this, we removed everything between the driver and co-driver seats, and turns it into a long coach. As to the things removed. We put the hand brake into left side of the driver, like the handle shown in image above. We also changed the shift into digital control in the centerstack. On the steering wheel, there is also two pressure-sensitive buttons for the gas and the brake, they are for same use as the gas and brake pedal, in aim of giving driver quick control. They can easily work like of motorcycles. This may help transforming the driving habit from stepping the pedal into using these buttons. (6)Emergency protection We also considered some different kind of emergencies situations that could emerge while driving. The first obvious one was What happends if the car crashes?. We decided that a car with this level of sophisticated technical system should be able to register values regarding the drivers health. Potentially by sensors in the steering wheel, eye movement tracking or other solutions. In case of a serious accident the car would contact the emergency number (in Sweden since that is our context of use) 112, and report these values and the position of the car. This would make the responding time to an accident potentially much shorter, and the first emergency crew on the scene would have some information of the condition of the driver that could help establishing a diagnosis faster. The emergency crew could even potentially activate the web camera in the car for a fuller understanding of what is going on at the scene and with the driver before they arrive to the scene. Then we realized that the driver could get acutely ill while driving and what would happen then? With a AD car with a sophisticated navigation system the solution to integrate an emergency button that when pressed would override everything else, and the car would drive itself to the nearest hospital seemed feasible and logical.

13

Screens from our presentation of our emergency and data protection concepts (7)Data protection Since we have decided that our target group is the business professional and established that he will do a lot of work in the car, thus making it into an ambulancing office, we realized that it would be important to protect all data from outsiders. In case of the car being stolen what would happen? Business secrets might be stolen or copied. To prevent this we established that all data would be protected by NOT being stored in the physical car at all, but kept in the cloud with an appropriate encryption process connected to sending and retrieving that information back and forth.

5. Discussion
It was important for us to keep the design close to the guidelines of using existing technology, not changing to much of the essentials in the cockpit, keeping costs down and not getting to futuristic in our concept. At the same time we wanted to try to think new thoughts and consider out of the box concepts. In this project, we think we created a concept of a HMI system that solved the trust and safety problem existing in the current AD car with the steering wheel design, and we also made our design user-friendly for the businessman with more driving space in the car and working and entertainment functions inserted in the system. However, there are still some problems that exist in our design. As mentioned before, what we have designed is only a concept. Thus it is really hard to test and evaluate the final design. All that we could do was to explain our design concept to the potential users and then to get feedback from them. Whether this design is implementable or practical for users in their daily life still remain uncertain. We are also not sure what problems would appear when this kind of car is put into use. Technique and implement testing would have to be performed and iterated before this design would be going into the open market. Whats more, even though we had designed many objects in our design, they are still not detailed enough due to the time limitation. Things such as colors for different objects and size of these objects were not designed in our project, more effort could be put into these details in the future.

14

Typically, we used the methods introduced in this course in a lot of our design. These methods went together with us through all the design processes. The issues addressed during the discussion part of the lectures encouraged and inspired us to consider aspects we might not have thought of, and therefore made our design concept richer in the end.

6. Conclusion: sum up the project

This design of this project mainly focused on how to create a trust worthy, user-friendly and functional HMI AD car for the professional businessman. As mentioned above, the trust is a very complex and unpredicted issue for people thus we decided to let the users themselves decide when to let the car drive itself and when to take over control and drive it. Some other functions that might be used by businessmen such as watching news, having meetings were also applied in our design. Moreover, as we want our design to be user-friendly, we also physically changed the drivers seat and sitting space for the driver so that they would have more freedom to move around and room for their legs. We also thought of other aspects like safety concerns , for both people and data, that might not be the first ones that come to your mind, like What would happen if the car crashed , What would happen if the driver has a heart attack or get acutely ill or What would happen if the car was stolen with all this office work going on inside? Jonas Landegrens lecture to think the unthinkable helped us when considering these aspects. This project also helped us in applying some useful methods in our course. Methods such as brainstorming, literature review, role-playing and scenarios and so on were used during the design process. And we found these methods were quite useful in coming up with different ideas and opinions in this project. As always, there are both advantages and drawbacks in our design. As this design was not depended on the users experience and was not tested by anyone, nobody could make sure there wont be any unexpected things happen during its implementation. Thus how to implement our design and make it more practical might be one big issue in its future work.

15

Reference
1.Wikipedia,2013.10.23.,Seatbelt, http://en.wikipedia.org/wiki/Seat_belt 2.Wikipedia,2013.10.16.,Humancomputerinteraction, http://en.wikipedia.org/wiki/Humancomputer_interaction 3.Volvo,ProjectTask:HMIforAutonomousDriving, http://ixdcth.se/courses/2013/tda486/sites/default/files/files/741132201%20PM%20Project %20Task%20CTH%20Human%20Centered%20Design.pdf 4.StephanieM.M.,DanielR.I.,April.2008,HumanFactors,"NotAllTrustIsCreatedEqual: DispositionalandHistoryBasedTrustinHumanAutomationInteractions",p.207, https://www.dropbox.com/sh/7lfb2l8qlhk2001/kSSE4Rhr2n/project%20work/Not%20All%20Tr ust%20Is%20Created%20Equal%20Dispositional%20and%20HistoryBased%20Trust%20in %20HumanAutomation%20Interactions.pdf 5.MaryT.D.,ScottA.P.,ReginaA.P.,LindaG.P.,HallP.B.,June2003,International JournalofHumanComputerStudiesTrustandTechnology,V.58,I.6,pp.697718,The roleoftrustinautomationreliance, http://www.sciencedirect.com/science/article/pii/S1071581903000387 6.Kuniavsky,M.,2010,SmartThings:UbiquitousComputingUserExperienceDesign 7.IanA.,NeilM.,April2005,Scenarios,Stories,UseCases:ThroughtheSystems DevelopmentLifeCycle,PREFACE,p.xiv