CGT-172-08|UX Studio I

Professor Colin | TA Nicole

February 16th, 2022 | Spring 2022

Project 2: Rate your Lyft

By Team17 - EnergyLyft
Jinwoo Lim, Zheng He, Amanda Huang
Assignment Briefing
‘This current rating system tacitly encourages users to only rank their driver using very
good or very bad ranking. You must consider the ways various rating and review systems
constrain the input of users, the ease of use (e.g., speed of input in a busy urban environment,
evolving pandemic policies), and the ways in which this information could be used maliciously
against drivers, while also building on the evolution of rating approaches in the gig economy at
large.’

Define the Problem

The limitation of the five-star scale. Find a solution to fix the 5-star review system.

Secondary Research
“The Problems with 5-Star Rating Systems, and How to Fix Them” by Harvard Business
Review’s Josh Breinlinger, Andrei Hagiu, and Julian Wright goes in-depth about the limitations
of the 5-star review system. They admit that it is a simple, yet effective system that differentiates
between the good and the bad products/services. However, it has a problem distinguishing the
great from the good. Users too often rate mediocre products as ‘great’, which heavily inflates the
ratings. A simple solution proposed by the writers is to provide the users with a median scale of
all suppliers to get an overall sense of the distribution. A problem was users that often give too
much of a negative/positive review. Here, the writers suggest adjusting these reviewer’s ratings
to not distort the mean rating.

Ilung Pranata, Geoff Skinner, and Rukshan Athauda’s “A Survey on the Usability and
Effectiveness of Web-Based Trust Rating Systems” revealed the results of a survey that asked
115 users which rating system they preferred the most. The paper goes over three main rating
systems: 5 star, binary, notation rating.
From all the participants, the 5 star rating was most preferred at 57% while the notation
and binary ratings got 18% and 19% respectively. The notation had one advantage over the 5
star: ‘it is easier to understand the differences between the rating scales...Thus, the rating
scales are more meaningful for the user.(Pranata.2)’

The study also asked about respondent’s opinion of the 5-star’s weaknesses- most
chose that it gave them choices, but a few had something different to say. 17 respondents
replied that the stars were ambiguous as they couldn’t tell the differences between the stars.
This meant we as designers had to do better to let the designer know what makes a 5 star
service a 5 star service.
Overall, each rating system provides a different user experience, but also has a conflict
between time requirement and personalization.

According to “Think like a Greek God: Psychology behind the 5 Star Rating” from
WebFx. They evaluate two choice systems, three star systems, and five star systems the
problem that each system has. It concluded that a bilateral system like asking yes and no
questions would only get the basic level of its feedback. People would be more likely afraid to
leave negative feedback in fear of receiving negative feedback. Three star system has the same
issue in which only a limited number of people would actually give negative feedback. Making
the result not that accurate. While a five star system gives the user more freedom to give
feedback without complication. The research also evaluated whether a more granulated scale
would improve the accuracy of the feedback. The result is it would actually allow deeper insight
and would make it harder for the user to make a decision. Which results in less review and a
smaller collection of data they can get.

Primary Research
We have presented some of the interview questions and also some follow-up questions
hoping to get a deeper insight into the user.
Preset questions we had are the following:
● How often do you rate a product or service?
● What are some complaints you have during a ride like a Lyft/Uber ride?
● Is there like some other rating system that you used that you prefer?
● Do most of the time, do you give a 1 or 5 star. Do you ever give anything in
between?
● How often/when do you give an extreme rating?
● Do you feel like the 5 star rating system limits the way you express your thoughts
in any way?
Take away:
● The importance of anonymity for rating systems. Interviewer said they are more
likely not giving the real feedback if people would see the rating right away when
they give a rate
● The interviewer thinks when the system default sets the rating as 5-stars and the
user can adjust later on. The information won’t be that accurate due to the fact
people are just lazy and don't want to do the rating. However, it might increase
people's engagement with the rating.
● The interviewer feels frustrated about the rating system when they would need to
leave a comment or reply more when they rate a score less than five stars. In this
case, she would adjust her decision on the rating by reason she doesn’t want to
spend time typing the comments.
● For a lot of people, whether they will spend their time on rating largely depends
on their mood. In most cases, they will choose to ignore it directly.
● The most common complaints about users would be long wait times and distant
pick-up points.
● Interviewer thinks the “five-stars rating system” is the most common rating
system he has ever seen; sometimes can see smiling faces and crying faces, But
he still thinks the “five-stars rating system” is the best one that could reflect the
user experience of all rating systems.
● Interviewer thinks it’s unnecessary to leave a review when service is mediocre.
He thinks written reviews are only kneely needed when customers encounter
problems, and the angry users will not hesitate to document the process and
describe them one by one.
● Interviewer seldom rates the product or the services. But, he points out that
giving an option of rating it later would increase him to do it.
● Interviewers had mostly been using the star system and they can’t really recall
any other system that they had used before. They didn’t find the five star system
a bad design, actually.
● Most of the time the interviewer stated that they would usually give a five stars
when they didn’t find any major issues while they were on the ride.
● Interviewers also point out that they are used to the five stars system. Changing
to another system might not be the best solution. They feel like the five stars
system is simple and effective.
● The limitation that came across from the interview is saying if they don’t leave a
review like writing in a comment box. It is hard for people to know the reasoning
behind them.
Target User Group
Our target demographic’s age range is around 18-20 year old college students. This
crowd can’t afford to buy a car while they don’t need a car every day. They use Lyft/Uber or any
ride-sharing app to get around places occasionally when needed. For example, weekly grocery
shopping or going to a party. According to the law, people under 18 years old can’t take a ride
alone while 21-year-olds or older have the ability to rent a car by themself. This is the specific
reason we target this narrow age range. Specifically, 18-20 year old riders that don't think the
overall rating is able to let them make the right decision to choose a driver. They would want to
know more additional information; while, they got specific preferences that hope the driver
would match with their wish. For example, they might have car sickness when the driver drives
too fast. Or the user has mysophobia which would hope the car is pretty clean and tidy.

Brainstorming

(Jinwoo)
After my research, I thought of keeping the 5-star rating system as it was already
preferred over the others, and only a few minute details had to be worked on to perfect it. During
the feedback session the class had during February 9th, I also noticed that too many of my
peers use the binary system. If my group were to go in that direction, presentations would be
saturated with too many of the same solutions. Also, the binary system brought its own sets of
problems as well: two-thirds of the users reported wanting more options.

The first problem was that users handed out 5-stars too generously. To solve this issue, I
thought of making the users tap a button on the screen to fill each star up separately. Granted,
this made the reviewing process longer, but made the users think twice about giving a perfect
score. Moreover, the distribution scale that was proposed by the Harvard Business Review
article was also considered, but was scrapped off as it would clutter the interface.

(Zheng He)
My goal, while brainstorming, was to make the user interface as concise as possible.
The reason for this was that an interface that is too complex will make the users lose interest in
evaluating. So we don’t intend to provide too many questions for users to answer. At the same
time, we think just clicking the stars is too simple, and sometimes users might not think about
the feeling of the trip. In that case, we figured out a different type of rating, which is “Fill the ring
by long press on the screen.” This design allows users to recall more feelings of their trips when
they are evaluating.

(Amanda Huang)
After doing some research on different kinds of rating systems. Seeing the pros and
cons of each system and concluded that 5-stars rating would still be the best fit for the rating
system. I thought about using it as a base and adding some additional features along with it. So,
there won’t be that much of a limitation on the rating feedback.
Additional features that I came up with are using tags. Tags just like hashtags or
keywords which our user group are pretty familiar with and are interesting in selecting these
things. Besides that, due to the user group, they might not have time to immediately do the
feedback and rating when they arrive at the destination. I came out with an idea that the user
could do the rating later today.

This is a brainstorming idea about different categories for the user to do the rating. Furthermore,
under each category what are the tags that can be represented and wording for each tag.
This is a sketch of our first idea about the feedback page. In this idea, we include the tag idea
that we previously came up with. While each category is separate on different pages. We hope
this would make the interface clean instead of clustering all the choices together. Additionally,
doing the brainstorming we also came up with the idea that the user might not have time to do
the rating. So, we add a feature at the bottom to save the rating at another time. The app would
pop out a notification on the time you choose whether is one or two hours after the ride or
before midnight.
This is a sketch of our second idea about the feedback page. On this page, we put different
categories on a single page. We were trying to figure out whether it would be a cluster or not. By
reason, separating each category into different pages might make the user feel annoyed and
would also be time-consuming.
This is a sketch of our third idea about the feedback page. We combine our previous idea about
letting each category set in one page to reduce the time needed when clicking through different
pages to finish filling up the rating. We also added another feature that was being discussed
during our brainstorming stage which is letting users press a button to give the rating. In this
case, we hope the user can think twice before they actually give a full mark to the driver.
This is a sketch for our fourth idea about the feedback page. The concept is still about the
duration of pressing a bottom would result in how many stars you would give the driver. We
were thinking about whether instead of making a simple ring using a cute figure would make the
user more engaged in giving the feedback or not.
This is a sketch for the driver rating score page. On this page, we include the categories that we
brainstormed previously. So, the user could see how other riders rate the driver instead of just
an overall score that covers every aspect. The page would still show basic information about the
driver. While we hope the 5-stars rating score would just be a reference. The main point would
be the rider can see whether the tag under each category matches with them before they place
the order.
This is a sketch for recommendation pop out. This feature isn't related to the rating system but
was a feature that would make the rider/user feel more user-friendly when they use the app.
When the user opens the app if the user has a routine to order a ride on a specific time of day.
The app would recommend the trip that they gather from the big data. So, the user doesn't have
to input all the information all over again. While the user can make small adjustments on place
to pick up or drop. This would save time for the user while they would feel more engaged when
they place the order.
Usability test
Goal:
Our primary goal for the usability test is whether the user feels any confusion
about the interface that we design. Whether they would realize our design and won’t get lost in
our design. Moreover, give us any feedback that they found when they were interacting with our
prototype.
Testing process:
We bring our sketches to our users and give them specific tasks for them to do.
The specific task we give them is a story as a reference and ask them how they would do the
rating. While they are using the prototype and trying to do the rating we observe any things that
hold them to think about before they do the next action.

Take away:

● Fill the circle by the pressing duration (might accidentally choose the wrong one)
● The color of the circle should be the same or it would be confused and make the
user feel messy
● For the departure time. We got three choices “early, on time, late”. The user
thinks it is unnecessary to include “early” due to the fact that the user won’t care
if the driver came early or not. And users usually won’t come early either
● For viewing the driver score, each tag was overlaid with a bar showing the
percentage of how many riders chose this tag but the user didn’t get the point
and wondered what that meant. They suggest adding a percentage next to it. Or
add overall how many people had rated this driver
● New users might be confused with the usage of the press button because of the
position and shape.
● New users might confuse the tags of “driver’s information page” with buttons and
think that’s ambiguous.
● An area for written comments is needed.
● Some users might think it’s annoying to force passengers to rate on the driver.
Final design:

This is a page where it would pop out when the rider finished their ride. It includes two
categories that the user needs to choose after the ride which includes safety and overall
satisfaction. These two categories could only be rated after the ride. On the top of the interface
which includes the pressing bottom for the user to rate the driver. The duration of the time
pressure would influence the score given. But, importantly it lets the rider think twice before they
actually give their score.
This is a page where it would pop out when the rider gets on the ride. This user group has a
behavior of being addicted to the phone. While they would frequently want to check the phone.
For example, how long they would get to the destination. Whether the driver detours or not. By
these specific behaviors, we finalize our design by popping out a simple rating page when the
rider gets on the ride. This page would only disappear when the rider gave a rate for these two
categories. These two categories include departure time and whether the car is clean. They are
obvious for riders to observe and could save overall time for doing the rating. These two
questions are quick and easy to answer so it could eliminate people from just randomly clicking
the button.
This is a page where the user can use this interface and see the rider rating score out of 5 stars.
Along with four different categories which include departure time, clean, safety, overall
satisfaction. Under each category would have different tags. Each tag would be colored by the
percentage of each category and would make it obvious for users to view individual categories.
The tag gives the user additional information to make a decision on which driver they want to
take.
Reference for secondary sources:

“The Problems with 5-Star Rating Systems, and How to Fix Them.” Harvard Business
Review, 2 July 2019,
https://hbr.org/2019/07/the-problems-with-5-star-rating-systems-and-how-to-fix-them.

Pranata, Ilung, et al. “A Survey on the Usability and Effectiveness of Web-Based Trust
Rating Systems.” IEEE XPLORE, 26 Sept. 2013,
https://ieeexplore.ieee.org/abstract/document/6607882. Accessed 14 Feb. 2022.

Jayathilake, Nelini, and Darshana Sedera. “A Critical Assessment of Online Vs

Traditional Review Characteristics.” Arxiv, 27 Oct. 2021,
https://arxiv.org/abs/2111.04452. Accessed 14 Feb. 2022.

“Ride-Sharing Industry Statistics for 2021: Policy Advice.” PolicyAdvice, 25 Apr. 2021,
https://policyadvice.net/insurance/insights/ride-sharing-industry-statistics/.

​“Think like a Greek God: Psychology behind the 5 Star Rating.” WebFX, w/
https://www.webfx.com/blog/internet/psychology-behind-5-star-rating-online-reviews/.