Title page

Motivation:
During our orientation week at IIMB as PGP1s, we were taken to an outdoor activity place
called- ‘Break Through’. The objective was to teach concepts like leadership, coordination, etc.
through games. One activity that we as a group vividly remembers is the one shown in the below
picture.

Students were divided into teams and asked to compete. The team that goes maximum distance is
declared the winner. In the initial round teams couldn’t even complete 1/4th lap. Before beginning
the next round, organizers deliberately mentioned that our senior batch could finish 2 laps in the
same given time. In the second round, it was not just the best team improved their performance
but all the teams outperformed themselves.
If we keenly observe the underlying phenomenon, we questioned ourselves if our senior batch’s
performance measure worked as a reference point in our mind and impacted the performance.
In an ECONS world, everyone performs his/her best to maximize their utility irrespective of any
other person’s performance, but we wanted to know if HUMANS’ performance can be impacted
by setting a past statistic as the reference point.

Problem Statement:
We decided to test the following observations:
A) Does reference point impact the performance?
B) If yes, does the difference in levels of reference point have a different impact on the
performance?
In order to test the above phenomena, the 2 following hypothesis were formulated giving the
reference point at two different levels (one is of overachievers’ and another of a similar group’s)
to see if there is any change in the performance:
NULL HYPOTHESIS 1:
Showing an average score of overachievers before taking a test, as a reference point, does not
change the performance of students
NULL HYPOTHESIS 2:
Showing an average score of a similar group before taking a test, as a reference point, does not
change the performance of students
EXPERIMENT:
Experiment was designed in such a manner as to test if the average time taken by a group of
respondents to finish a test changes significantly when the average score of another group of
people is shown before taking the test.
While conducting the experiment, some of the following assumptions were made:
1. Average score of a group of participants is taken as the performance indicator
2. Time taken to finish the Sudoku is used to measure the performance capability of
individuals
3. Average time to finish the Sudoku from a website dedicated to Sudoku enthusiasts is
taken as overachievers’ performance
4. The score taken from the website is treated at a different reference level compared to the
average score of IIMB students as not all the students are sudoku enthusiasts.
Design of experiment:
Test: Sudoku puzzle (easy level)
Sudoku puzzle was chosen as the preferred test to measure the performance of the candidates for
the following reasons:
I. Majority of us are familiar with it
II. It is a good measure of agility and thinking ability of respondent
III. Overachiever statistics available online for treatment 1 reference point
Metric to measure performance: Time taken to complete the entire Sudoku puzzle.
Platform: Qualtrics
Qualtrics offers features such as ability to add custom CSS, JavaScript to questions which helped
us a lot in building the Sudoku puzzle. It also has randomization inbuilt hence making it easy to
conduct the experiment for control and treatment groups.
Flow of the experiment:

Demographic data
Instructions Reference point Test
collection
Measures taken to ensure genuine responses:
I. Respondents were provided with incentives worth up to INR 1,000 to decrease the
percentage of fraudulent responses. The top three performers in terms of the least time
taken to solve the puzzle were awarded prizes in cash and kind.
II. The reference point data (average time taken) shown to the treatment groups was made
salient. Also, a 5 second gap was added before the respondent could move to the next
page to ensure they read the information provided.
III. Each of the responses obtained was verified by checking for correctness of each of the
cells of Sudoku. Fraudulent responses were removed before performing data analyses.
Reference point data source:
As mentioned in the hypotheses, we needed two reference points, one for each of the treatment
groups.
Treatment group 1:
The respondents present in treatment group 1 were shown the overachiever average time taken to
complete the Sudoku. This value was obtained from a website 1 which is used by Sudoku
aficionados.
The value of this reference point was 1 minute and 56 seconds (116 seconds).

1
nanpre.adg5.com/select_en.php?lv=1&mode=&page=6
Fig 1. Screen grab of
website from which
overachiever avg. was obtained
Treatment group 2:
The respondents present in treatment group 2 were shown the average time taken to solve the
Sudoku puzzle by a similar group of respondents. To obtain this value, we had 15 students of
IIM Bangalore play the Sudoku game without promising any incentives and keeping everything
else the same. The average time taken by this group of respondents, which was 3 minutes and
54 seconds (234 seconds), was quoted to the treatment group 2.
Sampling technique: Convenience sampling
The form was then floated mostly to the students of IIM Bangalore, with randomization option in
Qualtrics enabled to show different data (reference points) to control and the two treatment
groups.
Treatments:
Control Treatment 1 Treatment 2

Fig 2. Reference point page

shown to each of the groups
 The control group was simply shown a few instructions.
 The treatments groups were shown their respective reference points along with the same
instructions.
 Reference point was highlighted to increase salience.
 The button to proceed further appeared only after a gap of 5 seconds.
Game screen:
In the game screen, the respondent could see
their current time spent (in minutes and
seconds) at the top followed by the Sudoku
puzzle.
Initial entries in the Sudoku were presented
in a different colour to make it easier for the
respondents.
The Sudoku interface was mobile-friendly,
hence helping us reach out to a wider
audience.

Fig 3. Sudoku and timer

interface
Sample size and characteristics:
Along with the game responses, we also collected some demographic details such as age, gender
and education level. Overall, we collected 105 responses so that we have 90 valid responses after
filtering out the fraudulent ones. These 90 responses were equally divided among the control and
the two treatment groups. The responses were quite diverse with decent participation from
different genders and age groups.

F
i g

Gender Age
4. Responses distribution
Fig 5. Demographics of the respondents
Data analysis
Descriptive Statistics
The descriptive statistics of the three groups, control, T1 and T2 are shown in the table below.
We observe that the test metric of our analysis, which is the average time taken shows
considerable difference in control and T1 while the values for control and T2 are somewhat
close. The values of avg time taken for control recorded is 225.36 secs, T1 is 181.86 secs and T2
is 239.15 secs as per the bar graph shown below here. We see that the range of times taken has
also considerably decreased in case of T1 as in comparison to control as shown in the descriptive
statistics table.
Gender-wise Performance
If we look at the gender wise performance, the improvement in time taken to complete the
sudoku in males which was (243.8 – 195.80 = 48 secs) in Control vs T1 is found to be on the
higher side as in comparison to that of females which is (182.32 – 163.62 = 18.7 secs). However,
females have fared better than males in both Control and T1 if taken separately.
Results
In order to check if the mean difference for average time taken to complete sudoku between
Control and T1 and that between Control and T2 are significant or not, we performed 2 sample
independent t test for mean comparison. Results are shown in the table below here.
Hypothesis 1
If we look at independent T test results performed between Control group vs Treatment 1 which
was shown reference of 1 min 56 secs for first hypothesis, we observe the 2-tailed significance to
be 0.05 which is significant at 95% confidence level. Thus, through this we can conclude that the
reference point shown of average time taken by overachievers (1 min 56 secs) to T1 led to
significant improvement in time taken by them to complete the sudoku. Hence, we reject null
hypothesis 1 based on this conclusion which states that showing an average score of
overachievers before taking a test, as a reference point, does not change the performance of
students.
Hypothesis 2
Now, if we look at independent T test results performed between Control group vs Treatment 2
which was shown reference of 3 min 54 secs for second hypothesis, we observe the 2-tailed
significance to be 0.598 which is not significant at 95% confidence level. Thus, through this we
can conclude that reference point shown of avg time taken by similar group (3 min 54 secs)
shown to T2 did not lead to significant improvement in time taken to complete sudoku. Hence,
we fail to reject null hypothesis 2 based on this conclusion which states that showing an
average score of a similar group before taking a test, as a reference point, does not change
the performance of students.
Recommendations:
Outcome of our study is that displaying performance of overachievers can lead to an
improvement in the overall performance of a group of individuals. Some of the applications of
this outcome are:
Classroom Scenario:
 One possible application of the findings in a classroom scenario would be to display the
statistics of top 50 percentile students to the entire class before taking a test. Since the
statistics of top 50 percentile students will be better than that of the entire group, this
could lead to a significant improvement in their performance.
Business Scenario:
 Consider a scenario in which sales staff is being readied for their next project. Displaying
the sales figures achieved by the top salesmen for a similar project in the past could lead
to a significant improvement in their performance. Similarly, in a scenario dealing with
deadlines, displaying performance of overachievers could lead to improvement in the
turnaround time.
Limitations
 Some of the respondents could be Sudoku enthusiasts