Investigating the Effort Heuristic

Psychology HL Internal Assessment

Candidate number of the author: jlq408

Candidate number of other group members: jlq439; jlq440
Date of submission: spring 2021
Word-count: 2150
Contents

Contents 1

Introduction: 2

Exploration: 3

Analysis: 4

Evaluation: 6

Works cited: 8

Appendices: 9
Appendix I. Letter of consent 9
Appendix II. Standardized instructions 10
Appendix III. Materials 11
Painting by Deborah Kleven Morbeto 11
Low-effort condition form 12
High-effort condition form 13
Appendix IV. Debriefing notes 14
Appendix V. Raw data 15
Appendix VI. Calculation of inferential statistics 15

1
Introduction:

One of the first to study heuristics in human decision-making were the psychologists Amos
Tversky and Daniel Kahneman. Their work defines heuristics as an experience-based strategy for
solving a problem or making a decision that often provides an efficient means of finding an answer
but cannot guarantee a correct outcome (“Heuristic,” 2020). Heuristics can be explained by the
Dual-process theory which divides human cognition to two different processes - implicit and explicit.
Implicit processes are mostly unconscious, simple and require little mental energy, whereas explicit
processes are conscious, complex and energy-intensive. Therefore when an individual needs to make
a quick but complex judgement requiring process 2 such as answering the question “Which letter is
the most common in the English language?”, they think of a letter that is most familiar to them
instead of going through all words and counting each letter, therefore, using process 1 to save time
and energy. This particular cognitive shortcut is called the availability heuristic; however, there are
dozens other heuristics and new ones are constantly being identified.
The study by Kruger, Wirtz, Van Boven, Altermatt et al (2004) aimed to present a new type of
heuristic by demonstrating the influence of perceived effort on the rated quality of an artwork. The
researchers stated that we might use effort as a heuristic of quality. They explained it as follows: “The
more effort invested in an object—be it a painting, a poem, or a paper submitted to JESP—the better
it is deemed to be.” (Kruger et al., 2004) They conducted 3 experiments each containing different
artworks - Experiment 1 poem, Experiment 2 paintings and Experiment 3 medieval armour - in which
participants judged their quality. This study is replicating Experiment 1 that used a sample of 144
beginner psychology students. The participants were given a poem Order by poet Michael Van
Walleghen and on the side of the paper was information about the poem - title, author, age of the
poet, and the time he spent writing it. Half was told that it took the author 4 hours to write the poem
(low-effort condition) and a half was told 18 hours (high-effort condition). After they have read the
poem and written down all the information, participants were asked to rate how much they like it
(from 1 “hate it” to 6 “it’s OK” to 11 “love it”), the overall quality of the poem (from 1 “terrible” to 6
“OK” to 11 “excellent”) and how much money they thought it would get if sold to a poetry magazine.
The mean value of liking and overall quality for the high-effort condition was 6.43 points, whereas for
the low-effort condition it was 5.84 points (F(1,136)=3.98, p= .048). The participants in high-effort
condition indeed tended to favour the poem more and the results supported the hypothesis that
perceived effort does indeed influence one’s rating of a product.
Our study aims to explore whether teenage boarding students use effort as a heuristic for
the quality of an abstract painting. The results might suggest to what extent students grown up in

2
isolated environments perceive the effort as a cue for quality in comparison to university students
from the original study. It might also be helpful for school counsellors to know if the students tend to
judge each other's work based on the time spent on it.
The independent variable is the information about the time it took the artist to finish the
painting - low-effort condition (4 hours), high-effort condition (26 hours). The dependent variable is
the participants’ rating of the painting on the scale of overall quality (from 1 “terrible” to 6 “OK” to
11 “excellent”).
This study hypothesizes that participants presented with the high-effort condition (26 hours)
will rate the painting significantly higher on the scale of the painting’s overall quality than those
presented with the low-effort condition (4 hours). The null hypothesis is that effort will not
significantly affect the participants’ rating of the painting on the scale of overall quality.

Exploration:

This lab experiment used an independent samples design of two randomly-divided treatment
groups. We did not choose repeated measures design as it was important that participants were not
aware of our independent variable and at the same time, were shown only one painting. The two
groups were also instructed simultaneously so that their awareness of two conditions would not
result in demand characteristics as they would try to guess the difference.
Our sample included 23 boarding students at Open Gate School in the Czech Republic 18.1
years old on average and all of Czech nationality. 15 were male and 8 female. Every participant was
an IB diploma candidate without a psychology course in 1st or 2nd year. All of the participants had
the English language on at least the level C1. We have chosen this sample because we specifically
aimed to study highschool students as a target population and these years were the ones we had the
easiest access to.
We have chosen to present the participants with a painting instead of a poem because we
assumed that our students might not have yet created their personal criteria for quality regarding
poetry and it could result in a wider spectrum of answers. Therefore, we chose a painting by Deborah
Kleven (see Appendix III.) which we believed might be more natural for our sample to assess. We
have also created two forms (see Appendix III.) that contained information about the painting (some
of which were fabricated and differed only in the time of creating the painting), space column to
rewrite the information into, two scales where participants circled their rating and a question
whether the participants have seen the painting before. The scales were: how much they like the

3
painting (from 1 “hate it” to 6 “it’s OK” to 11 “love it”) and the overall quality of it (from 1 “terrible”
to 6 “OK” to 11 “excellent”). The usage of two differently worded scales aimed to avoid the possible
biases connected to people focusing only on one question. Even though the comparison of the
difference in rating of the two scales might be insightful, in spite of simplicity, we will be focusing
only on the results of the overall quality.
The experiment was carried out on Monday morning as there was a free lesson. In the
beginning, we invited all the participants to randomly choose a seat at individual tables in a
classroom. The participants were read the standardized briefing ( see Appendix II.) and then given an
informed consent form to sign (see Appendix I.). Afterwards, they were randomly given one of the
two forms and a photograph of the painting was projected on the whiteboard at the front of the
class and stayed projected till the end of the study. Next, participants filled in the forms. Participants
were asked to rewrite the information as a control, in order to make sure they have processed all of
the data and used it in the evaluation of the artwork. After 2 minutes, the filled out forms were
collected and all of the participants were debriefed (see Appendix IV.).

Analysis:

The type of our experiment was independent measures and it produced ordinal data in the
form of points on a scale from 1 to 11 (see Appendix V.). Therefore, we used the Mean, Standard
deviation, Range and The Mann Whitney U test in order to analyse the rating of the painting.

Table 1: Descriptive analysis of data

Group Mean rating of the Standard deviation Range of rating

painting [points] [points]

high-effort 5.58 1.88 3 - 10

low-effort 6.00 1.48 3-8

Graph 1: A comparison of the mean of rating and effort conditions

4
 As can be seen above in table 1 and graph 1, participants in low-effort condition rated the
picture slightly more positively than participants in high-effort condition. However, the difference is
only slight, since both means of rating are in the neutral section of the scale from 1 “terrible” to 6
“OK” to 11 “excellent”.
The standard deviation of the high-effort condition was 1.88 and the range 3 - 10 points,
whereas the low-effort condition had the standard deviation of 1.48 and the range 3 - 8 points.
Therefore, we can conclude that there was a greater variance and range of data in the high-effort
condition than in the low-effort condition. The difference in deviation and range can be due to a
divergent range of art appreciation between the groups as both use a small sample. Or it could be
noted that the smaller range could be linked to the difference in our variable between the
conditions. The mean values of rating points were 5.58 for the high-effort condition and 6.00 for the
low-effort condition. This indicates that participants who were exposed to the condition of greater
effort actually rated the picture on average by fewer points. Therefore, even if we found significance
among our results, we still would have to reject our hypothesis as the results are directly against it.
The Mann Whitney U test had been used to calculate the significance of the data and it
indicated that the data are insignificant, since p = 0.870 and U = 85.5. If we were to reject the null
hypothesis there is 87.0% chance of error, and it would not support our hypothesis as well, since the
findings would mean the opposite. Hence, p-value > 0.05, the null hypothesis is accepted and the
hypothesis is rejected. The randomly selected value of a low-effort condition’s population is
considered to be greater than or equal to the randomly selected value of the high-effort condition.
Therefore, effort does not significantly affect the participants’ rating of the painting on the scale of
overall quality.

5
Evaluation:

Our results are significantly different in comparison to the original study. Our participants
rated the painting by 6.00 points in the low-effort condition and by 5.58 points in the high-effort
condition; whereas, participants in the original study rated the art piece by 5.84 points in the
low-effort condition and by 6.43 points in the high-effort condition. Our data show that the
participants in condition with smaller perceived effort rated the painting with more points on
average. Both conditions also produced a much greater variance in ratin. Overall, our results
challenge the validity and reliability of the research method.
The differences might have occurred because of the age difference of participants since our
participants were slightly younger and still very greatly influenced by the high school environment.
Yet, this fact brings to question the generalizability of the research as it suggests that the perception
of the relationship between time, effort and quality by different age and social groups is more fluid
than the effort heuristic theory suggests.
A strength of our experiment was that we had prepared two forms and randomly gave them
out as people sat. This avoided participants being aware of the two conditions and it also excluded
any other sampling or researcher bias. Given the simple and quick process we were also able to avoid
demand characteristics and the order effect which would otherwise devalue the validity of the study.
However the type of medium used in the experiment has shown to be a limitation of the
study. During the debriefing, many participants claimed not to be sure by which standards they
should rate the quality of the image. For the following research, we suggest using a medium that is
more familiar to the participants such as an essay for students, so that they have already existing
normalized standards by which they would rate the medium. Another limitation of our research was
that it was not clear to what extent participants took the information about time into account when
providing a rating. When debriefed, many participants showed surprise when being told about the
role of the time taken to create the painting and reported not focusing on the information at all.
Therefore, we suggest providing the participants with one more painting with a different time taken,
so that they can compare the two informations about invested time and can base their rating more
on the perceived effort. This would prevent the participants overlooking the key information and it
would raise the validity of the research.
The results obtained might have been also affected by the confounding variables. We were
not able to control the previous whole-school events which got delayed, the participants' time of

6
arrival as some came late and also the other activities the participants had after the experiment. This
all could have affected their focus and mood which devalues the study’s reliability and credibility.
Since we have not supported the findings of the study by Kruger, Wirtz, Van Boven, Altermatt
et al (2004) which also pioneered the idea of the new effort heuristic, we cannot support the concept
of this very heuristic either. Since we focus solely on effort heuristic which involves only a fraction of
the theory of cognitive biases and heuristics, our findings do not affect the theory.
As a conclusion our research suggests that it is not clear to what extent effort put to an
activity affects the perceived quality of it and we encourage further research.

7
Works cited:

Heuristic. (2020). In APA Dictionary of Psychology. American Psychological Association.

https://dictionary.apa.org/heuristic

Kruger, J., Wirtz, D., Van Boven, L., & Altermatt, T. W. (2004). The effort heuristic. Journal of

Experimental Social Psychology, 40(1), 91–98.

https://doi.org/10.1016/s0022-1031(03)00065-9

Morbeto, D. (n.d.). DSC_0125 [Painting]. Deborah Morbeto Fine Art.

https://www.thedebweb.com/gallery?lightbox=dataItem-k1zf294w

8
Appendices:

Appendix I. Letter of consent

IB Psychology Informed Consent Statement

Study Title: The effort heuristic

Experimenter(s): xxx, yyy, zzz

Description of Experiment:

Participants will be presented with a painting by artist Deborah Kleven and basic information about the art
piece. They will be asked to record the information in a form and rate the artwork on two scales: how much
they like it (from 1 “hate it” to 6 “it’s OK” to 11 “love it”) and overall quality of the painting (from 1 “terrible”
to 6 “OK” to 11 “excellent”). Lastly, the participants will be asked to write down whether they have seen the
painting before.

In order to participate in this research study, it is necessary that you give your informed consent. By signing
this informed consent statement, you are indicating that you understand the nature of the research study
and your role in that research and that you agree to participate in the research. Please consider the
following points before signing:

● I understand that I am participating in psychological research.

● I understand that my identity will not be linked with my data and that all information I provide
will remain confidential.
● I understand that I will be provided with an explanation of the research in which I participated
and be given the name and email address of an individual to contact if I have questions about
the research.
● I understand that certain facts about the study might be withheld from me, and the
researchers might not, initially, tell me the true or full purpose of the study. However, the
complete facts and true purpose of the study will be revealed to me at the completion of the
study session.
● I understand that participation in research is not required, is voluntary, and that, after any
individual research project has begun, I may refuse to participate further without penalty.
● I understand that the results of this research will be used only for the purpose of IB Psychology
Internal Assessment.

By signing this form, I am stating that I am over 16 years of age and that I understand the above information
and consent to participate in this study being conducted at Open Gate Boarding School.

Signature: _______________________________________________(of participant)

Today’s Date: ________________

Your Name: ________________________________________________

Your Age:___________________

9
Appendix II. Standardized instructions

Hello, thank you for coming and welcome to our IB psychology experiment. The aim of this
experiment is to investigate what factors influence how we evaluate art. We will show you a
painting by Deborah Kleven, provide you with some basic information both about her and about
the art piece, and then ask you to rewrite the information into a prepared form to ensure you have
really processed what you read. Then we will want you to rate the painting on a scale from 1 to 11
in two categories, firstly by how much you like it and then by how good and well-done, you think
the painting is. In both cases, number 1 signifies the worst possible rating, number 6 is the middle
of the scale and number 11 is the highest score you can give. Lastly, we want you to answer yes or
no to whether you have seen this painting before.
During the whole experiment, we ask you not to communicate among yourselves, verbally or
otherwise. After you finish filling out the form, raise your hand and we will collect it.
Please know that all data gained in this experiment will remain anonymous, will be used only for
the purpose of our psychology IA, and you can decide to stop participating at any time during the
experiment. If you wish to do so, raise your hand and tell us. Your data will not be processed if you
decide to withdraw from the experiment.
If you have any questions, please ask them now. If there aren’t any, we will begin with the
experiment.

10
Appendix III. Materials

Painting by Deborah Kleven Morbeto

11
Low-effort condition form

INFORMATION ABOUT THE PAINTING

Name: Fish in a Pond
Author: Deborah Kleven Morbeto
Author’s age: 55 years
Author’s nationality: American
Year: 2007
Time spent creating the painting: 4 hours
Art technique: acryl on canvas

Dear participant,
please rewrite the information about the painting below:
Name:
Author: 
Author’s age:
Time spent creating the painting: 
Art technique:

Have you seen the painting before?      YES       NO

How  much do you like the painting on a scale from 1 (hate it) to 6 (it’s OK) to 11 (love it)?
1 2 3 4 5 6 7 8 9 10 11
How would you assess the ‘‘overall quality’’ of the painting on a scale from 1 (terrible) to 6 (OK) to 11 (excellent)?
1 2 3 4 5 6 7 8 9 10 11

12
High-effort condition form

INFORMATION ABOUT THE PAINTING:

Name: Fish in a Pond
Author: Deborah Kleven Morbeto
Author’s age: 55 years
Author’s nationality: American
Year: 2007
Time spent creating the painting: 26 hours
Art technique: acryl on canvas

Dear participant,
please rewrite the information about the painting below:
Name:
Author: 
Author’s age:
Time spent creating the painting: 
Art technique:

Have you seen the painting before?      YES       NO

How  much do you like the painting on a scale from 1 (hate it) to 6 (it’s OK) to 11 (love it)?
1 2 3 4 5 6 7 8 9 10 11
How would you assess the ‘‘overall quality’’ of the painting on a scale from 1 (terrible) to 6 (OK) to 11 (excellent)?
1 2 3 4 5 6 7 8 9 10 11

13
Appendix IV. Debriefing notes

Firstly, we would like to thank all of you for participation in this experiment. The aim of this study
was to explore whether people use effort as a heuristic for the quality of an artwork. A heuristic is a
mental shortcut to making a decision that relies more on approximation than exactness. Our
experiment replicated part of the 2004 study by Kruger et al. They have observed 0.59 points
deviation of rating caused by changed perceived effort.
We divided you into two groups. Each was presented with the same information about the artwork
projected except for the time the artist has supposedly spent painting it. This was done so that we
could observe whether the time spent on an artwork influences the observer’s perspective of the
work’s quality because people seem to use time as a unit of effort.
Our hypothesis was that those of you presented with the information that the painting took the
artist longer to create would rate the painting higher both on the scale of how much you like it and
on the scale of the painting’s overall quality than those told it took the artist fewer hours.
If the results support the hypothesis, it would mean that people might use effort or time spent as
an index of quality. This finding would be helpful for school counsellors or teachers dealing with
students’ unreal expectations. It would also provide an insight into the mechanisms of how people
evaluate themselves and others.
Had you any questions, do not hesitate to ask us now or contact us on our e-mail addresses: xxx;
yyy; zzz. If for any personal reason you decide that you do not wish for your results to be part of the
study, contact us on the aforementioned e-mails as well.

14
Appendix V. Raw data
Table 2: ratings of individual participants
Low-effort High-effort
Group condition condition

Scale like it overall like it overall

10 6 2 6

8 8 9 10

7 5 7 5

7 7 7 6

4 4 3 5

8 7 4 3

8 6 5 4

9 7 8 5

4 3 1 5

9 7 9 8

10 6 5 6

8 4

Appendix VI. Calculation of inferential statistics

Calculation of standard deviation for the high-effort condition

15
Calculation of standard deviation for the low-effort condition

Calculation of p-value:

16