University of Toronto ECO331: Behavioural & Experimental Economics

Department of Economics Robert Gazzale, PhD

Term Paper Assignment

Winter 2019

1 Due Dates and Weights

First Submission Friday March 15, 2019, 11:00 AM (25%)

Peer Assessments Friday March 22, 2019, 11:00 AM (20%)

Final Submission Friday March 29, 2019 at 11:00 AM (45%)

Reflection Piece Friday March 29, 2019 at 11:00 AM (10%)

2 Project Goals
The purpose of the assignment is to provide you an opportunity to write a coherent, well-constructed
essay thinking critically about 1) the design of one particular experiment (internal validity); and 2)
how the choices researchers make in the design of an experiment impact the ability to extrapolate
experiment results to related environments (external validity). As part of this exercise, you will
also read and provide constructive feedback to your classmates on their writing. Symmetrically,
you will receive comments from classmates that you can use in revising your final draft.

3 Overview
Each of you has selected (or has been assigned) a prompt article.1 You will assess both the internal
and external validity of your prompt article. Based on your assessment of the article’s external
validity, you will propose a follow-up experiment.
For procedures to use in assessing internal and external validity, I refer you to
the reading notes posted on Quercus for Topic 2 (Experiment Design: Internal and
External validity).

4 Structure of the Assignment

The final product will be a 1,000 word2 essay. Rather than handing in a single draft, however,
students will submit a first draft which will receive comments and suggestions from classmates.
Based on these comments, and other lessons learned during the peer assessment phase, students
will revise their paper for final submission. In addition to the final version of the paper, students
will include a brief explanation of how they responded to their peers’ comments, and what changes
(if any) they made to their paper in the revision process. As part of this process, students will
comment on the first submissions of three of their classmates.
The final grade in this assignment will depend on all four elements, with the most weight on
the final version of the essay (see above). Note that your Peer Assessment grade depends on the
quality of the feedback you provide your peers. The feedback you receive from your peers does not
directly impact your grade.
1
For what it is worth, a couple of experimental economists are writing a book on the most influential articles
using human-subject experiments. They have compiled an initial list, and have asked the experimental economics
community to vote. All of the prompt articles come from this initial list. As luck would have it, almost every
experimental economics article in the ECO331 syllabus is on this list. Well done, Prof. G.
2
About 4 double-spaced pages.

20190206 1 Term Paper Assignment

University of Toronto ECO331: Behavioural & Experimental Economics
Department of Economics Robert Gazzale, PhD

4.1 Structure Details

First, four double-spaced pages constitutes a very scarce resource. Keep your introduction and
conclusions brief. I can assure you that if either exceeds half of a page, it is too long. Relatedly,
you should assume that the reader has read the paper. DO NOT SUMMARIZE!
In the first half of your analysis,3 I want you to address internal validity. Ask yourself why it is
hard, even in a laboratory setting, to establish the proposed causal mechanism. Then consider the
design details that help them establish causality. Keep the following in mind:

• More clearly arguing fewer points is definitely preferable to making many points but not
being able to convincingly make any of them (due to space constraints).

• Your goal is to convince the reader you understand what the authors did and why they did
it. Your reader should leave this section thinking “The experiment design was pretty nifty
because . . . ”

In the second half of your analysis,4 you will focus on external validity. In particular, you will
identify one experiment-design choice that plausibly contributed to your prompt article’s the main
empirical result (your hypothesis). You will concisely argue why this factor is relevant. You will
then identify how you would modify the experiment in the prompt article to test your hypothesis.
4.2 An Example
We already have a very good model of this. In lecture, we discussed how Shurchkov (2012) con-
ducted a follow-up experiment to Gneezy, Niederle and Rustichini (2003). If I was Olga Shurchkov
writing this assignment, this might be my outline.

• The subjects in Gneezy et al. (2003) completed mazes.

• This task is plausibly perceived to be male biased.

– Citing a paper here would be awesome.

• The authors’ preferred hypothesis was that because women were less confident in their ability
to complete mazes, they responded with less effort when competing against men.

• We therefore might get a different outcome if the task was perceived to be female biased.

• In my follow-up experiment, the task in both the piece-rate and competitive treatments would
be a word task . . .

– Citing a paper here would be awesome.

5 Some Details
It is worth reiterating:

1. More clearly arguing fewer points is definitely preferable to making many points but not being
able to convincingly make any of them (due to space constraints).

2. Likewise, the paper that does little more than summarize the prompt article will not do well.
Think value added . . .
3
Probably comprising a little less than half of the boy of your assignment, excluding introduction and conclusion.
4
Probably comprising a little more than half of the boy of your assignment, excluding introduction and conclusion.

20190206 2 Term Paper Assignment

University of Toronto ECO331: Behavioural & Experimental Economics
Department of Economics Robert Gazzale, PhD

3. Does your term paper show that you have a solid understanding of the experiment (big picture
and important details) reported in the article you chose? Does your paper demonstrate an
understanding of experimental or behavioural economics that you (likely) did not have in
December? If the answer to both of these question is yes, you will do fine.

4. It is not a bad idea to follow the standard structure (hopefully) drilled into our heads in high
school.

Introduction Tell the reader what you are going to do.

Body Do what you told the reader you are going to do.
Conclusion Summarize what you have done.

5.1 Important Elements

References. You must include properly formatted references to all works cited in your paper. This
is not a research assignment per se, and you will be able to earn good marks without consulting
external sources. I expect, however, that most of the better papers (e.g., those in the top 25%) will
cite three to five sources (from economics journals and recent working papers).

• The default format for references and citations is the Chicago Manual of Style. Other formats
are acceptable if they convey the same information.

• For each source that you cite, you must include in your references a brief, 2-3 sentence
annotation explaining precisely how the source was used in your paper.

• The references (and annotations) do not count towards the 1,000 word limit.

Audience. Your paper should be aimed at third-year economics majors who have never en-
countered behavioural experimental or behavioural economics, but who has read the journal article.
This means, for example, that you will need to clearly but briefly define such terms as loss aversion,
present bias, availability bias, etc. It also means that there no need to summarize the article.

6 Grading
• The grading rubric will be posted on Quercus. This describes the basic elements of the paper
that will be evaluated, and also how they score and contribute towards your final grade.

• The First Draft will be judged by the same criteria as the Final Version, with some allowance
for lack of polish. First drafts must follow exactly the same format as the Final Version (e.g.,
including annotated references, etc.). Incomplete first drafts will be heavily penalized.

• Peer Assessments will be judged on the quality (and evident effort displayed) of the feedback
provided to your peers’ papers.

• The Reflection Piece need not be long. It will be graded by the quality of insight you offer
on the revision process (including your assessment of your peers’ comments).

7 References, Citing and Quoting

Academic inquiry is a conversation. As an upper-year student at one of the top universities in
the world, you have every reason to be a part of this conversation. In this paper, I want your
contribution to the conversation.

20190206 3 Term Paper Assignment

University of Toronto ECO331: Behavioural & Experimental Economics
Department of Economics Robert Gazzale, PhD

7.1 Of Utmost Importance

1. If the idea, piece of evidence or analysis comes from someone else, you must use citations in
the text to give credit.

• It you have not credited someone else for an idea, you are claiming ownership of that
idea.5 This is plagiarism if the idea is not in fact yours, and plagiarism is amongst the
most serious of violations of the University of Toronto’s Code of Behaviour on Academic
Matters.

2. If the words you use are not your own, you must put the words in quotations and attribute
the words to the original author.

• You implicitly claim authorship of any and all phrases and sentences that are NOT
enclosed in quotes. This is plagiarism if the words are not in fact yours, and plagiarism
is amongst the most serious of violations of the University of Toronto’s Code of Behaviour
on Academic Matters. If you are using someone else’s words, you MUST quote
in addition to citing!

3. Bottom line: Citing gives credit only for the underlying intellectual output (idea, evidence,
analysis, etc) but not the words used to express the idea. Therefore if you cite an author but
the text is not in quotes, you are saying that while someone else gets credit for the idea, you
are claiming that the expression of the idea is yours.

4. Even better: Do not quote at all. See my arguments below.

5. Every citation in the text must have a corresponding entry in the list of references.

6. The Chicago Manual of Style 6 is the default method for citations and references. If you are
familiar with an alternative format, please feel free to use it. The bottom line is that it must
be clear who is getting credit for an idea, and where I can find the original “document.”

7.2 Details
List alphabetically cited articles and books in a references section at the end of the paper. A
standard entry should include the title, author, journal or book title, date published, and page
numbers if appropriate. For internet references, include title, author, hosting organization, date
published (if possible), and complete URL.
I have included at the end of this document a series of references using a specific style. Honestly,
I really do not care whose bibliography style you adopt as long as you adopt a standard style and
remain consistent.
In addition to the references at the end of the paper, you must use citations in the text to give
credit for any idea, piece of evidence, or quotation that is not your own. Every citation in the text
must have a corresponding entry in the list of references. If an idea emerged from discussions with
a colleague, include a footnote to the effect of “This point emerged through helpful discussions with
‘Joliet’ Jake Blues.” Here are some examples of acceptable citations:

• “While List (2003, p. 67) does acknowledge the real-world existence of the endowment effect
...”
5
The exception is for knowledge expected of any UofT economics major. You do not have to provide a citation for
the fact demand slopes down. You also do not have to provide a citation for the fact that Canada is, geographically,
a large country.
6
http://www.chicagomanualofstyle.org/tools citationguide.html

20190206 4 Term Paper Assignment

University of Toronto ECO331: Behavioural & Experimental Economics
Department of Economics Robert Gazzale, PhD

• “Burtless (1991) argues that although international trade has indeed grown as a share of the
U.S. economy in recent decades, the increase is too small to explain much of the rise in wage
inequality over this period.”

• “Although international trade has indeed grown as a share of the U.S. economy in recent
decades, the increase is too small to explain much of the rise in wage inequality over this
period (Burtless 1991).”

• If a series of sentences draw on the ideas or arguments of a particular author, you should
make it clear in the text exactly which parts of your paper are drawn from which author’s
writing. For example, put the arguments in a single paragraph started with “Burtless (1991)
makes the following set of arguments . . . ”

To reiterate (and reiteration is a device used to indicate importance), if you must quote directly,7
enclose in quotes any words written by anyone except you, and include in the text a proper citation
to the source. If you explain another author’s idea in your own words only, you just need to provide
a citation. However, if you use any sentences, sentence fragments or phrases written by someone
else, you absolutely must put them in quotes. Just as not citing the source of an idea is an implicit
ownership claim on the idea, you implicitly claim ownership of all text not in quotes. If you claim
ownership of text not in fact your own original writing, this is a violation of the University of
Toronto’s Code of Behaviour on Academic Matters.
7.3 Good Practises
The following “rules” greatly improve writing quality.

• Authors make points and report findings, not papers or articles. I thus prefer “Chen and
Gazzale (2005) present experimental evidence that . . . ” or “The authors present evidence
that . . . ” rather than “Their paper presents . . . ”

• Reserve the word “prove” (or disprove) for theory papers. Authors of an empirical paper
(such as one reporting the results of a human-subject experiment) will have results. These
results may support a particular theory, or they may be inconsistent with that theory. These
results, however, neither prove nor disprove anything.

• In general, you need not provide page numbers when citing the works of others. One exception
to this rule might be the case where a casual reader might miss an idea very specific to a
particular page.

• In general, avoid direct quotes. Assume that when you quote directly, the reader is
going to infer that you are too lazy to do your own writing. You should therefore
limit direct quotes to those cases where it is clear that your use of quotes is not due to laziness.
I can think of three valid exceptions to “no-quote” rule:

Eloquence However, economists write little so eloquent that only the original words do
justice to the idea.
Outrageousness
Subtlety

A good rule of thumb: unless use of the exact words is necessary, do not quote.
7
and it is rare that you must . . .

20190206 5 Term Paper Assignment

University of Toronto ECO331: Behavioural & Experimental Economics
Department of Economics Robert Gazzale, PhD

References
Bjöernerstedt, Jonas and Karl H. Schlag, “On the Evolution of Imitative Behavior,” ELSE
Working Papers Number 029, ESRC Centre on Economics Learning and Social Evolution
November 1996. Available at http://ideas.repec.org/p/els/esrcls/029.html.

Charness, Gary and Uri Gneezy, “Incentives and Habits,” May 2006. Unpublished manuscript
available at SSRN: http://ssrn.com/abstract=905026.

Fackler, Martin, “PlayStation 3 Pushed Back for Delivery in November,” The New York Times,
March 16, 2006.

Gigerenzer, Gerd, “Fast and Frugal Heuristics: The Tools of Bounded Rationality,” in Derek J.
Koehler and Nigel Harvey, eds., Blackwell Handbook of Judgement and Decision Making, Ox-
ford, UK: Blackwell Publishing Professional, 2004, pp. 62–88.

Gneezy, Uri, Muriel Niederle, and Aldo Rustichini, “Performance in Competitive Environ-
ments: Gender Differences,” Quarterly Journal of Economics, August 2003, 118 (3), 1049–
1074.

Gross, Daniel, “Dis-Integration?,” Slate, August 17, 2006. Available at http://www.slate.com/

id/2147952/.

Kephart, Jeffrey O., Christopher H. Brooks, Rajarshi Das, Jeffrey K. MacKie-Mason,

Robert Gazzale, and Edmund H. Durfee, “Pricing Information Bundles in a Dynamic
Environment,” in “Proceedings of the 3rd ACM Conference on Electronic Commerce” The
Association for Computing Machinery ACM Press New York 2001, pp. 180–190.

List, John A., “Does Market Experience Eliminate Market Anomalies?,” Quarterly Journal of
Economics, February 2003, 118 (1), 41–71.

Shurchkov, Olga, “Under Pressure: Gender Differences in Output Quality and Quantity under
Competition and Time Constraints,” Journal of the European Economic Association, October
2012, 10 (5), 1189–1213.

20190206 6 Term Paper Assignment