Intro To Research Student Manual 2017-2018

INTRODUCTION TO RESEARCH SKILLS
2017-2018
Student Manual
EUROPEAN STUDIES
YEAR 1, SEMESTER 1
FACULTY OF MANAGEMENT & ORGANISATION

Table of Contents
How to use this Manual .................................................................................................... 3

Course Introduction .......................................................................................................... 4
Your Assignment: Description and Instructions ..................................................... 7
Week 1 ................................................................................................................................... 9
Lecture: Introducing Research ...................................................................................... 9
Homework for Seminar 1 ............................................................................................. 21
Seminar 1 .......................................................................................................................... 23
Week 2 ................................................................................................................................ 25
Lecture: Making a Research Plan .............................................................................. 25
Seminar 2 .......................................................................................................................... 33
Week 3 ................................................................................................................................ 34
Lecture: Dealing with Sources .................................................................................... 34
Seminar 3 .......................................................................................................................... 41
Week 4 ................................................................................................................................ 44
Lecture: Argumentation and Logic ........................................................................... 44
Seminar 4 .......................................................................................................................... 53
Week 5 ................................................................................................................................ 55
Lecture: The Structure of Your Research Paper ................................................... 55
Seminar 5 .......................................................................................................................... 67
Week 6 ................................................................................................................................ 69
Lecture: Finalising Your Findings ............................................................................. 69
Seminar 6 .......................................................................................................................... 78
Appendix 1: Quick APA Reference ............................................................................ 80
2
How to use this Manual
This is the course manual for Introduction to Research Skills. It includes a detailed overview
of the course, including a course introduction, assignment instructions and a complete outline
of the course programme, detailing what students will do and discuss in class and what they
need to do in order to prepare.
If you are reading this, you have probably just downloaded and/or printed this manual. You
will likely have noticed that it’s a pretty thick book (here’s a helpful hint: if you haven’t printed it
yet, it would be a good idea to print double-sided and/or to print two pages on one, both of
which are possible when printing at school. But, please do not be intimidated by the size of
this manual, because the good news is that YOU ARE NOT EXPECTED TO READ EVERY
WORD OF THIS BOOK, unless you want to. You are, however, expected to have a copy of
this book and to bring it to every class, either on paper or on a digital device, because it
includes a lot of information, discussions and exercises that will be discussed in class, as well
as all of the homework assignments and detailed information about the main research
assignment for this course. You will be expected to know all of this information – to know what
to expect and what will be expected of you.
This manual is part textbook and part workbook. In class, your teacher will discuss the
material with you and you will use the book during in-class exercises. Outside of class, you
are welcome to use the book for reference or, if you so choose, for detailed reading. The
manual is supplemented by online materials, which are posted on Blackboard weekly.
Students will need to check the Blackboard course regularly to stay up to date.
Course Introduction
Research is one of the most important aspects of a University education and it goes without
saying that the ability to do research quickly and effectively is an essential skill for anyone
who wishes to succeed at school or in their career.
The dictionary defines research as “the careful study of something, especially in order to
discover new facts about it,” but this definition barely scratches the surface of what Research
means in terms of your education, career and life. At its core, Research is about gathering up
information, analysing it and coming to conclusions. You want to know about a specific
subject and come up with some questions, you search for information to help you answer
those questions and analyse that information in the hopes of reaching some useful
conclusions.
That sounds simple enough, but the truth is that the research process isn’t always easy. It’s
true that we live in an age of unprecedented access to information. Never before have so
many people had as much access to knowledge as they do today. The myriad of (multi-
)media sources provide constant content while technology lets people carry an encyclopaedia
around in their hip pocket and yet, many of us struggle to keep up. It’s hard to find the time to
take everything in and it’s even harder to know what to do with the information we find. An
upcoming professional needs to know how to find trustworthy information without getting lost
in all the details and different sources and, even more importantly, the upcoming professional
needs to know how to evaluate and analyse information in search of meaning so that they can
reach useful conclusions and acquire knowledge. These are not just academic or professional
skills. They are essential survival skills. If you want to be able to keep up and to keep growing
as a person or as a professional, you need to be able to find, organise and analyse
information.
That is exactly what this course is about: learning how to navigate the twists and turns on the
information superhighway in search of answers. You will learn how to create relevant
research questions about topics of interest and how to go about looking for information and
how to analyse that information to reach the answers you need. In doing so, you will learn
how to make and implement a research plan and you will learn about basic research
methods.
Desk Research
A significant part of this course is focused on Desk Research, which does not necessarily
refer to using a desk, but to any kind of research that can be done without going out into the
field. Desk Research involves visiting libraries (in person or in Cyberspace), reviewing
literature, news and other sources and, of course, using the Internet effectively and
responsibly.
Students will practice gathering information based on carefully thought out research questions
and evaluating that information’s credibility and validity, to determine whether or not the
information is trustworthy and appropriate to their research.
Critical Thinking
One of the most important competencies students will work one in Research Skills I is their
Critical Thinking Skills. Critical Thinking refers to the way in which a person gathers,
organises, processes and analyses information as well as the use of logic and reason to
reach conclusions or to make arguments.
One of the great challenges to researchers is the difficulty of interpreting information –

separating fact from opinion, determining the credibility of sources, finding relationships
between different factors and forming logical opinions. Critical Thinking will give you the tools
you need to accomplish those difficult tasks, while also enabling you to be more professional
and effective in constructing and communicating your own thoughts and opinions.
Critical Thinking goes hand in hand with Research Skills, so much so that these skills will be
dealt with throughout the course. After all, you need Critical Thinking to come up with
questions, to determine the right methods to answer those questions, to sift through the
information you find and to reach and argue on behalf of the conclusions you’ve reached.
The Research Process
Research is the process of answering questions and the research process itself can be
summarised as a series of simple steps. Because research is about answering questions,
most of these steps involve asking them. It starts with a central question: something you want
to know. Once you have a clear question, you have to figure out the best way to answer it by
determining what kind of information you need and choosing the right tools to get that
information. Then, all you have to do is use those tools to gather that information, analyse
what you’ve found, and, if you’re lucky, you should have an answer.
Relationship to other skills
Research and Critical Thinking are not just academic skills. They are survival skills that all
students will need throughout their studies and their careers. No matter what study or career
path you choose, you will always need the ability to find and analyse information to answer
key questions.
The processes associated with Research and Critical Thinking are not exclusive to this
course. They overlap and connect with virtually everything that students are asked to do
throughout their studies and their eventual careers. Introduction to Research Skills will help
you to learn the basics of research and critical thinking, but you will need to continue
developing and practicing these skills throughout your study and career.
In a sense, this course never ends, because your teachers in subsequent courses will expect
you to demonstrate the skills covered here, at a higher level.
There is a great deal of similarity between the steps taken in the research process and the
different phases of Project Management. Students are encouraged to study both courses
closely to identify theory and technique that can be used to improve both processes.
Course Format
This course consists of 2 classes of 90 minutes each (2 contact hours) per week. The first is a
lecture, held with a large group (2-3 classes) in which students receive instruction on the
research process. The second class is a seminar in which students review, discuss the
homework assignments that are explained in the manual and review each other’s research.
Students are expected to be working on their assignment every week and to bring the work-
in-progress to each seminar for discussion.
The course manual is part workbook and part textbook and is divided into 12 parts (one for
each lesson). Students are not necessarily expected to read the entire manual, but it is
necessary for in-class exercises and homework and should be used for reference during and
after this course. Each lesson is accompanied by reference material and exercises in the
manual and each ends with homework assignments and important reminders. Students are
expected to be aware of their assignments and to know what is expected of them at all times.
Supporting materials
Support materials are posted on BlackBoard. Inside the Introduction to Research Skills folder,
students will find any PowerPoint slides used in class, links to online study materials and
5
other useful items. Students are advised to check this Blackboard course regularly. Students
should also familiarise themselves with The Hague University’s library and library website
which is available in English and Dutch and provides a lot of useful reference information and
tutorials on a wide range of subjects, including effective internet searching and plagiarism.
The address is: www.dehaagsehogeschool.nl/bibliotheek.
Students can also consult the European Studies Writing Database on Blackboard for quick
reference materials.
Attendance
Because of the practical nature of the workshops, and the amount of material that is covered
in class, students are strongly advised to attend every lesson and should be warned that
students who do not attend regularly are responsible for catching up on their own (by reading
this manual and/or studying the review podcasts where possible) and are not entitled to
detailed teacher feedback on their work.
Assessment and Feedback
Students will be assessed based on a Research Assignment on a topic of their own choosing
(within an assigned field). Work starts right away in week 1 and continues throughout the
course with personal support and coaching during seminar. More specific instructions are
provided throughout the manual and the portfolio components and criteria are included
in the final lesson.
Students will receive oral feedback on their participation in class and written feedback on
major assignments, either as written comments or through an assessment checklist (see
Appendix 2) that clearly establishes the course criteria and demonstrates how grades were
computed.
The assignment is explained in more detail below (see The Course Assignment: Description
and Instructions). An overview of the products students have to hand in is in Appendix 2,
followed by an overview of the criteria used to grade the assignment in Appendix 3.
Plagiarism Statement
Plagiarism, the use of ideas and/or text from outside sources without appropriate referencing,
is unacceptable, even if committed accidentally. Students are expected to be familiar with the
Haagse Hogeschool’s plagiarism policy and should be familiar with proper APA referencing.
Any text included in a student’s work that is not in their own words must be properly
referenced using APA style or it qualifies as plagiarism. Writing Assignments will be submitted
through the plagiarism scanner Ephorus.
Plagiarised or copied work will not be marked and no re-sits will be offered in cases of
plagiarism. Assignments that included plagiarised text will be submitted to the exam
board.
6
Your Assignment: Description and
Instructions
Stating from Week 1, students will begin working on a large-scale individual research
assignment. Students can choose their own topic within assigned fields (see below) and will
have to come up with a central research question and then plan and conduct research to
answer that question.
The exact nature of the research will depend on the topic but all students are expected to
conduct extensive desk research, using different types of sources.
The end result of this research will be a formal, APA-style report, roughly 6-8 pages in length
(excluding references and appendices) that documents the research and provides
conclusions to the central research questions, as well as recommendations, where
appropriate.
An overview of the components of the assignment can be found in Appendix 2 and detailed
assessment criteria have been included in Appendix 3. Students are advised to study both
before handing in their work.
If possible, example reports will be posted on Blackboard for reference.
Topics for European Studies
European Studies students must choose a topic that relates to the political, economic and,
especially, cultural dimensions of Europe and are strongly encouraged to consider using a
topic they are studying in other courses (like the Cultural Dimension of Europe). Students will
need a clear research question about a political, cultural or economic issue.
The topic should have an international focus and the research should incorporate different
international perspectives.
There are limitless possibilities and it is very important that students choose a topic that
interests them. Below are some examples of suitable topics. +
Suitable topic areas include:
European Culture – issues related to national identity and/or European identity, localization
vs. globalization, media, art, philosophy, the role of enlightenment thinking in contemporary
Europe and other issues.
Political/economic/social issues – issues related to political philosophies and parties, national

and European decision-making, (international) business and regulation, distribution of wealth,
poverty, racism, etc. You can, for example, imagine that you are a junior policy advisor to a
national, local or international government agency and want to get an issue on the agenda.
Here are some more specific examples of suitable research topics:
1. The cultural differences between neighboring countries (such as the Netherlands and
Belgium or Finland and Russia) and how those differences affect relations between them.
2. The influence of globalization (in terms of business, media or other factors) on

local/national cultures.
st
3. The causes and effects of the rise of populism in the early 21 century in Europe/one or
two European countries.
4. The impact of new media (Facebook, Twitter, Tumblr and other Web 2.0 applications) on
politics/economics/culture in Europe/ one or two European countries.
5. The impact of American culture/politics/business on Europe (please note that you would
have to be very specific about what American influences you would look at and what types of
effects). Or, the impact of European culture/business/politics on America or the world.
6. The political/cultural/economic effects of immigration on The Netherlands/Any other

European country (please note that you would have to be specific about what type of
immigration, etc.)
Re-sits
Students will have two opportunities to hand in the assignment. A re-sit will be announced
after the first attempt is graded. Students must keep the following in mind:
• A student with a failing grade can do a re-sit by improving their original assignment
but can only earn a maximum grade of 6
• A student with a failing grade can also choose to start the assignment over with a with
a new topic to earn a higher grade
• Students with a passing grade who want to do a re-sit to earn a higher mark may also
do so but only if they start the assignment over with a with a new topic.
8
Week 1
Lecture: Introducing Research

In this class:
• Your teacher will introduce the course and assignment
• You will discuss the basics of doing academic research
• You will begin familiarising yourself with information sources and tools
Lecture Outline & Reference Material
Here is a summary of some of the most important content covered in the lecture, for your
reference. You can use this material, together with any notes you have taken and/or any
available PowerPoint slides, to guide you through your research and for future reference.
1. Course Introduction
Your teacher will briefly introduce the course and tell you what to expect in the next 6 weeks,
but be advised that you are still expected to study the information included in this reader
closely, as there may not be time to discuss every detail in class. Students are expected to be
familiar with the course format and content and to know what is expected of them at all times.
If time allows, your teacher may discuss the specifics of the Course Assignment.
2. Defining Research
Believe it or not, the process of conducting research is simple and involves a handful of basic
steps that you always use when you want to answer a question, whether you want to know
about European Union policies or what’s on television tonight.
Step 1. Figure out what you want to know and why
Research projects always start with a CENTRAL QUESTION that the researchers want to
answer, a basic statement of what it is that you want to know. Most researchers decide on a
central question after conducting some preliminary research to see what is already known
about the topic, so they can come up with a new question that hasn’t been answered yet. This
central question is usually divided into several sub-questions and the purpose of your
research is to answer these questions for some specific purpose.
No one does research for its own sake. You need a reason or a PURPOSE. There has to be
something that you want to know or understand in order to achieve a specific goal of some
kind. There are many different reasons why you might choose to do research (academic and
professional) and it is essential that your reasons be clear before you start.
Step 2. Select research methods
Before you hit the library or boot up your computer to get to work, you need to take some time
to determine what research methods you should use in order to answer your central question
and sub-questions. There are several methods at your disposal and, just as in the above
example, you need to select the best method for finding the information you need, bearing in
mind that every method has advantages and disadvantages. It starts with figuring out what
information you need to answer your question and considering where you need to go to get it.
Step 3. Conduct your research by applying the chosen Methods
Once you know how you want to find your information, get to it. Hit the library, go out into the
field and surf the Internet until you find what you’re looking for. Get some facts.
Step 4. Analyse Your Data
Once you’ve gathered up all the facts, figures and opinions you need, take some time to
analyse them. Read the relevant documents and evaluate them. Make sure they’re
trustworthy, relevant and that their meaning is clear. Measure the information against your
research question and sub-questions to see if the answers appear. Compare what you’ve
found to whatever criteria you might have for useful information and get ready for…
Step 5. Make Conclusions (and Possibly Recommendations)
Now that you’ve analysed your results, you’re ready to make some decisions. You should be
able to answer all your questions (if you can’t, perhaps you didn’t choose the right research
methods) and reach some decisions. You should, at this point, feel like you know what you
needed (or wanted) to know.
Often, it doesn’t stop here, because often, research is meant to provide meaningful
RECOMMENDATIONS – advice about what should be done now that we know what we
know. They are similar to, but different from conclusions. A conclusion is what you know, now
that you’ve done your research. A recommendation is what you think we should do about it.
3. Reasons for Researching
As stated above, there is no point in doing research without some kind of clear purpose or
goal. There has to be a reason to go out in search of information. Here are just a few
examples:
At school
1. Essays and Presentations

Students are often asked to write academic essays in order to demonstrate their
understanding of a specific topic and their ability to write or speak about it in a professional
way. You might be asked to write an essay answering a specific question like: “Explain the
causes of World War 1,” for example, or “Compare and Contrast the immigration policies of
two EU countries” (the possibilities are endless). Sometimes you might be free to choose your
own topic, which can present an extra challenge.
2. Research Reports
University students often have to produce research reports on assigned topics, often with the
goal of producing useful recommendations. You could, for example, be asked to research the
implementation of a government policy or business plan in order to see if it works as well as it
should and make recommendations for how to improve things. The possibilities here are also
endless.
At Work
1. Business reports
Report writing is a big part of the job for many professionals. Wherever you work you may be
asked to ‘look into’ something and produce a report, often with the goal of making important
recommendations on everything from policy changes to budget cuts. In each case, you would
need to gather up enough information in order to give the right advice to your boss.
2. Policy Documents, Proposals and Plans

There is a wide range of documents associated with the decision-making processes in any
working environment (government, non-governmental organisations, businesses, etc.) They
range from Position Papers (internal documents in which an organisation lays out its opinion
on an issue), Problem Analyses (in which problems are researched in the hopes of finding
solutions) and Proposals, in which authors try to make a persuasive argument on behalf of an
10
idea. In order to write any of these documents, you would need to do some research in order
to come to the right conclusions.
4. Research Papers in APA Style
All researchers document their findings in scientific reports that follow specific standards for
structure and style. The Hague University of Applied Sciences uses APA Style – a standard
for report writing in social sciences that was invented by the American Psychological Society.
APA style, which includes a standard format for a research report as well as rules for using
information from sources, will be covered in detail during this course, but here is a brief
introduction.
The format should not surprise you because it follows the same steps as the research
process.
An APA Style report includes:
1. An Introduction – in which the topic is introduced and the central research

question is explained.
2. Methodology – an explanation of what research methods were chosen and
why.
3. Results – a summary of the information that was discovered in the research.
4. Discussion – a brief discussion of any particularly interesting findings.
5. Conclusion(s) – the answer to the central question, based on the information
reported and analysed above.
6. Recommendations – Advice about what could be done next, to follow up on the
research or to act on its conclusions.
There is some additional content. An APA report always has a title page, an Executive
Summary, a Table of Contents, a List of References and Appendices, but we’ll be dealing
with those later.
Please remember that research is a process, not a product. Research reports don’t exist
for their own sake. They exist to document the steps you took in conducting your research –
the same way you documented science experiments or ‘showed your work’ on math tests in
high school. That's why the trick to writing research reports is to write as you go, not at the
last minute. Once you have a research plan, it’s easy to write the Introduction and
Methodology. Once you start researching, you can start summarising your results and writing
your List of References. Once your results are in, you can analyse them quickly, draw a
conclusion and make recommendations. Then, the last thing you do is write your summary.
Summarising
Once the actual research is done and the report is finished, it’s time to write the Executive
summary, which goes on the first page but is always the last thing you write.
5. Getting Started: Creating a central question
A lot of students say the hardest part of any research project, whether it’s for an essay, a
thesis or any other purpose, is getting started. Sometimes it’s hard to understand an assigned
topic, and it’s often even harder to come up with your own topic if you’re given a choice.
The first step, as you now know, is to establish a clear, specific central question that
summarises what you want to know.
If, for example, you were asked to write an essay explaining the causes of World War 1, you
might start with a central question like “What were the causes of World War 1?” It’s that
simple.
11
It starts with a topic
Research usually starts with a broad, general topic. Often this topic is assigned to you by your
school or employer, but often you are free to choose.
PLEASE NOTE THE FOLLOWING: If you are allowed to choose your own topic, do yourself a
favour and CHOOSE A TOPIC YOU ARE PERSONALLY INTERESTED IN. Never choose a
topic because you think someone else will find it interesting or because you think it will be
considered ‘academic.’ Research and writing are complex processes and if you’re working on
a topic you aren’t interested in, the risk of getting bored or doing sloppy work increases.
Refining topics
Once you have a general topic, the trick is to refine it so that it is not too broad or too narrow.
If it’s too broad or general, you will have a hard time sifting through all the information you find
and your research won’t have a clear, specific focus. But, if it’s too specific or narrow, you
won’t be able to find enough information and will have a hard time making solid conclusions.
General topics like ‘Euthanasia’ or ‘Immigration Policy’ are too broad. You need to narrow
them down and focus on specifics like “Euthanasia policies in the Netherlands (or another
country) or ‘Immigration policies of the European Union.”
This will bring you a step closer to your central research question.
6. Getting Ready to do Research
Once you have a clear topic that isn’t too broad or too narrow, you’re ready to start working
on your central question. To do that, it’s a good idea to do some preliminary research to
define keywords and stakeholders.
Keywords
Before you get started, you will want to analyse your topic a little bit further in order to make a
list of any important TERMS or KEYWORDS that could be relevant to your research. You will
need to use these as SEARCH terms when using search engines and databases and they will
help you to make your research measurable.
Define any important concepts. If you’re researching Euthanasia policy, you need to define
what euthanasia is (doctor-assisted suicide). Keywords might include “physicians, suicide,
ethics (ideas about what’s right and wrong), etc.
Stakeholders
You should also try to find out who the stakeholders are. A STAKEHOLDER is any person or
organisation related to your issue (an expert in the field, an organisation active in that area, a
government, etc.). If you know who the stakeholders are, you can look them up to get even
more information about the issue – most, if not all will have websites and publications that
could really help you.
In the euthanasia example, stakeholders include terminally ill patients, doctors, medical
boards, and the governments that make laws about life and death.
Formulating the Research Question
Once you have an overview of the topic, the related keywords and the stakeholders involved,
it’s relatively easy to come up with a central research question. All you have to do is decide
what it is that you want to know about your topic.
If, for example, we started with the above-mentioned topic of “Euthanasia policy in the
Netherlands,” we could come up with several questions depending on what we want to know,
such as:
12
1. What is the Dutch policy towards euthanasia?
2. How does the Dutch euthanasia policy work?
3. How do Dutch people feel about the euthanasia policy?
4. How does the rest of the worldview the Dutch euthanasia policy?
5. How does Dutch euthanasia policy compare with the policies of other nations?
Now you’re ready to do some real research.
Be SMART
The central question is the most important step towards doing solid research and a good
question will make the work easy, because it will allow you to focus on what you NEED to
KNOW and keep you from getting lost in the details. A good suggestion is to make sure you
have a SMART Question (one that is Specific, Measurable, Assignable, Realistic and Time-
related).
You’ll be learning about SMART goals in other courses, so to keep a long story short, you
want to make sure your research question is concrete and specific, that it can be answered
with measurable facts, figures and findings and that it can be done within a realistic amount of
time with the resources available.
7. Introducing Desk Research
In a practical sense, the term Desk Research refers to research you can do sitting down.
More specifically, it refers to methods of gathering up information from existing sources. The
most common forms of desk research for most students are LITERATURE REVIEW
(consulting books, magazines, journals and other publications) and INTERNET SEARCHING
(using Internet sites and databases to access information) or a combination of the two.
Most research projects start with desk research. If you want to do research on a specific topic,
it makes sense to first study all the information that is already available. Otherwise, you might
find yourself working really hard to answer a question that has already been answered.
In Applied Research, it is common to do a lot of desk research to find a ‘theoretical

framework’ for other research – to study all the important theory related to a topic and then to
test that theory out in the field
Whether you’re doing your desk research in a LBRARY or on your personal computer, you
often fun into the same problem: NOT KNOWING WHERE TO START.
Tips for getting started with Desk Research
1. Use your notes
Having a list of keywords and important stakeholders can really help you get off to a good
start. You can look up keywords, either online or in a reference book, and look up the
stakeholders, many of which will have websites (especially if they are companies,
governments or organisations) or see if those stakeholders have been in the news.
2. Consult general reference sources first.
Especially if you’re researching a topic you don’t know much about, it’s a good idea to consult
some general information sources.
Encyclopaedias and dictionaries are a good place to start, because they typically offer short
but thorough explanations of things that you can use to understand big concepts. They might
also provide you with more keywords to search for or give you an idea of how one topic
relates to another.
Here are some general sources worth considering:
13
1. Encyclopaedia Britannica (www.britannica.com)
Encyclopaedia Britannica is one of the most widely read

Encyclopaedia’s in the world and can be found in many libraries. It’s
website also offers a wide range of content.
2. About.com (www.about.com)
About.com is a popular North-American website that provides

reference information on a wide range of topics. You can search
for topics by keyword or you can try typing a full question into the
search field.
What about Wikipedia?
By now you have no doubt been told that you should not use
Wikipedia, the free, Internet-based encyclopaedia that has become
one of the world’s most popular websites, as a serious research
tool. This is because the collaborative nature of Wikipedia (anyone
can add or edit articles) raises questions about credibility.
However, this is not completely true.
It is true that YOU CANNOT TRUST EVERYTHING YOU FIND

ON WIKIPEDIA, because not all the information on the website
is verified or confirmed and because the open nature of the
site leaves it open to abuse by people who might want to
change articles to suit their own purposes. However, it should
be noted that Wikipedia’s does have quality control procedures
and improvements are being made every day.
Either way, you should know that IT IS POSSIBLE TO USE

WIKIPEDIA RESPONSIBLY and it may even be a good idea,
especially when you’re just getting started.
How to use Wikipedia in a responsible and academically acceptable way
1. You should only look at Wikipedia to get general information on a specific subject.
Use free articles to get a general sense of what you should be looking for.
2. Look for references and links.
Many Wikipedia articles include references to where the information on the site can be found.
Links are often included as well. If the article you’re reading does not have any references or
links, you probably should not trust it.
3. You should never, under any circumstances, assume that the information you find is
accurate unless you can confirm it somewhere else.
You can do this by clicking on the links Wikipedia offers and checking references, or by
selecting the bits of information you found and searching for them elsewhere on the Internet.
If you can’t verify something, you can’t use it.
4. Use Wikipedia to find keywords and follow through on links.
Reading a general article on your topic may help you to find more keywords and/or
stakeholders worth looking at. Follow up and get more information about them. At the same
time, you should follow through on any of the links Wikipedia provides.
14
5. Look at the DISCUSSION page
Every Wikipedia article has a link at the top of the page (see
example on the right) to a ‘Discussion’ page where editors discuss
articles and their content, focusing on credibility and quality. You
can learn a lot by looking there.
Example of Links
At the bottom of most Wikipedia Articles (like this article on
euthanasia) you’ll find a section called ‘See also,’
recommending other topics you might want to look up.
You’ll also find a ‘References’ section that might include useful

links to other sources that might be more trustworthy and could
confirm what you’ve found in the Wikipedia article.
Many articles even recommend ‘Further Reading’ to point your

research in the right direction, or have a section of ‘External
Links’ that you may want to follow up on.
Get ‘Googling’
Once you’ve got some general information on your topic, you will probably have enough
keywords to start doing some more specific searches either in libraries or online. For most
students, this process starts with Google, the most popular search engine on the Internet.
Google is a very useful tool, but it is not the only tool and it has risks.
Keep in Mind…
Many students are unaware of two simple facts about Google that are important to know:
1. Google is not the Information business. It’s in the Advertising business.
2. Google is not capable of finding most of the content on the Internet.
Risks associated with search engines
If you were to search for the same keywords on

different search engines (Google, Yahoo, etc.) you
would notice differences in the results you find.
Links will not be listed in the same order and in
some cases, one search engine may have links
that others don’t.
This is because each search engine uses different

technology to find information. Google, for
example, typically puts the most popular links at
the top of its list. Please note that the most popular
links may not always be the best.
You should also keep in mind that Google and other search engines earn their money through
advertising. This and other factors (your geographic location and user profile, for example)
can affect your search results in ways you might not realise.
15
Working around Search Engine Limitations
There are a couple of different things you can do to work around the limitations of commercial
search engines.
1. Use Meta-Search Engines
A META-SEARCH ENGINE is a search engine that searches other search engines. You can
enter keywords or questions just as you would with Google or Yahoo, and the Meta-Search
Engine will search Google and Yahoo (or even Microsoft’s Bing) for you and provide a wider
variety of results.
There are several Meta-Search Engines out there, including Metacrawler

(www.metacrawler.com). Dogpile (www.dogpile.com) and Surfwax (www.surfwax.com). You
should play around with each to see which one works best for your research topic.
2. Consult the media
Virtually all newspapers and magazines now have websites that have searchable archives
and often, a lot of the content is free. Check to see if your topic or any stakeholders have
been in the news lately.
3. Use Search Engines more wisely
Even though there are countless other (meta) search engines, it’s likely you will find yourself
using Google at some point. There are several tricks worth trying to get more out of Google.
You can search for keywords, or even enter a complete question, or you can use MODIFIERS
– symbols and words that refine your search a little. On the next page you’ll see a list of
1
modifiers made available by Google . Try some of these out the next time you’re searching
for information.
These modifiers are similar to what are called BOOLEAN OPERATORS, tools you can use to
narrow or widen your search. These are particularly useful when doing an advanced search
or using a database. Boolean operators fall into one of three categories: AND, OR, and NOT.
You can narrow your search by using AND to search for topics that are connected (for
example: European Union AND Constitution) or you can broaden your search by using OR
(for example: European Parliament OR European Commission) or you can narrow a search
by using the word NOT to eliminate topics you don’t want to look at (for example: European
Union NOT Commission). Look at the Google Tips on the next page for more examples and
consult the library’s online trainings for more detail.

1
Retrieved on 21 April, 2011 from Google: www.google.com/librariancenter/downloads/Tips_Tricks_85x11.pdf
16
17
8. Databases: Entering the ‘Invisible Web’
As you already know, DESK RESEARCH is any research you can do sitting down – visiting a
Library, reading books and articles and Internet research. It is the main method you’ll be using
in this course, but there’s more to it than just sitting down and Googling away.
Almost all research starts with some (preliminary) desk research – you need to know what’s
already out there before you narrow down your question – and some research is exclusively
based on information gathered through desk research. That may seem easy, but it’s more
complicated than you might realise, because each source has to be carefully chosen, for a
specific reason and because there is so much information available, it is often easy to get
lost.
We have already discussed search engines and general information sites, but now we have
to dive deeper into desk research tools by talking about databases.
If you’ve done your library training,

you will probably be aware that The
Hague University of Applied Sciences
Library provides you with access to a
wide variety of digital databases.
These databases are a hugely
important research tool, because they
give you access to the ‘Invisible Web.’
Google and other search engines are
useful tools, but what you might not realise is that they can only scan about 20% of all the
information on the Internet. That’s just the tip of the iceberg. Eighty per cent of the information
is kept in locked databases that have to be accessed by authorised users. THUAS provides
access to dozens of useful databases through its Library Website. To see what’s available,
simply go to the Library from the Student Portal (in the GO TO) box. Then, go to the DIGITAL
LIBRARY section of the Library Site, then DATABASES, where you’ll find a list of Databases
and a link to the TOP 10 DATABASES.
It might seem a little overwhelming at first,

because there is so much to choose from,
but it’s easy to narrow things down. You
should take some time to explore all of the
available databases, but while you’re getting
started, you might want to focus on the two
most commonly used databases for
European Studies Students: Academic
Search Premier and the Lexis Nexis, both
of which are in the Top 10.
The Lexis Nexis searches through news archives while Academic Search Premier gives you
access to thousands of research articles from credible, peer-reviewed sources.
Each database has it’s own search features, and the online library training (BIBIT) will help
you get to know them, but the general principles of Internet searching (including Boolean
searches with and, or and not) also apply here.
If time allows in class, you may be able to take a quick look at these databases together.
8. Reading Articles & Information Sources
One of the challenges researchers face is trying to sort through information quickly. Enter any
query into Google, or into an academic database and you’ll find dozens, if not hundreds or
thousands of articles, many of which are lengthy and full of intimidating words and/or
statistics. This can cause an inexperienced researcher to panic, because they assume they
have to understand everything. The good news is that you don’t need to read or
understand every single detail. If you understand how articles are usually structured, you
can learn to study them quickly and get the information you need.
Encyclopaedias and Wikis
General information sources like Wikipedia are a

great place to start because they are usually short
and easy to read, usually because the follow the
same structure. An encyclopaedia or wiki article
always starts with a summary of all the most
important information about the topic, followed by
specific details that are organised and labelled so
that you know where all of the information is.
Most informative websites follow the same format,

allowing you to use the various sub-headings to
skip to the information you need quickly.
News Articles
News articles are a good source of general information

about a topic, especially if it is a current affair. Like most
wikis or websites, they are easy to read because they
always follow a set structure, often referred to as the
2
Inverted Pyramid because it puts all the important
information at the very top in a lead paragraph that gives
you the who / what / when / where / why / how of the story.
Most of the time, the lead paragraph is written in such a
way that you don’t have to read the rest of the news article
unless you really want to (and most newspaper readers do
not).

2
Image retrieved on 3 April, 2014 from Wikimedia Commons:
http://commons.wikimedia.org/wiki/File:Inverted_pyramid_in_comprehensive_form.jpg
19
If you’re using news articles for your research, use the lead paragraph to find the key
information and then zoom in on the body to get any details you might need.
Academic Journal Articles
News articles, like wikis and encyclopaedias, are easy to read, but other research sources are
more complicated. In order to write quality academic research, you will need to read other
quality academic research. That means going to databases and pulling out (scientific)
research articles that appear in academic journals. Journals are usually reliable resources
because they require peer review before publication: any article that is going to be published
has to be studied by credible experts who can evaluate it before it goes to print. That provides
an extra layer of credibility and that’s why it’s essential that you learn how to find and read
them.
At first, many journal articles appear intimidating. The vocabulary is often advanced and
sometimes the charts, graphs and statistics scare us off, but there’s no need for that if you
know what to do.
It starts with knowing the structure of an academic article, which is very similar (or sometimes
identical) to the type of research reports you will be writing, though sometimes the format and
the titles are slightly different. Most journal articles consist of the following:
An Abstract
An abstract is the same as an Executive Summary. It’s a short summary of the entire article
that explains the purpose of the article and describes the research question before
summarising the research and its conclusions.
Introduction
Like any introduction to any research report, this will introduce the topic and establish the
central question. It will usually include background on why the research was undertaken and
other relevant information.
Literature Review
A lot of research papers follow their introduction with a ‘Literature Review.’ This is where the
author summarises and comments on previous research that has been done on their topic,
looking at major theories or explanations that may help them narrow their research down. In
some articles, it follows the introduction and in others, it follows the Methodology.
Methodology
This is where the author explains the research methods they used to answer their questions
and explains why those methods were chosen. Sometimes it comes right after the
introduction, but some authors put their literature review first.
Results
This is where the authors summarise their findings, often with the help of charts and graphs
and statistics. Those visuals may seem confusing at first, but the trick is to remember that all
of the information in those graphs is also described in the text, so if you read this section
carefully, it gets easier.
Discussion
Just like in your research report, the Discussion area is where researchers make observations
about the information they researched. They point out significant or surprising findings and
sometimes comment on the research itself – noting whether or not it was successful. It’s
essentially a bridge that helps the researcher go from their results, to their conclusion (though
in some cases, the discussion and conclusion are put together in one section).
20
Conclusion and Recommendations
If they haven’t already done this in the discussion, the author will state the answer to their
research question here, and may also make some recommendations about what should
happen next, or about what could be done with the information.
Basically, most journal articles are like APA Style reports, which is good, because it makes
reading them easy. To read and understand a scientific article, take the following steps.
1. Skim the Article
Go straight to where all the main information is: the Abstract. Read that and then quickly look
at the Introduction and Conclusion – and, where appropriate, the Discussion – so that you can
find the main points and write them down.
2. Check the Details
You don’t have to read every word, but you do need to make sure you understand the main
details and make sure the article is trustworthy. Take a look at the Methodology, so you know
how they did their research. Then go through the results and get the main points (Remember
that the charts, tables and graphs can help to break down information visually, but if you find
them confusing, you should read the text carefully, as it will help you understand them). Write
down anything you think is significant.
Homework for Seminar 1

Students are expected to the do the following assignments BEFORE attending their seminar.
A. Find a specific article
In your next class, you will review what you’ve learned about finding and reading academic
literature. In order to practice, you need to use the library’s website, access the online
databases and find the following article, which you will need to bring to class for discussion:
The article is called “Does Satisfaction with Democracy Really Increase Happiness? Direct
Democracy and Individual Satisfaction in Switzerland” and was written by Isabelle
Stadelmann-Steffen and Adrian Vatter. It appeared in an issue of a journal called Political
Behaviour in 2012. Find it and read it.
B. Think of one or two topics for your Research Assignment and do some Preliminary
Research using what you’ve learned.
Soon, you will start work on your own research, but to do that, you need a topic. Your
homework is to find one or two topics you might be interested in researching and do some
general, introductory research.
Please follow these steps:
1. Read the description of the course assignment and study the guidelines and
example topics closely.
2. Come up with one or two topics you would like to research.
3. Come up with an idea for your central research question.
4. Do some preliminary desk research using at least two sources that are not
Google (a meta search engine, for example).
5. Make a brief list of any important stakeholders and/or keywords you can identify.
21
You will need to bring everything you find to Seminar 2 and be prepared to discuss the
following:
1. Any differences in search results between different sources.

2. The easy and hard parts of the process.
3. Whether or not you trust the results you found.
This assignment is necessary for your in-class activities in Seminar 2. DO NOT COME TO
CLASS UNPREPARED.
22
Seminar 1
In this class:
• You will review the homework
• You will discuss possible research topics and preliminary research findings
• You will discuss desk research and academic databases
A. Review Lecture
At the start of class you will have time to discuss the lecture and to ask questions if
necessary.

B. Review the Assigned Article
For homework, you were asked to track down an article about the relationship between
Democracy and Happiness. In class, you will discuss it, keeping in mind what you learned in
this week’s lecture about reading research articles.
Take a few minutes to study the article and then prepare to discuss it by answering the
following questions:
1. Where and how did you find the article?
2. What is the main point/purpose of the article and how can you tell?
3. Does the article contain a central research question and if so, what was it?
4. What research methods did the authors use?
5. What conclusion did they reach and how did they reach it?
6. Does the article seem like a trustworthy information source? State why or why
not.

C. Presenting and Discussing your Research Topics
All students should have prepared a few possible research topics and done some preliminary
research for discussion. Be prepared to explain your topic and present your preliminary
findings. Use the following questions to guide you:
1. What topic(s) are you considering?
2. What keywords are related to the topic?
3. What stakeholders are involved?
4. What information have you found so far and how did you find it?
5. What would be your central research question?
Your teacher will lead a discussion about each topic and question, and you will have a chance
to comment on one another’s work. Be ready to discuss each topic, using the following
questions to structure the discussion:
1. Is the topic clear and manageable (not too broad or too narrow)?
2. What suggestions do you have for improvement?

Students are expected to the do the following BEFORE attending class in Week 2.
1. Turn your research topic into a research question.
Go over your research topic and, if necessary, revise it, or come up with a new one. Read on
your topic. Using the model provided in the lecture, identify the main issues related to your
topic. Select one aspect you want to focus on and formulate a research question.
24
Week 2
Lecture: Making a Research Plan

In this class:
• You will discuss how to make an effective research plan
A. Introducing Research Plans
All research should start with a proper plan in which the researcher states a clear question
and lays out a plan for finding the answer. A good plan can make the difference between
success and failure, because a good plan will keep you focused on what you need to
know, and protect you from going off topic or getting lost. It’s just like a road map.
The funny thing about making a research plan is that you should already know how to make
one. It’s a process you’ve been practicing since the start of this course (and even before).
You start by determining what you want to know (your research question) and why you want
to know it (your research objective). You determine the best way to find the information you
want (your research methods) and you start compiling a list of keywords and relevant
stakeholders. Then, all you have to do is think about some practical matters (resources,
scheduling and – if you’re working with other people – a division of tasks.
It really is that simple. But there are a few things you’ll want to remember:
Two things you need to remember about Research Plans
1. A research plan is a means, not an end
Students often run into trouble when they start thinking of a research plan as an individual
assignment that has to be handed in for a grade instead of seeing it for what it is: a necessary
step to help you get started with your research. RESEARCH PLANS ARE NEVER WRITTEN
FOR THEIR OWN SAKE.
2. A research plan is not set in stone
A good researcher always starts with a good plan and tries to stick to it, but is also flexible
enough to accept that their plan is not permanent. It can and most likely will change. Most
researchers do much of their desk research early on, in part because it is easy and does not
require many resources or other people, and often that desk research will point them in a new
direction. They might find new research questions or keywords. They might find new angles
they haven’t thought of or, in some situations, they might find that someone else has already
done similar research, which would require them to adjust their plan or, in extreme cases, to
think of a new one.
Research is not a static process. Things change, and as you go further into a research
process and get better informed about your topic, you might find that your research needs will
change. That is okay. It’s part of the process.
Keep in Mind
A research plan should be short and simple and not overly detailed, but it should be clear and
give a complete picture of what you’re doing and why you’re doing it. Remember this rule of
thumb: YOUR RESEARCH PLAN SHOULD BE SO CLEAR THAT SOMEONE ELSE COULD
USE IT TO DO THE SAME RESEARCH AND REACH THE SAME CONCLUSIONS. That,
after all, is how the peer review process works in the ‘real world.’
Planning, Step by Step
As stated above, creating a research plan is a relatively simple process that involves going
through a couple of basic steps.
1. Define your topic and research question
What do you want to find out/know? Choose & explore your topic, create an overview of
keywords and stakeholders, do preliminary desk research and create a central question.
Divide your central question into sub-questions.
2. Figure out what (types of) information you need
What specific information do you need to know order to answer your sub-questions and/or
the central question?
3. Decide on your methods and/or sources
Once you know what information you need, it’s easy to choose your research methods
and develop a strategy. Figure out what type of research you’re going to do and be
specific.
Don’t write a research plan that says “I will do desk research.” Explain what types of
sources you plan on using, how you plan on using them and why you think they’re the
right choice.
4. Plan your time
Research is a time-consuming process. It takes time to sift through library books and
search the Invisible Web, especially given how thoroughly you need to check the
credibility of sources, and it can take even longer to do other types of research because
there are other variables involved – the availability of people for interviews, for example.
Consult your Project Management course for tips on planning your time.
D. Selecting Research Methods
There is a lot more to research than libraries and Internet sites and there are a great many
tried and true methods of gathering information. To be a skilled researcher, which is
necessary to becoming a successful student and becoming competent in your career, you
need to be skilled in using a variety of methods, such as case studies, interviews, focus
groups and questionnaires.
Quantitative vs. Qualitative
Most research methods fall into one of two categories. They are either Quantitative or
Qualitative.
26
QUANTITATIVE methods are methods that will get you results that can be expressed in
quantities (numbers). Quantities can be added, subtracted, averaged, and put into charts and
graphs.
Statistical research is usually quantitative in nature. You gather up some numbers and
analyse them, looking for meaning. Questionnaires are also quantitative. They enable you to
get a large number of people to answer a set of questions. These questions usually have a
limited number of answers (yes, no, maybe, etc.) that can be turned into numbers (1, 2, 3,
etc.) and analysed like statistics.
Quantitative methods have several advantages. They are often (relatively) cheap and quick –
it's easier to survey 1,000 people than it is to interview them face-to-face. However, there are
also disadvantages. Questionnaires (structured lists of questions that can be filled in
anonymously), for example, can provide a broad amount of information, but they rarely allow
for researchers to get any depth.
QUALITATIVE METHODS are methods that result in information that cannot be quantified as
numbers, facts and figures. This applies to most research interviews (in-depth interviews in
which detailed questions are asked of an expert or other stakeholder) as well as some case
studies (detailed study of specific examples) and a lot of desk research. Qualitative methods
have the advantage of offering depth – in an interview you can ask more detailed questions
and get more detailed answers and insight. There are also disadvantages. Qualitative
research is time consuming and there are limits to what you can accomplish – you simply will
not have time to do a large number of interviews and will have to base your research on
smaller number of sources, which has risks.
The right tool for the right job
It’s very important when selecting research methods, that you choose the right tool for the
right job. You do this by thinking critically about what you want to know and making sure that
you choose the right method to get that information. If what you’re looking for is an expert
explanation of a complex issue, desk research and interviews are a good way to go. If you
want an idea of what the general public thinks about that issue, desk research and
questionnaires (polls) are a better technique.
Different Approaches in Desk Research
Desk Research is the only method required for this course and that may seem simple, but
there is still more planning involved. It isn’t enough to write a research plan that says “I will do
some desk research.” You have to be specific about what types of information you’re looking
for and where you hope to get it.
If you’re doing research about crime, for example, you will probably need some crime
statistics, and you will need to find out where to get them. Depending on the specific subject,
you may also need general media coverage and expert opinion, both of which can be found
on the Internet or in the databases you have access to.
You need to be as specific as possible about what kind of information you need. This will help
you to focus and will protect you from going off topic or working too hard.
Criteria for results
When selecting methods, think about the kinds of results you are going to get. In order for
your research to be useful, your results should meet certain criteria: They need to be
RELIABLE, VALID and GENERALISEABLE.
27
1. Reliability
Reliability refers to whether or not your research will result in consistent findings. If
someone else where to follow your research plan…would they reach the same
results?
Threats your reliability include human error and bias, either on the part of the
researcher or on the part of the sources consulted (especially when it comes to
people being surveyed or interviewed). If you want reliable data, you will need to be
aware of this.
2. Validity
Validity refers to whether or not things are what they appear to be. Are the
relationships between different factors really what you think they are? This is
especially important when looking for the causes of a situation or problem.
Remember – just because one thing follows another does not mean you can assume
that one thing caused the other.
3. Generaliseability
Gerneraliseability refers to whether or not generalisations could be made based on

your results. Would you get the same findings in another setting? Would your
methods result in consistent information?
Each of these criteria are things to think about when deciding what methods to use. For each
method you’re considering using, ask yourself the following questions:
1. What do I hope to find out using this method?

2. Why should I choose this method over another?
3. Will this method provide reliable results?
4. If I were to do this over again, could I expect the same or similar results?
5. Will this method provide accurate information?
6. Will this method lead to results that could be applied to other situations?
7. What are the risks associated with using this method?
8. How can those risks be overcome?
E. A Basic Example
An Example:
Imagine you’re at home with nothing to do and you want to know if anything good is going to
be on Television because, if there isn’t, you might have to rent a movie or go out.
The first thing you do is decide how you’re going to get the information you need. You
probably have different options, such as:
1. Consulting a newspaper, as newspapers usually have a TV guide

2. Consulting a TV Guide, a magazine dedicated to providing an overview of what’s
on television within a given period of time
3. Consulting electronic sources (the text feature many European televisions have
or the electronic programming guides that many digital cable services provide.
4. Consulting the Internet.
Obviously, not all methods are necessary and not all methods are appropriate to your
situation. What if you don’t have a newspaper subscription or Internet access? What if you
don’t have digital cable? Some sources might also be better than others. The newspaper
might not list all of the channels you have on your TV and some TV guides give more detailed
descriptions of programmes, for example.
28
In order to do this research, you have to select the method you think would be MOST
USEFUL.
Once you’ve selected an information source, you go to that source (pick up the paper or TV
guide, turn on the TV and press buttons or run over to the computer) and start looking for the
specific information you need.
It’s likely that you’ll find, in any of the above sources, more information than you really need.
You don’t need a complete overview of the whole week. You just want to know what’s on
tonight, so you NARROW DOWN YOUR SEARCH by turning to the page (real or digital) that
has the TV listings for the specific date and time you’re concerned with. There, you’ll find a list
of all the programmes broadcast at that time. You are free to go through the list, read the titles
and ANALYSE THE INFORMATION you’ve found. Chances are that you might see the name
of a programme you like, but you might not. Either way, what you’re doing is comparing the
information you found with your own personal preferences (what you like to watch on TV) and
you will be able to COME TO A CONCLUSION by deciding whether or not you’re interested in
any of the programmes on TV and whether or not you want to stay home and watch TV or do
something else.
Congratulations, you’ve just completed a research project. You started with a general
question (what’s on TV tonight?), selected a method of gathering the information, looked up
the information and analysed it so that you could make a decision.
F. From Rough Idea to Research Plan – a real European Studies example
Once upon a time, a group of European Studies students was having difficulty figuring out
what to do with their research assignment. They wanted to do something about on-going
developments in Afghanistan, a country that receives a lot of financial and other aid from
Europe and from the Netherlands, but they were having trouble coming up with a central
question and a research plan.
Because development aid was a hot political topic at the time, these students wanted to find
out if the development aid that the Netherlands gives to Afghanistan was working. After some
discussion, a central question was formed:
“How effective is Dutch development aid to Afghanistan?”
This question could also be rephrased as:
“Is Dutch development aid to Afghanistan effective?”
Once they knew their overall question, things got a little easier. All they had to do was break
that question down into a handful of sub-questions and then figure out how to find the
information they needed to answer those questions.
In this example, there are really only three main sub-questions – three things you need to
know in order to decide whether or not the aid is effective. You need to know:
1. What aid is being given?

2. What are the goals of that aid?
3. Are those goals being achieved?
Once they had a central question and sub-questions, they needed to figure out where and
how to do their research. To do this, they needed to stop and consider the keywords and
stakeholders involved in the issue. In this case, the stakeholders include the Dutch
government (specifically the Ministry of Foreign Affairs, which oversees development
cooperation), the Afghan government, NATO (the international military force overseeing
efforts to stabilize and rebuild Afghanistan) and a wide variety of non-governmental
organisations and media outlets who are monitoring the situation.
29
If you take a look at the main research questions and at the list of stakeholders, it should be
easy to make a plan for how to answer each research question. Here is an example:
1. What aid is being given?

This question is about facts and figures and can probably be answered
easily with desk research. A visit to the Ministry of Foreign Affairs website,
where information on development cooperation is available, might be
enough, though a brief search of the news media might also be useful.
It would also be possible to get this information through field research –

approaching the ministry or any NGOs that are involved with questions –
but since most of this information is detailed and factual, it would be smart
to start with Desk research.
2. What are the goals of that aid?

This is also a question that can be dealt with through Desk Research, as
this information is likely to be found in the same place as the answer to the
previous research question – you can find it by investigating the relevant
stakeholders and/or consulting the news media through databases. You
could also attempt some field research (Interviews, for example) for more
detail if necessary.
3. Are those goals being achieved?

This sub-question is harder to answer through desk research, especially
because you would have to be careful about sources. This question
demands a critical analysis of the results of development projects in
Afghanistan and, while the government agencies involved probably publish
progress updates, you cannot simply take their word for it because there is
a potential for bias. You would need, for example, to consult objective
news coverage and to get the perspective of other stakeholders (local
organisations, international organisations, experts, etc.). Here is an area
where a mix of desk research and field research would be ideal. Interviews
with experts on Afghanistan or development cooperation might be able to
provide the objective, critical perspective you would need to find out
whether or not Dutch development aid is meeting its goals. Surveys of
stakeholders could also help.
In answering this sub-question, you have to be careful, think critically and

consider alternative explanations for the outcomes you discover and make
sure your answer is supported by the facts. If, for example, the Dutch
government had the goal of reducing violent crime in Afghanistan by 20%,
and you found a 20% reduction in crime statistics, you might conclude that
the goals are being achieved, but that is not enough. You need to make
sure that there isn’t another explanation, and prove, by using facts and
expert opinion/analysis, that the reduction in crime is, in fact, due to the
Dutch aid programme.
This is just one example of how the planning process could work. It’s just a question of
carefully considering the research question(s) and figuring out the right way to answer them. If
you’ve done that, you basically have a research plan, you just need to add some details.
G. The Research Proposal Form
Part of your homework will involve creating a concrete research plan to discuss in your
seminar. To make this a little easier, we have provided a standard Research Plan/Proposal
form at European Studies that is used for all research courses and projects, including the final
paper that all students write at the end of their studies. A word version of the form is available
on Blackboard for you to fill in, but here is the form with a little bit of explanation.
30
Research Plan/Proposal
Name
Student Number
E-mail address (Haagse Hogeschool)
Purpose of the form This form is designed for you to create a concrete
research plan for your assignment. You will use a
similar form for future projects.
Give a brief description of what you Central Research Question and Sub-questions
want to know

What is the central problem or question that
you hope to address with your research?
The word ‘problem’ here refers to any
theoretical or practical issue that you
believe would benefit from further research.

At this stage, you have to formulate a
central research question that indicates
what you intend to find out about the
problem you have identified. Further, you
need to outline sub-questions related to
your central research question.

Explain why you want to answer Context/Justification of the Research Question

this question

Use the adjacent box to reflect on key social,
political or economic issues as well as any
historical events or general circumstances
that define the context of your research
question.

Introduce this context by reference to

quality sources that you found and listed in
the ‘Tentative bibliography’ (see below). Use
APA style.
Relevance of the proposal to one or

more topics that are part of the
European Studies programme

Briefly explain how your topic and research
relate to the field of European Studies. Can
you link your research to specific modules
within the European Studies programme?
Methodology

What are the research methods you will use
and how will they enable you to answer
your research question? What (types of)
information sources do you plan on using
and why? In the adjacent box, describe your
methods and justify their suitability for your
research. Refer to literature on research
methodology. Use APA style.
Tentative bibliography and

stakeholders

Use the adjacent box to provide a tentative
list of academic (3) sources and non-
academic reports, articles or books (3) that
you plan to use in your research. Also add
literature on research methodology (at least
one source). Use APA style.

Further, be sure to mention any key
stakeholders.

Timeline

What is your timeline for completion?
Describe the key steps in your progression
to the successful submission of your
research project. What do you plan to do,
when and why?

Hand the form to your seminar instructor for approval.

Approved by:

Seminar Instructor: Date: Signature:
32
C. The Next Step
Once you have a solid research plan that has been approved by your teacher/supervisor, it’s
time to get straight to work on your research. All you have to do is follow your plan and work
with the end product in mind. The most important tip here is to STICK CLOSE TO YOUR
PLAN and WORK AS YOU GO.
The smartest thing you can do is get started on your research report right away. In fact, you
can start that report before you’ve done any research at all. Here’s how:
Using your Research Plan to Start your Report
As you already know, your research will culminate in an APA-Style Research report,
consisting of the following elements:
1. Executive Summary
2. Table of Contents
3. Introduction
4. Methods
5. Resutls
6. Discussion
7. Conclusion & Recommendations
8. A List of References/Annotated Bibliography.
Once you have a research plan, you already have what you need to write the
INTRODUCTION and the METHODOLOGY. In fact, you can probably copy and paste from
you research plan right into the report. Then, while you’re actually doing your research, all you
have to do is summarise the information that you find under RESULTS (while adding your
sources to your LIST OF REFERENCES) and when you’re done with that, you can look the
information over and write a short analysis or DISCUSSION, draw a CONCLUSION, make
RECOMMENDATIONS and then summarise everything in your EXECUTIVE SUMMARY.
We’ll be reviewing the structure and formatting later, but this is to help get you started and to
remind you that research is a process, not a product. Start up a template and add new
information as it comes in. This will help you to work efficiently and avoid stress, and if you
stick to this process, you’ll never have to worry about writing your whole report at the last
minute.
Seminar 2

You will bring to class your research question. You will include the steps you took to turn your
topic into a question. Each student will present their questions to their classmates and will
have feedback from the classmates and seminar instructor.

Students are expected to the do the following BEFORE attending their seminar:
Prepare a Research Plan
Using the form on Blackboard, take your research topic and use what you’ve learned in class
to prepare your research plan. Bring it to class to hand in to your teacher.
While you’re at it, be sure to start doing some preliminary desk research and be sure to bring
in any materials you find for discussion.
Week 3
Lecture: Dealing with Sources

In this class:
• You will discuss how to evaluate the credibility of sources.
• You will learn how to document sources in APA style.
A. Researching Credibility
By now you should be on your way towards doing some serious research and you should
already have gathered some information from sources. The question is, how do you know if
you can trust what you’ve found so far?
Quick Questions About Credibility

How can you determine the credibility of an information source? How do you know how to
trust?
Imagine you are about to write a research paper about a well-known politician. You are asked
to write about their background, their opinions, and their record of public service. How are you
going to get the information you need? How can you be sure it’s accurate?
Consider the following sources. What advantages and disadvantages does each of these
sources have if you view them with a critical eye?
1. Wikipedia
2. The website of the politician’s political party
3. A national newspaper
4. YouTube
5. A website you found on the first page of Google search results
What other sources could you consult? What advantages or disadvantages do they have?
B. Assessing the credibility of an information source
If you search for anything on the Internet, you’re likely to find something you can print off and
bring to class, but that doesn’t mean it’s accurate, true or trustworthy. After all, the strength
(and weakness) of the Internet is the fact that anyone can put something out there.
Tips for assessing sources
1. Identify and evaluate the Author
Make sure you can identify the individual or organisation that created your source. If you’re
using a book, you can look up the author or publisher to help you determine their credibility.
Here is another area where Wikipedia and similar sources might come in handy.
If you don’t see an author identified, look at where you found the information to see if a
corporate or organisational author can be identified. If your source is a newspaper, you can
assume that the newspaper is the author. You can do the same with many websites. More
often than not, the organisation that owns the website can be considered the author,
especially when dealing with governments, companies or organisations. If you’re looking at a
Blog (an online web-log), chances are that it's the work of one individual or a small group.
34
There are many interesting and trustworthy blogs out there, many of which are used by
officials, experts and organisations to distribute information, but many blogs are the work of
individuals with strong opinions that aren’t always backed up by facts. UNLESS YOU CAN
VERIFY THE AUTHOR’S CREDIBILITY, YOU SHOULD NOT TRUST BLOGS.
2. Consider the source
Take a careful look at the actual source of the information (the publication, database, journal
or website, etc.) and try to figure out if it is to be trusted.
News sources
If what you’re looking at is a newspaper or a news-oriented website, find out whatever you
can about that news source and its reputation. Newspapers and news websites are usually
good sources (and they often have useful databases), but are not all equally credible. You
must, for example, be able to separate reputable newspapers from tabloids and to identify
what types of news sites are reliable. Can you?
Journals and Databases
The databases you learned about earlier provide access to a wide range of publications,
particularly journals and magazines, in different fields. When looking at a journal or a
magazine, the same rules apply as above (identify the author and find out what kind of
publication you’re dealing with).
One important aspect of most respectable academic journals is that the articles are PEER
REVIEWED. That means that before a research article is published, it is reviewed by other
researchers in the same field to verify its quality. This process provides these journal articles
with a higher level of credibility.
***Please note that this is one of many reasons that you should acquaint yourself with the
databases and publications available to you through the university, as many are more
trustworthy than the sources you find on the Internet and, since they are on the Invisible Web,
you aren’t going to find them through conventional searches.
Internet Tips
Here are a few tips for making sure your favourite websites, and any other websites you come
across, can be trusted. Break down the Internet site’s address to see what kind of website it
is. To do this, you’ll need to familiarise yourself with the format of Internet addresses (URLs) –
see the Internet Detective tutorial on Blackboard for details – to determine if you’re looking at
a government site, a company site, a university, a blog or any other type of site.
1. Identify the website’s owners/authors (look to see if there’s an ‘About us’ page on
the site, for example).
2. Look for references to the site elsewhere. Search for the site in any search
engine (or Meta-Search engine) and see what pops up, or look the site and its
authors up on Wikipedia or elsewhere.
C. Bypassing Bias
Please read the following excerpts from two different Internet encyclopaedia articles about US
President Barack Obama.
What differences do you see in the way the information is presented? Which do you think is
more trustworthy, based on what you know about credibility, argumentation and logic?
35
Sample Text 1:
Barack Hussein Obama II (born August 4, 1961) is the 44th and current President of the
United States, having taken office in 2009. He is the first African American to hold the office.
Obama previously served as a United States senator from Illinois, from January 2005 until he
resigned following his election to the presidency in November 2008.
A native of Honolulu, Hawaii, Obama is a graduate of Columbia University and Harvard Law
School, where he was the president of the Harvard Law Review. He was a community
organizer in Chicago before earning his law degree. He worked as a civil rights attorney in
Chicago and taught constitutional law at the University of Chicago Law School from 1992 to
2004.
Taking a closer look
Can you find examples in the text of information that seems to be unbalanced or
untrustworthy?
Sample Text 2:
Barack Hussein Obama II (aka Barry Soetoro, said to have been born in Honolulu August
4, 1961) is the 44th President of the United States, and the first President who is biracial. He
has been associated with numerous radical fringe groups and causes and served less than
four years as a first-term Democratic Senator from Illinois (2005-2008). In a controversial
decision, Obama was the recipient of the 2009 Nobel Peace prize.
Under Obama's stewardship the United States has suffered the largest deficits in history and
a credit downgrade from the rating agency Standard & Poor. Poverty increased to levels not
seen in 50 years. After his economic program was adopted unemployment stagnated in the
10% range, leaving tens of millions without hope of finding a job. During what the White
House billed as Recovery Summer, one half million workers per week continued losing their
jobs. His signature legislative achievement, ObamaCare, the federal courts have found to be
unconstitutional and is credited as a major cause of uncertainty, high unemployment,
laging[sic]** job creation, and risk of a credit default crisis. Since the rejection of his party's
agenda in the 2010 Midterm elections, Obama has continued class warfare as a major theme
of his remaining two years and cornerstone of his re-election bid.
Taking a closer look

Can you find examples in the text of information that seems to be unbalanced or
untrustworthy?
Sources
The first text is from Wikipedia (http://en.wikipedia.org/wiki/Barack_Obama). The second was
taken from an American website called Conservapedia (http://www.conservapedia.com). This
website was created in 2006 by an American lawyer who feels that Wikipedia is not to be
3
trusted because it has “a strong liberal bias” (in the US, the term ‘liberal’ is applied to the
political left and followers of the democratic party.” Conservapedia alleges that Wikipedia
articles distort the truth for political purposes and presents itself as “a clean and concise
4
resource for those seeking the truth.” While the site instructs its users that “Everything you
5
post must be true and verifiable,” the site acknowledges that it is not neutral and has a
specific point of view that is influenced by American conservative ideas and Christianity.
Defining Bias
The Oxford English Dictionary defines bias as: “a strong feeling in favour of or against one
group of people, or one side in an argument, often not based on fair judgement.” Basically, it
means taking a side instead of remaining objective or impartial and it is something we see
every day, whenever someone allows their pre-existing opinions about a subject influence the

3 http://www.conservapedia.com/Wikipedia
4
http://www.conservapedia.com/Conservapedia:About
5
http://www.conservapedia.com/Conservapedia:Commandments
36
way they interpret new information and the way in which they communicate their opinions to
others.
You will often hear the word bias in connection with complaints about the news media. These
complaints are nothing new. They are as old as the news media itself. Often these complaints
are grounded in the fact that many newspapers and other news providers do have a clear
political perspective on the news, but this does not necessarily mean that those news sources
are biased.
Bias at work
The Conservapedia text about Barack Obama has a clearly biased opinion about President
Obama. This opinion is overwhelmingly negative and careful study of the article will give you
an idea of how to spot bias.
Who is Barry Soetoro?
It starts in the first line, when Obama is introduced as “Barack Obama, aka Barry Soetoro.”
The abbreviation A.K.A stands for ‘also known as’ and is used when referring to someone
who is known by many names. This article makes it appear as though this name is somehow
officially related to President Obama. The name comes from an unproven, Internet-based
rumour claiming that Obama took his stepfather’s last name (Soetoro) and used it to register
as a foreign student for financial aid at an American University. This story has been exposed
as a hoax by Snopes.com, a website that reports on rumours and urban legends – but, be
6
aware that this website has also been accused of having a liberal bias .
If you do a Google search for the name Barry Soetoro, you will find a number of links to
different websites (including Snopes), but you will notice that it is difficult to find any reference
to the name in the mainstream media or on any reputable source. If you search for it on
Wikipedia, you will link to a page about Conspiracy Theories related to Obama and his past.
The Wikipedia article (Example 1) states that Obama is “native to Hawaii,” while the
Conservapedia article says that Obama is “said to have been born in Honolulu…” a sentence
that casts doubt on whether or not the President was really born in Hawaii. This has proven to
be a controversial issue in the US, where many conservative groups have claimed that
Obama was not born in America (which would mean he cannot be President) and demanding
that the President show them his birth certificate. The Conservapedia text does not
necessarily support this claim, but their choice of language does imply suspicion.
Exercise: Find your own examples of Bias

Look back to the Conservapedia text and see how many other examples of bias you can find.
**By the way: If you are unfamiliar with what [sic] means, it is a tool to use when you are
quoting from a source that has either misspelled a word or used an unusual spelling. You put
[sic] after the word to indicate that the odd spelling or mistake was in the source. The authors
of this course book added it here. It is one of many spelling and grammar mistakes in the
Conservapedia article. You should keep this in mind because, when evaluating sources, the
quality of the writing says something about the credibility of the source – though it isn’t the
only criteria.
***Just for fun: You may want to check out Conservapedia, or RationalWiki – a site that
campaigns against Conservapedia – to see interesting examples of bias. Look up articles on
Democrats like Obama, and then compare them to the Conservapedia articles of noted
conservatives like John McCain and Sarah Palin and see what you notice. You might find that
articles with a positive bias, or you might find that articles about subjects Conservapedia is
sympathetic to are simply more neutral in tone and similar to Wikipedia articles. You might
also want to look at the Obama article again, as it has been revised regularly. References to
the birth certificate conspiracies were removed after President Obama released his birth
certificate in April of 2011 and officially disproved these conspiracy theories.

6
This about.com article addresses and seeks to disprove claims that Snopes.com cannot be trusted. This mini-
controversy provides yet another example of how important it is to be able to find trustworthy sources and to identify
bias: http://urbanlegends.about.com/od/internet/a/snopes_exposed.htm
37
7
Here is an excerpt of what Conservapedia says about John McCain, for example :
John Sidney McCain III, born in the U.S. Panama Canal Zone, August 29, 1936
(age 74), was the Republican candidate for President in 2008 as well as a prominent
maverick politician and Vietnam War hero. He is also the senior Senator of Arizona.
He has been an influential leader in American politics since 1986, when he ran for
office attacking corruption, attacking pork barrel spending and working for campaign
finance reform. He was defeated by George W. Bush for the Republican presidential
nomination in 2000, and lost to Barack Obama in the 2008 Presidential Election.
As a pilot for the United States Navy and former Prisoner-of-War (POW), he served in
the military for 22 years. McCain, a Baptist, has impressive credentials on issues of
foreign policy and national security, which have propelled him in the forefront of
national politics as both an outspoken voice for the United States military and as a
Presidential contender for the Republican Party since 1993.
You may have difficulty loading the site on some computers. According to Wikipedia and
RationalWiki, this is because Conservapedia tries to block people outside the US from
viewing the site.
Recognising Bias
Bias is relatively easy to recognise, especially since you already know how to recognise the
type of language used to establish and argue an opinion and bias is essentially an opinion
presented as fact in a misleading way, but there are other indications of bias:
1. If a statement is supported by a generalisation (e.g., “all computers are

expensive,”) or an (unproven) assumption, (e.g., “everybody knows that…) or it
provides no evidence or room for discussion, it is more than likely a biased statement.
2. Another key indicator is the use of value judgements – words and phrases that
imply a strongly positive or negative interpretation of what is being discussed (e.g.,
“this wonderful government programme should be extended,” or “filthy men’s
magazines should be banned.”). These kinds of statements make it impossible to
discuss or debate a given point of view because value judgements cannot be debated
in a logical way (you cannot logically discuss whether or not a magazine is filthy).
Bias is often based on preconceived notions.
D. What to do with sources: an Introduction to APA Style, In-text citations and

Referencing
Desk research is part of virtually all research projects and most of the time, you have a lot
more than three sources to work with. If you want to be able to make effective use of your
sources, there are two things you will need to remember to do: ORGANISE and DOCUMENT.
Organising
If you are the type of person who makes photocopies and printouts of everything, it’s
important to have some kind of filing system. Whether you use a binder or a notebook, you
will want to put your documents in some kind of logical order (by author, by title, or by subject,
for example) so that it’s easy to find the information you’re looking for.
If, on the other hand, you want to save a few trees, there are alternatives. You can (and
probably should) save digital copies of your sources on your computer (PDF files work best
when dealing with Internet sites), which can easily be filed in an organised way.

7
Retrieved on 29 April, 2011 from Conservapedia: http://www.conservapedia.com/John_Mcain
38
Whatever you choose to do, it is important that you KEEP YOUR SOURCE MATERIALS
HANDY. Don't ever assume that something you found on the Internet will always be where
you found it. Website content changes frequently (up to the minute in some cases) and you
may never again be able to locate the information you need.
Documenting sources
As you will no doubt remember from high school, all research projects are accompanied with
some kind of Bibliography or List of References. This list is usually found at the end of your
report, but that doesn’t mean it’s the last thing you should write. In fact, you should start
working on your bibliography immediately. As soon as you start doing research, you should
set up a list and add all the sources you find to it.
Every time you find a source you want to use, you should add it to your list and write down all
the information you would need in order to find it again (author, date, publication, etc.) and
you should write down a brief summary of what the source is, what it says and what you think
about it – whether or not it is credible, for example.
Your reference list/bibliography should be in APA style.
How to reference sources in APA style
You already know that APA Style is a set of rules for writing research reports and
documenting sources that is commonly used by universities and other researchers around the
world and that APA is the house style of The Hague University and will be used throughout
the programme. As a result, you can expect that EVERY REPORT WRITING ASSIGNMENT
YOU DO AT HHS SHOULD BE IN APA STYLE.
APA includes rules for how to make a List of References in APA style.
Creating a List of References in APA Style
APA Style includes rules for how to make a list of references. Consult an APA manual (such
as the student manual on Blackboard or one of countless web tutorials available on sites like
the official APA style site: www.apastyle.org) for details, but here is a summary:
Referencing Books:
Author (year ). Title . (Subtitle ) . (Edition).

Place of publication : Publisher
Non-periodical documents:
(Reports, fact sheets, published and unpublished dissertations and (conference) papers,
brochures)
Author (year, Month day ). Title . (Subtitle )

(Additional information if applicable). Place of publication : Publisher .
Articles from a periodical:
(Magazines, Newspapers, etc.)
Author (year, Month day ). Title . Subtitle .

Title of Periodical , volume number , issue number , page numbers .
If you don’t know the author, you can still use the same rules, but start with title.
39
Articles from a Website:
More and more students are turning to the Internet as a primary source of information, so it’s
important to know how to reference a website. Generally, you use the same format as above
(author, date, title, etc.) and then add the following:
Retrieved on Month day, year from Name of Website or Database : URL
Here are some examples:
Amnesty International USA. (2005, May 13). Guantánamo and beyond: The continuing pursuit
of unchecked executive power. (AI Index: AMR 51/063/2005). Retrieved from
Amnesty International Web site:
http://web.amnesty.org/library/pdf/AMR510632005ENGLISH/$File/AMR5106305.pdf
Bajaj, V. & Whitmire, K. (2006, May 25). Ignorance claim did not sway Enron jury. New York
Times. Retrieved May 26, 2006, from New York Times Web site:
http://www.nytimes.com/2006/05/26/business/businessspecial3/26jury.html?hp&ex=1
148702400&en=f9f7a276c89ed548&ei=5094&partner=homepage
Bank of New York. (1999). Banking for the millennium. New York: Author.
Biography of Martin Luther King jr. (n.d.). Retrieved July 14, 2005, from The Martin Luther
King Jr. Research Institute at Stanford University Web site:
http://www.stanford.edu/group/King/institute/
Chou, L., McClintock, R., Moretti, F., Nix, D. H. (1993). Technology and education: New wine
in new bottles: Choosing pasts and imagining educational futures. Retrieved August
24, 2000, from Columbia University, Institute for Learning Technologies Web site:
http://www.ilt.columbia.edu/publications/papers/newwine1.html
Franklin, M.N. (1995). Voting behavior. In The Encyclopedia of Democracy. (Vol. IV, pp.
1346-1353). Washington, D.C.: Congressional Quarterly.
Hanson, D., & Williams, G. (1991). The weight loss obsession. British Psychologist, 21(2), 5-
32.
A GOOD TIP
Be advised that the latest editions of software like Microsoft Word have tools you can use to
insert APA references easily. Get to know how they work and you can save yourself some
time. You can also use programmes like Zoetero to sort your information.
Making an Annotated Bibliography
Usually, when you have to hand in a research report, a simple reference list like the above
example is enough to provide readers with the information they would need if they wanted to
go out and find the information themselves. However, an Annotated Bibliography is even
better.
An ANNOTATED BIBLIOGRAPHY is a list of references that includes additional information.

It is an invaluable tool to use while doing research because it helps you to organise and
analyse your sources, and you should make one whenever you’re asked to do a major
research project.
Each time you locate a source, you should add the following information to your list:
1. A complete, APA-style reference (as seen in previous examples)

2. A brief summary of the source and what it says.
40
3. An analysis of whether or not the source is credible.
4. An Analysis of whether or not the information contained within the source
seems trustworthy (does it seem logical? Does there appear to be any bias?
etc.)
5. A statement whether or not the source would be useful for academic research.
Here is an example:
Seminar 3
In this class:
• You will discuss your Research Plans with your peers and teacher
41
Peer Review of Research Plans
In this week’s seminar you will have the opportunity to discuss your research plans with your
peers and with your teacher. If time allows your teacher may be able to approve your plans in
class. If not, they will be collected and returned to you at an agreed-upon time.
Criteria for Research Plans
In order to evaluate and improve each other’s research plans, consider the contents and
requirements of a good research plan.
A research plan should include the following:
1. A short introduction to the topic.

2. A central question that summarises what you want to know and an explanation
of your objective (why you want to know this).
3. Some sub-questions that break the larger question down into manageable units.
4. An indication of what research methods you will be using, what you will be using
them for and why you have chosen them.
5. A general plan or schedule.
6. A list of key terms, stakeholders and possible information sources that could
prove useful.
7. A list of all the information sources you have found so far.
It should also be a SMART plan, which means it should be SPECIFIC and the outcomes
should be MEASURABLE, REALISTIC and possible within the TIMEFRAME. The methods
chosen should be VALID and lead to GENERALISABLE results.
In small groups, exchange research plans and evaluate them using the following questions as
a guideline. Your teacher will come around and check.
1. Does the Research Plan contain a concrete, specific, measurable research

question?
2. Is the topic/Research Question relevant to European Studies?
3. Is the Central Question broken down into RELEVANT sub-questions?
4. Does the Research Plan provide an overview of relevant keywords and

stakeholders?
5. Does the plan contain a specific plan for research methodology, including an
overview of what types of information is needed and what types of information
sources will be used?
6. Do you think the methodology seems valid? If you had to do the research using
this plan, do you think you would reach the same results as the author?
7. Are sources mentioned and do they look trustworthy?
8. Do you have any suggestions for the author?

All students should do the following before attending their next seminar:
42
1. The CRAAP test
Go to https://www.youtube.com/watch?v=lAWhE0mj69I to learn more about this test.
2. Start your Annotated Bibliography.
Start up an Annotated Bibliography and include all of the sources you have found so far, with
a short summary and evaluation of credibility. Be sure to check your sources carefully, given
what you’ve learned. Print it off and bring it to class.
While working on your research, please do the following:
1. Find at least five sources of information relevant to your research topic – at

least three must be from academic databases available through the library.
2. Evaluate those sources using the CRAAP test and what you learned in class.
3. See if you can identity any opinions and decide if you think the source is
objective and trustworthy.
4. Update your Annotated Bibliography
Be sure to bring everything to class and be prepared to present your findings to the class.
3. Watch this short video about how reasoning trees can help you in your writing:
https://www.youtube.com/watch?v=PDPahgPEg1Y
43
Week 4
Lecture: Argumentation and Logic

In this class:
• You will learn about Argumentation, looking specifically at Logical Reasoning
• You will learn how to use a Reasoning Tree as a tool to evaluate (or construct)
arguments
• You will learn to identify bias and fallacy
A. Truth and Fiction: Introducing Argumentation and Critical Thinking
The credibility of any research source is very important, but it’s not the only important factor to
consider. After all, just because your information came from a source that is generally
trustworthy and believable, doesn’t make it true.
Discussion Question:
What is the truth? How can you tell whether something is true or not?
Argument and Agreement
In a certain philosophical sense, there is no truth. There is no one source of true information
in the world. Each of us has ideas that we hold to be true, that come from our upbringing,
experience and cultural background, but we have to keep in mind that this is our individual
truth and is not shared. This is because each of us sees things from an individual point of
view.
Imagine you went to a movie with your best friend. You discuss the movie afterwards and
learn that while you think it was the best movie ever seen, your friend hated it and wants his
money back. You don’t understand, because you thought it was so good and you argue about
it for a little while, but you realise, eventually, that your friend has a different way of looking at
things and you accept that. You agree to disagree
This is the same process that applies to knowledge and to research. Everyone looks at
information from their own point of view and not everyone agrees on everything.
But what about scientific facts?
There are many things that are held to be true. Like gravity, or the fact that the earth is round,
but, believe it or not, we don’t believe these things because they are true. We believe them
because most people agree that that they are true.
Science is, ultimately, a process in which someone observes or studies something in search
of an explanation. They gather evidence and form a theory and once they’re convinced
they’re right, they go out and share their theory with the world by making an ARGUMENT,
using their evidence as support. If the argument is convincing, the scientist’s explanation will
be accepted (though future studies may yet prove it wrong) but if the argument is not
persuasive or supported by the evidence, the scientist will have to start over again.
Another example: A Murder Case
If you watch television, there’s a good chance you are generally familiar with how a murder
case is investigated and tried. TV plays around with the facts, but the basics of the process
are more or less accurate.
A murder investigation starts with a body. Someone is found dead and police investigate by
gathering up evidence and conducting interviews (essentially doing a research project) and
finding out as much as they can about the victim’s life and relationships.
In this case, let’s say the victim was shot and that police discover that the bullet was fired
from a 357 Magnum handgun. The victim was in a relationship with a partner who owned a
357 Magnum handgun and who has no alibi (they can’t prove they were someone else at the
time of the murder) and neighbours say the couple was having problems.
Eventually, the police come up with a theory, based on their research. In this case, they think
the victim’s partner did it. They arrest the partner and a trial follows.
During the trial, the Prosecution will present the police department’s theory to a judge and/or
jury and will argue that this theory is correct. The suspect will hire a lawyer who will argue that
the police theory is incorrect, and will possibly present evidence of their own to defend their
side of the story. Ultimately, the judge and the jury will have to decide which theory they agree
with, but THEY WILL NEVER KNOW THE WHOLE TRUTH.
Discussion Question:
If you were on the jury in this case, what would you decide? Is the suspect guilty or not?
There is evidence to support the police’s theory, but does that make it true?
But what does any of this have to do with Research Skills?
If you want to become a better researcher, you have to be aware of the fact that no one
speaks “THE truth.” Most people speak from their own point of view and make arguments on
behalf of their ideas to try and convince others to agree. You will need this awareness
whenever you consult sources, so that you can use your CRITICAL THINKING SKILLS to
identify the arguments that are being made and the evidence used to support them so that
you can MAKE UP YOUR OWN MIND based on LOGIC and REASON.
Identifying Arguments
Every day you are confronted with information from teachers, fellow students, family, friends,
the news, the government, advertisers, etc. etc. Your job is to analyse that information in a
critical way so that you can determine its worth. In order to do that you must ALWAYS
CONSIDER THE ANGLE OR POINT OF VIEW OF THE SOURCE AND SEE IF THE
ARGUMENTS ARE SUPPORTED. You also need to be able to separate OBJECTIVE
information (information that has a neutral point of view, is based on agreed-upon facts and
can be verified) from SUBJECTIVE information (information that comes from a specific point
of view, is based on opinion and cannot be verified).
Critical Thinking is a two-fisted approach. A good grasp of the process enables you to do two
things:
1. Critical Thinking enables you to evaluate other points of view and information
sources in order to make rational decisions and get closer to (the) truth.
2. Later on, when you’ve completed your research and analysed all the information
you’ve found, you can use your critical thinking skills to make strong arguments
on behalf of your own conclusions.
How arguments are made
An argument usually consists of two components: A CLAIM (basically an opinion or a point of

view, sometimes called a contention) and at least one REASON offered to support that claim.
45
Examples:
A lawyer defending a client accused of murder will likely argue that their client is
innocent, and will offer the judge or jury multiple reasons such as: “my client was
somewhere else at the time.”
A teenager trying to convince their parents to allow them to stay out late will usually
also provide a reason, such as: “I should be allowed to go out with my friends
because I work hard at school and get good grades and deserve to have fun once in
a while.
The TRICK TO ANALYSING ARGUMENTS is to look critically at the reasons given to support
the claim and decide if you think those reasons are good enough.
Common Types of Arguments
There are many different types of arguments, each of which has strengths and weaknesses.
Study each one and see if you can find an example, either in the materials you collected for
Desk Research Assignment 2 or from other experience.
1. Evidence
Evidence-based arguments support their claims with facts. These are strong arguments, but
there are risks because facts (like statistics) can be misrepresented. It’s important to be
critical.
Example: 1. “My client can’t have committed the murder because someone else’s
fingerprints were on the knife.”
2. Authority
These arguments are often based on an expert’s authority or qualifications.
Examples: 1. “If the doctor says smoking is bad for you, he must be right because he
went to medical school.”
3. Analogy
This type of argument attempts to prove a point of view correct by making comparisons,
mentioning similar situations in which the outcome matched the arguer’s point of view. These
are tricky because they only work if the comparison is fair.
Example: “The Hague should try congestion charging to fight pollution because it works
in London. “
4. Cause and effect
Often, arguments are based on the idea that one thing caused another, or that one thing will
cause another. This is another area where you need to be critical, because they
Examples: 1.”The city is polluted because of all the cars. If we ban cars, the air will be
cleaner.”
2. “Raising the drinking age to 21 will stop teenagers from ending up in the
hospital for alcohol poisoning.”
The speaker’s point of view is that the drinking age should be raised. The
pragmatic argument is that it will have the effect of stopping teens from
ending up in hospital.
46
B. Introducing Basic Logic and Reasoning
Whenever you encounter arguments like the examples above, you should be able to identify
them and analyse them to help you make up your own mind. One important tool that will help
you accomplish this is your awareness and use of LOGIC. Logic, as defined in the Longman
Dictionary, Fourth Edition, is a way of thinking about something that seems correct and
reasonable, or a set of sensible reasons for doing something. It is our way of reaching
conclusions based on reason.
You probably already did some logic in your high school math classes, where you looked at
problems like this:
If x=1 and y=2, x+y=__________?
If we agree that x is equal to 1 and y is also equal to one, the answer is easy to find. If you
add x and y, the answer has to be 2. That is logical and cannot be disproven.
One of the most famous examples of a conclusion reached through logic concerns the
mortality of the Greek philosopher Socrates. At the time, philosophers were concerned with
the same concepts you are learning in Research Skills: finding ways to reach conclusions by
gathering and analysing information and arguments.
They asked themselves the question: How do we know that Socrates is Mortal (that he can
die)? They used the following steps to reach that conclusion:
1. First, they established and agreed that all men are mortal. (Premise 1)
2. They then established and agreed that Socrates was a man. (Premise 2)
3. They were therefore able to conclude that Socrates was mortal. (Conclusion)
This type of exercises is called a SYLLOGISM and a good example of the basics of logic. You
start with a PREMISE (a statement, reason or idea that you can build on), and continue to
build on that premise with other statements. If you can get everyone to agree to each
premise, you should be able to reach your conclusion.
While it would be fair to say that you don’t often encounter formal syllogisms in daily life, you
do often see this logical process at work.
Usually, you encounter logic in the form of an ENTHYMEME, a statement that starts with the
conclusion or claim and then provides the supporting reasons. Summarise the argument in
one sentence.
Examples:
1. Socrates is mortal because he is a man (and all men are mortal).
2. You should not smoke because smoking can cause serious illness.
This is the most common kind of argumentative statement and if you want to be an effective
researcher, you need to be able to recognise statements like this and critically analyse the
reasoning to see if you think the claim is supported.
Example
You might hear someone make an argument like “Women should not be allowed in Formula 1
races because women are not as good at driving as men.”
Is this logical?
Remember that in order for your logic to work, you have to base it on premises that people
will agree with. If people don’t agree that women are not as good at driving as men are, the
logic falls apart.
47
Different ways of using Logic
There are three main types of logical reasoning: Deductive, Inductive and Abductive logic.
Deduction
DEDUCTIVE LOGIC is logic that starts with a major premise, a general rule, and adds a
minor premise, a specific application of that rule, in an attempt to reach a conclusion. It is said
to work ‘from the top down.’
Example:
Premise 1. All men are mortal (a major Premise/general rule)
Premise 2. Socrates is a man. (a specific premise applying the rule)
Conclusion: Socrates is mortal.
For deductive logic to work, it is important that the major premise be accepted as true. If we
cannot agree that all men are mortal, it does not matter if Socrates is a man or not. The
strength of deductive logic is that if your premises are true, your conclusion cannot be proven
false. However, building an argument on an inaccurate or faulty premise guarantees that your
argument will not be successful.
Induction
INDUCTIVE LOGIC is a bit more complex, as it involves making inferences (assumptions and
indirect conclusions) based on the information you have. Because you don’t have a major
premise to work with and the information you have is often incomplete, Inductive logic is not
so much about finding the TRUTH as it is about reaching a conclusion that SEEMS
REASONABLE. Because it is less certain, it is often more open to attack.
Inductive logic often works as a mirror image of Deductive Logic. Instead of taking a general
rule and applying it to a specific circumstance, Inductive logic uses specific circumstances to
demonstrate a rule.
Example: Premise 1. We polled 1,000 voters.
Premise 2. 750 of those voters think it’s time for a change.
Conclusion: Therefore, 75% of all voters want change.

Abduction
ABDUCTIVE LOGIC is a more abstract method based on the process of finding the most
likely explanation for an unexplained or incomplete set of circumstances, observations or
evidence. Two fields in which this form of logic play a central role are in medicine (where
doctors make a diagnosis by considering the symptoms a patient has and making an
informed guess about what is causing them based on their knowledge of what diseases can
cause those symptoms) and criminal justice, where authorities examine the evidence left
behind after the crime and use it to determine who the most likely suspect is. Even during a
murder trial, no one knows the exact truth about what happened, only a likely explanation
based on the evidence and testimony gathered.
Abduction is the primary method of logical analysis seen in Sherlock Holmes stories, though
the fictional detective often mistakenly claims to be engaged in deduction.
In a sense, it requires looking at the conclusion of a logical syllogism and then inferring the
premise. When applying abductive reasoning, it is wise to remember ‘Ockham’s Razor,’ a
48
logical principle that argues that when you have competing theories to explain a conclusion,
the simpler explanation is likely to be the correct one.
Example:
Premise 1: The ground shook under my house in California
Premise 2: There are a lot of earthquakes in California
Premise 3: Earthquakes can cause a house to shake.
Conclusion: It was most likely an earthquake that caused my house to shake.
F. Using Reasoning Trees to break down logic
So far this all sounds very theoretical, but logic is actually a very practical tool that you can
and should use regularly. If you can become skilled at identifying arguments and analysing
their logic, you will not only be a better researcher, you will be better equipped to study and to
communicate your ideas to others or to make your own arguments.
One of the best ways to analyse logic is to create a REASONING TREE, a visual diagram
that breaks arguments down into their core components. Here is an example:
Example Reasoning Tree 1
To make a reasoning tree, draw a diagram and put the claim or conclusion at the top. Then
add the reasoning (the premises and evidence you used to make the argument) underneath.
Be sure to demonstrate the relationships between boxes with words like ‘Because,’ ‘and,’ and
‘but.’ The end result is a simple diagram that allows you to consider each individual part of
the argument, to see if the logic holds up. There are a variety of tools you can use to make a
Reasoning Tree, such as Rationale, or a large number of other ‘mind-map’ software
programmes and websites (many of which are free) BUT YOU SHOULD NEVER FORGET
THAT YOU DON’T NEED A COMPUTER TO MAKE A REASONING TREE. You can and
should be able to draw one by hand or to conduct the analysis in your head.
There are a lot of different ways of making a Reasoning Tree (sometimes called an Argument
Map). To get a better sense of the possibilities, check out some of the Critical Thinking
tutorials offered online. Austhink, the makers of Rationale, has a particularly useful website on
Critical Thinking (http://austhink.com/critical/) that includes Argument Mapping Tutorials
(direct link here: http://austhink.com/reason/tutorials/) that can be helpful even if you aren’t
using Rationale.
You can and should use Reasoning Trees to analyse arguments, but you can also use them
to construct your own arguments or to reach your own conclusions. In doing so, you can also
incorporate counterarguments into the tree and you can try to defeat those counterarguments
for rebuttal.
49
C. Finding Fallacy
Closely related to Bias but slightly different is the concept of fallacy. FALLACIES are a form of
bad logic: common mistakes, irrelevant points or poor arguments that undermine or destroy a
person’s position and credibility when carefully analysed. Any argument that doesn’t work
when all the facts are added up is a fallacy. The more research you do, the more often you
are likely to encounter fallacy. The good news is that you already know how to identify it:
simply analyse the logic or make a reasoning tree outlining the point of view you’ve just
encountered. If it doesn’t add up, if there’s no support for the argument, it’s probably a fallacy.
There are many different kinds and if you’re interested in learning more about them, do some
online research at the Fallacy Files (www.fallacyfiles.org) to see an impressive list and to quiz
yourself.
Tips for detecting Finding fallacy
In a sense, detecting and avoiding bias and fallacy is what Critical Thinking is all about. You
already have the tools to analyse and create logical arguments. All you have to do is put
those tools to good use. If, for example, you want to detect bias or fallacy in an argument, use
the following steps:
1. Determine the overall opinion or point of view.

2. Analyse the evidence (if any) given to support that point of view.
3. Note what kind of rhetoric is being used.
4. Determine whether or not the evidence and supporting arguments are logical.
5. Analyse the language – strip away positive and negative value judgements and
view the argument from a neutral perspective.
6. Consider the credibility of the source.
7. Analyse the entire argument using a Reasoning Tree.
If you are preparing your own argument, remember to the following:
1. Use evidence to support your claims.

2. Be aware of what type of rhetoric you are using and why.
3. Use neutral language and avoid value judgements or reasoning based on
emotion.
4. Avoid generalisations or assumptions not backed up by facts or logic.
5. Anticipate counterarguments and be prepared to defeat them.
6. Outline your own arguments using a Reasoning Tree.
‘Evaluating Logic’ test
The following will test your knowledge of logical reasoning based on what you’ve learned so
far by asking you to complete and evaluate some logic.
Part 1: Review
Look at the following logic exercises and determine the logical conclusions that can be made
from them. Then, create an enthymeme expressing the syllogism and explain whether or not
you are dealing with deductive, inductive or abductive logic. Be sure to review the material
covered in class if you’re unsure about terms.
1. The boy’s fingerprints were on the knife.

The knife was used to murder the butler.
Conclusion:
Type of Logic:
2. Five dentists interviewed for this commercial.

Four of the dentists agreed that our toothpaste was the best.
50
Conclusion:
Type of Logic:
3. Natural resources are limited.

Coal is a natural resource.
Conclusion:
Type of Logic:
4. All squares are rectangles.

The plaza is a square.
Conclusion:
Type of logic:
Analyse the following. Are they sound logic? Why or why not?
5. Some quadrilaterals are squares.

Figure 1 is a quadrilateral.
Conclusion: Figure 1 is a square.
6. Americans are not good soccer players.

Bill is American.
Conclusion: Bill cannot be a good soccer player.
7. When it rains, the sidewalks get wet.

It is raining right now.
Conclusion: Right now, the sidewalks are wet.
Take at least two arguments and make reasoning trees that separate the claims from
the support given.
Part 2: Bypassing Bias

See if you can identify any kind of bias in the following statements:
1. Studies show that a penal system that focuses on rehabilitation is more likely to result in a
reduction in repeat offenses than a penal system focused on punishment.
2. Criminals don’t need rehabilitation. They need to be punished. If prison isn’t a deterrent,
there will be nothing to stop people from breaking the law.
3. It’s obvious that smoking pot leads to heavier drugs.
4. Golf is boring and anyone who watches it is boring too.
5. Blonds are almost always dumb and brunettes are almost always smart.
6. Rap videos are insulting to women and should be banned.
7. The Ministry of Health says that smoking is bad for you.
8. Politicians are usually dishonest so the politician running for mayor of my city is probably a
liar.
51
9. The only reason why so many women voted for Hillary Clinton is because she is a woman
too.
Part 3: Finding Logic in your sources
Go back to the information you found while doing your Desk Research. You should have
many different sources that provide you with information about your research topics and it is
likely that they contain some argumentation. Make a list of all the arguments you find and
evaluate whether or not you think they’re logical, based on what you’ve learned in this course
so far. If you do not have enough examples, you can always do further research. At the same
time, see if you can find any evidence of bias or fallacy.
52
Seminar 4

In this class:
• You will review what you’ve learned about identifying credible sources,
• You will discuss the CRAAP test,
• You will discuss and get feedback on your research and Annotated
Bibliography.
A. Review the CRAAP test

B. Apply Research Criteria to Your Research
Taking what you’ve learned from the Internet Detective and class discussion to heart, take a
look back at the information you’ve found so far and evaluate your sources. Be prepared to
discuss the following with the class:
1. Are you satisfied that all of your sources are credible? Why or Why not?
2. Have you encountered any sources on your topic that might not be credible,
based on what you know now?
C. Peer review of Research Progress and Annotated Bibliography
By now you should have done a bit of desk research and you should be summarising it in
your research report. At the same time, you should have started an Annotated Bibliography,
containing an APA reference for each source you use, as well as a summary of the source
and an assessment of its credibility.
In this seminar, you will show and discuss your work with your peers in pairs or small groups.
Your teacher will come round to check the work, answer questions and provide feedback
where necessary.
If you haven’t yet finished summarising your information or developing your Annotated
Bibliography, you should get started in class and take advantage of the opportunity to get
feedback and pointers on your research.

Before Week 5, all students must do the following:
1. Write a 1-page draft of any section of your paper. Include signal phrases and properly
formatted in text-references.
2. Create a reasoning tree for your argument
Bring both to class for seminar 5.
3. View APA Style tutorial
Visit the APA Style website (www.apastyle.org) and look at the online ‘Basics of APA Style’
tutorial on the site (http://flash1r.apa.org/apastyle/basics/index.htm). It will review what we
covered in class. Note that there may be some variation.
4. Do The Following Online Plagiarism Tutorial

If you were to do an Internet search for ‘plagiarism tutorial’ you would find that dozens of
universities and other institutions have created online resources to help students learn to use
sources responsibly. Check out this tutorial from The University of Southern Mississippi:
http://www.lib.usm.edu/legacy/plag/plagiarismtutorial.php
The tutorial invites you to ‘Pre-test your knowledge of Plagiarism’ Select that link and do the
quiz before moving on to the tutorial. At the end of the quiz you can fill in your e-mail address
and the university will e-mail you your results automatically. Print your results out and bring
them to class.
This tutorial is particularly useful and worth doing.
54
Week 5
Lecture: The Structure of Your Research

Paper
In this class:
• You will discuss ways to reach conclusions
• You will learn the basics of how to communicate your conclusions in a
research report
• You will discuss using APA style
A. Review
Take some time to look back on what you’ve learned in this course so far. If you have any
questions or would like to review anything, ask your teacher. If you weren’t able to finish all
the exercises in previous classes, you may want to ask your teacher if answers could be
discussed now or at another moment.
B. Coming to Conclusions
Once you’ve gathered up your information and have assessed it carefully, you should be
ready for the next step: coming to some kind of conclusion. That is, after all, the whole point
of doing research.
Discussion: what is a conclusion?
You hear the term conclusion thrown around a lot, but have you ever given much thought to
what it means? What does it mean ‘to conclude’ and how do you go about doing it?
The Oxford Dictionary defines a conclusion as: “something that you decide when you have
thought about all the information connected with the situation.” In research, this refers to
something even more specific.
REMEMBER – Earlier in this course you learned that research always starts with a question:
something you want or need to know. The whole purpose of doing research is to ANSWER
THE QUESTION. If you’re capable of answering a central research question, you have your
conclusion right in front of you. Then, all you need to do to defend your conclusion is provide
some logical or scientific reasons why your conclusion is correct, as you do in a reasoning
tree.
It is really that simple.

Example
A company that wants to launch a new product will first conduct market research to find out if
people are likely to by the product. They would typically do some desk research to find out all
kinds of things, like how much it would cost to make the product and how popular similar
products are and then they would do field research (interviews and surveys) to see how
people would react to the product, asking questions like “would you buy this product” and
“how much would you be willing to pay for this product.”
The central question for this research project is:
Is product X likely to prove profitable for our company?

Imagine that the desk research showed that producing the new gadget would be really
expensive and that there are several similar products on the market right now, and that
interviews with members of the general public showed some interest in buying it, but only at a
low price. What conclusion would you make and why? What is the answer to the central
question?
__________________________________________________________________________
__________________________________________________________________________
What if production for the product turned out to be low in costs and people seemed interested
in buying it?
__________________________________________________________________________
__________________________________________________________________________
In both cases, your answer should include the question and be phrased as follows:
Product X is/is not likely to prove profitable for our company

because…(here is where you offer your reasons).
Practice Exercise: Predicting Success in Hollywood
Here is a brief example research project that we’ll do together in class, going through each of
the steps of a research process. Imagine you have been assigned to do some market
research on the movie business. You were asked to find out what types of movies are more
likely to be successful at the box office. Your central question is: WHAT KINDS OF FILMS
ARE THE MOST POPULAR?
Step 1. Figure out what you want to know and why
The main thing we want to know is obvious: what types of movies do people like. Why do we
want to know? Movie Star Will Smith famously made this same analysis once to determine
what projects he should say ‘yes’ to if he wanted to be a big star. Studios decide on
investments based on past performance:
Step 2. Select research methods
Chances are that you would choose to do desk research on this subject, as the Internet
provides plenty of information on topics like this.
However, before you start ‘Googling,’ you need to make sure you know exactly what you’re
looking for. You need to identify any relevant stakeholders (like movie studios, etc.) and
keywords that would help you to do research. This will likely involve some Critical Thinking.
Ask yourself: How can you measure the popularity of a movie in an objective, relatively
scientific way?
One way you might consider is to look at the ‘bottom line’ by finding out what types of movies
make the most money. Financial success – while certainly not an indication of quality – is one
way of measuring a product’s popularity.
So, your method would be to search for the most successful movies
Step 3. Conduct your research by applying the chosen Methods
In order to search for the information you need, you need to determine any relevant
stakeholders or keywords.
If you search for ‘Most successful movies,’ you won’t find many results. But if, for example,
you were to do some preliminary research, you would likely learn that financial success for
56
movies is measured in terms of ‘Box Office Gross,’ which refers to the money brought in
through ticket sales. If you search for ‘Highest Grossing Movies,’ you will get more results.
However, you would want to NARROW things down and BE SPECIFIC in what you’re
searching for. If you search for Highest Grossing Movies, you’ll likely get a list of the most
successful movies at the moment. If you want a historical trend, you’d want to search for
‘Highest Grossing Movies in history’ or ‘Highest grossing movies of all time.’
If you do that search, you will get a number of results and will be able to choose from different
sources. If you scroll past Wikipedia, you’ll probably find at least one link to a site called ‘Box
Office Mojo,’ which keeps track of ticket sales.
REMEMBER TO CHECK CREDIBILITY: If you looked up Box Office Mojo, you would learn
that the website is quite popular and is owned by Amazon.com, the Internet bookseller that
also owns the Internet Movie Database. You might conclude that there is no major reason to
doubt the source’s credibility.
Box Office Mojo keeps lists of the highest grossing movies in the US and worldwide. Since
we’re interested in the whole world, we’ll go straight to the international list.
Here is a list adapted from Box Office Mojo’s list of the Top 20 Highest Grossing Films as of
early 2015. Amounts are in millions of US Dollars:
Rank Title Gross
1 Avatar $2,788.0
2 Titanic $1,86.8
3 Start Wars: The Force Awakens $2,067.0
4 Jurassic World $1,670.4
5 Marvel’s The Avengers $1,519.6
6 Furious 7 $1,516.0
7 Avengers: Age of Ultron $1,405.4
8 Harry Potter & the Deathly Hallows: Part 2 $1,341.5
9 Frozen $1.276.5
10 Iron Man 3 $1,215.4
11 Minions $1,159.4
12 Captain America: Civil War $1,131.5
13 Transformers: Dark of the Moon $1,123.8
14 The Lord of the Rings: The Return of the King $1,119.9

15 Skyfall $1,108.6
16 Transformers: Age of Extinction $1,104.1
17 The Dark Knight Rises $1,084.9

18 Pirates of the Caribbean: Dead Man’s Chest $1,066.2
19 Toy Story 3 $1.063.2

20 Pirates of the Caribbean: On Stranger Tides $1,045.7
57
Step 4. Analyse Your Data
Once you have a list like this, you can start analysing your information and looking for
meaningful trends or patterns or anything that might lead you to the answer to your research
question. Take a look at this list and see what kinds of conclusions you can make about what
types of movies are more popular. What analysis would you make?
If you want to make a meaningful analysis, you’d need to find out more. Since your central
question involved finding out what kinds of movies are usually successful, you need to know
what kinds of movies are on the list. Here, the work is done for you. Each of the movies has
been categorised in this chart:
Rank Title Gross Genre

Science
1 Avatar $2,788.0 Fiction
2 Titanic $1,86.8 Drama
Science
3 Start Wars: The Force Awakens $2,067.0 Fiction
Science
4 Jurassic World $1,670.4 Fiction
5 Marvel’s The Avengers $1,519.6 Superhero
Action-
6 Furious 7 $1,516.0 Adventure
7 Avengers: Age of Ultron $1,405.4 Superhero
8 Harry Potter & the Deathly Hallows: Part 2 $1,341.5 Fantasy
9 Frozen $1.276.5 Animation
10 Iron Man 3 $1,215.4 Superhero
11 Minions $1,159.4 Animation

12 Captain America: Civil War $1,131.5 Superhero
Science
13 Transformers: Dark of the Moon $1,123.8 Fiction
14 The Lord of the Rings: The Return of the King $1,119.9 Fantasy
Action-
15 Skyfall $1,108.6 Adventure
Science
16 Transformers: Age of Extinction $1,104.1 Fiction
17 The Dark Knight Rises $1,084.9 Superhero
18 Pirates of the Caribbean: Dead Man’s Chest $1,066.2 Fantasy
19 Toy Story 3 $1.063.2 Animation
20 Pirates of the Caribbean: On Stranger Tides $1,045.7 Fantasy
So, what can you observe here?
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________
58
Step 5. Make Conclusions (and Possibly Recommendations)
To draw a conclusion, you simply look at your analysis and see if it leads you to an ANSWER
TO YOUR CENTRAL QUESTION: Try to answer your central question below:
The most popular kinds of movies are __________________________

________________________________________________________
________________________________________________________
In order to convince anyone, you have to make sure you PROVIDE SUPPORT for your
conclusions. You can easily do this by making a quick Reasoning Tree to support your
answer (the most popular kinds of movies are_______________ because of
_______________ and ________________).
REMEMBER: a Conclusion refers to WHAT YOU KNOW NOW.
You might take this conclusion and make some RECOMMENDATIONS, depending on why
you were doing the research. Recommendations refer to WHAT YOU THINK SHOULD BE
DONE WITH WHAT YOU KNOW.
If for example, you were asked to find out what types of movies are successful for a film
studio, you might be able to give them advice on what films they should consider making.
BE CAREFUL
Don’t forget that it is important to be careful about conclusions and recommendations. Don’t
make promises you can’t keep.
Looking at our data, we could conclude that many of the most successful films ever made
belonged to the genres of Fantasy or Science Fiction, but you cannot take that and argue that
a new fantasy or sci-fi film is definitely going to be successful for certain, because there are
other variables to consider and there are no guarantees. This is the kind of poor logic that
gets Hollywood into trouble from time to time (If you don’t believe me, do some research and
read up on the film Battlefield Earth).
It would be reasonable to conclude that fantasy and sci-fi films are generally popular and that
new films in this category may be more LIKELY to sell tickets than other kinds of films, but
you would really need to do more research and look at other factors as well (special effects,
the number of famous movie stars involved, whether or not the films are original stories or
adaptations of other stories, etc., etc.).
59
OPTIONAL: An extra bit of Critical Thinking
There are, of course, other issues that could complicate things a little if you use your critical
thinking skills. If, for example, you did some more research on these films, you’d notice that
virtually all of them were released within the last decade. Does that mean that these movies
were more successful/popular than movies released before the 1990s?
Probably not. Historically, movies make more money every year for two reasons: 1. There are
more cinemas than they’re used to be. 2. Ticket prices have gone up dramatically throughout
history and all prices are subject to inflation. If you were to incorporate your research, you
would have to search for something like the ‘highest grossing movies of all time adjusted for
inflation’ which would give you a completely different list and might lead to some different
conclusions.
C. Getting Your Point Across: Communicating Conclusions
Once you’ve reached a conclusion, you are likely to want to share it with others. At University,
there are likely two ways in which you would try to do this: in a RESEARCH REPORT or in an
academic essay.
1. An Academic essay
We will not be writing essays in this course, but it’s a good

idea to be familiar with the general format. Essays are
argumentative: you write them to try and convince others of
your point of view, so instead of starting with a question, like
you do in research, you start with the answer and go on to
back it up with arguments and evidence. They typically consist
of the following components (see sample outline):
1. An Introduction
An introduction usually attracts the reader’s attention to a

specific topic and then introduces a main point or THESIS (a
statement summarising your argument.)
2. A Body
Here is where you support your point with evidence and

examples found in your research and provide some analysis.
3. A Conclusion
In a conclusion, all you have to do is refer back to what you said and make sure that you have
proved your point. Then you might reflect on the topic in a more general way and tell the
readers what they should do with this information.
One of the easiest ways to write an essay is to make a REASONING TREE for your main
point and then use it to plan your supporting paragraphs. See the Rationale video used in
previous tutorials to look at how you can do this:
http://www.youtube.com/watch?v=HoEu7xy7fjM.
2. Research Reports in APA Style
While you won’t be writing any essays in this course, you will soon begin writing research
reports using APA Style.
60
Though many students find this hard to believe, APA style is actually very simple and easy to
use and the report writing rules are actually very helpful, because they provide structure.
An APA style report includes the following elements:
1. Title Page
Title pages typically give a title to the report and include the author’s name and other relevant
information (student ID, course, teacher, etc.). Please note that the words ‘title page’ should
NEVER appear here.
2. Abstract/Executive Summary
An Abstract or Executive Summary is the first thing a reader sees after the title page, but it’s
the last thing you write, because it’s purpose is to summarise the ENTIRE REPORT, telling
readers what research was done and why, what the findings were and even what the major
conclusions and recommendations are. They are usually very brief (around a half a page) and
should communicate all the essential information.
If the person reading your report is in a hurry and only has time to read the abstract, they
should still know everything they need to know.
3. Introduction
The Introduction to a research report should accomplish a number of goals. It must, for
example, get the reader’s attention and introduce the general topic of the research. It must
also establish the central question and sub-questions of the research and the purpose of
researching it. Since answering that question is the reason you’re doing the research in the
first place, it must be clear.
4. Methods
Here you should describe, step by step, the methods that you used (note the past tense) to
conduct the research. You must explain what methods you chose to use and why you chose
those methods. In a major research project, you will use multiple methods and in your
methods section, you explain your choices.
To do this right, there are two things to remember:
A. Research is impersonal (no “I”)
You cannot write “I did desk research.” You can avoid this problem
by using the Passive tense and writing “desk research was
conducted…”
B. Research reports are written in the past tense.
The work is already done by the time someone reads your report, so
write it in the past tense. Instead of “I will interview…” write “An
interview was conducted…”
5. Results
One of the most important parts of the report is your summary of the results of your research.
Here, you answer the question: what did I find out from my research?
In this section you describe the results but you do not interpret or analyse them yet. If you’ve
done desk research, you simply summarise and/or quote the information you found. If you
61
interviewed someone or conducted a focus group, you summarise and/or quote what they
said. And if you conducted a survey, you report what the respondents said.
It is very important to stick to “just the facts” here and not to overdo it.
6. Analysis/Discussion
In this section you take the results you reported in the last section and explain, analyse and
interpret them. Here is where you start to compare answers and look for significant trends
that might have meaning for your central question. If, in your Results section, you reported
that 80% of the people who took the survey would like to buy your product, here, you can
comment on what that might mean. Essentially, you go through your data and look for
anything interesting and relevant.
7. Conclusions
The importance of a conclusions section should be somewhat obvious. Now that you’ve
reported your findings and analysed the most relevant findings and correlations, you should
be able to answer your central research question.
Please note that, even if your research doesn’t work out the way you planned, you can still
reach a valuable conclusion. Even if you can’t answer your central question, good research
should provide some insight that will help you figure out how to find the right answer next
time.
8. Recommendations
Recommendations, as mentioned earlier, are suggestions about what should be done with
the new knowledge acquired through the research. They are not always included in a report
but often are when the research was conducted for a specific purpose. That purpose will
determine the nature of the recommendations.
9. References (or an Annotated Bibliography)
The final section of your research report is your list of references, where you document all of
the sources you used for information, usually in alphabetical order.
Proper referencing is very important, especially if you want to avoid being accused of
PLAGIARISM.
The Style of Your Research Report
Research reports are formal in tone. That means there are a few rules to remember. As
stated above, the writing must be impersonal and avoid informality.
Here are a few rules to remember:
• Avoid Contractions (It’s, don’t) and stick to full forms (it is, do not)
• Avoid personal pronouns (I, you, us, etc.) and use passive constructions instead
• Avoid informal terminology
Here are a few examples:
Informal Acceptable
My/Our central question is… The central question of this research is…
We did a questionnaire. A questionnaire was used (passive).
To understand this question, you should… To understand this question, one should…
I/We conclude It can be concluded (passive)
I/We recommend It is recommended (passive)
62
Example: the Movie Report
A research report about the movie research discussed above would probably look a lot like
this:
Sci-Fi Suggestions
An analysis of the likelihood of box office success for different genres
Executive Summary
The purpose of this report is to summarise research conducted by XXXX on the highest
grossing movies of all time, for the purposes of identifying which types of movies are most
likely to perform well financially. The central question of this research is: what types of films
are the most likely to achieve financial success?” To answer this question, research was
conducted to obtain a list of the highest grossing films of all time and a thorough analysis was
conducted of each title on that list to determine genre, after which a patter emerged, showing
a significantly high number of science fiction and/or fantasy films, often based on pre-existing
stories, among the most successful films of all time. This lead to the conclusion that science
fiction or fantasy films seem the most likely to succeed.
(Table of Contents)
1. Introduction
The purpose of this report is to answer the central question, “What types of movies are most
likely to achieve financial success?” In the hope of providing recommendations for which
upcoming films are likely to prove lucrative.
The central question has been broken down into the following sub-questions:
1. Which movies have been most successful in (recent) history?

2. To which genre do the most successful movies belong?
2. Methodology
In order to answer the relevant sub-questions, research was conducted to identify the most
successful films in history. For the broadest possible picture, a list of the most successful
movies of all time was chosen for analysis, though it should be noted that this list was not
adjusted for inflation, which skews the data in favour of more recent developments.
3. Results
According to Box Office Mojo, a reputable online resource for box office statistics, the highest
grossing films in history are….(list or chart, followed by APA-Style referencing). XX number
were Science Fiction, Xx number were dramas, etc.
4. Discussion
Analysis of the highest grossing films of all time has lead to the discovery of a pattern. Most of
the highest grossing films were science-fiction, superhero or fantasy films, many of which
were also adaptations of well-known source material, such as Marvel Comics characters, etc.
5. Conclusions
Based on the information documented in this report, the answer to the central question “what
types of movies are most likely to achieve financial success?” seems clear. Science Fiction
and fantasy films, as well as superhero films, are the most likely to succeed, based on
historical trends, especially if they are based on established characters or titles.
It should be noted that the statistics used for this research were not adjusted for inflation,
which could lead to slightly different results because of different trends in the distant past.
However, given that the statistics give an accurate picture of what has been popular in recent
years, the above conclusion still seems sound.
63
6. Recommendations
It is recommended to consider making/investing/performing in a science fiction, superhero or
fantasy film. It is also recommended to continue doing research to identity other trends and an
adjustment for inflation could be considered in order to seek out long-term patterns.
7. Annotated Bibliography
“All Time Box Office: Worldwide Grosses,” 2015. Retrieved from Box Office Mojo on 13 April,
2015: http://www.boxofficemojo.com/alltime/world/
D. What to do with sources in your report
Another important aspect of APA style is its system of rules for citing sources. Whenever you
use words or ideas that come from a source (anything besides your own brain) in the body of
an essay or report, you must always make sure that it does not look like you are passing off
their work as your own. You have to clearly mark EVERYTHING that is not your own creation.
In-Text Referencing
Whenever you want to include ideas or text in an assignment that you did not think of
yourself, you need to follow three simple steps:
1. Use a Signal Phrase

Writers must introduce borrowed words or ideas by using phrases like "According to" or "As
Smith said," in order to be clear about where the information came from.
2. Quote or Paraphrase
Once you have a signal phrase in place, you can include the information/text from your
source, ether as a direct quote (the exact words from the source, inside quotation marks) or
as a paraphrase (a close summary in your own words that communicates the same idea.
3. Provide a Citation
After you’ve warned your readers that you are borrowing content, and included the content as
a quote or paraphrase, you need to tell the reader where it came from with an in-text
reference. In APA Style, this is done by putting the information between brackets – (like this).
You do not need to include detailed information, just the last name of the author, the date of
publication and the page number. If you don’t know the author, use the title. If you don’t know
the date, use n.d. and if you don’t know the page (if it’s a short text), use the paragraph
number).
Example Citations
According to author Bob Smith, "APA style is easy to use" (Smith, 2011, p. 12).
According to Smith, APA style is not difficult (Smith, 2011, p. 12).
As author Bob Smith (2011) said, "APA style is easy to use" (p. 12).
Please note that punctuation comes after the citation.
Dealing with titles
You should be aware that the titles of books, magazines, newspapers, films or other
significant works are usually italicized (like this) while the titles of articles, essays and other
short works are usually put in quotation marks (“like this”)
More Examples
64
A recent study shows that "Students often had difficulty using APA style," (Jones, 2003,
p.199), but she does not offer an explanation as to why.
Here, a signal phrase (A recent study shows that) is followed by a direct quote (Students
often had difficulty using APA style) followed by a citation (Jones, 2003, p.199) that shows the
author, date and page.
You could also do this without a direct quote by paraphrasing – using different words but
communicating the same idea. Then you would still need a citation:
A recent study shows that students struggle with APA style (Jones, 2003, p.199), but
she does not offer an explanation as to why.
You could also mention the author and date before the quote and do your citation differently:
According to Jones (2003), "Students often had difficulty using APA style, especially
when it was their first time" (p. 199).
There are other ways of doing this. You can consult the APA manual on Blackboard for
details. Just remember that a failure to properly cite sources is Plagiarism, which is not
allowed at The Hague University.
Whether you do it on purpose or by accident (by being sloppy with your sources) plagiarism is
a serious offence and it can stain your career. Making someone else’s work appear as if it
were your own is a form of intellectual theft and The Hague University considers it fraud. In
2011, Germany’s Minister of Defence was forced to resign in disgrace after it was discovered
that he had used sources in his doctoral dissertation without crediting them. This resulted in a
scandal that irreparably damaged people’s careers.
Guidelines for avoiding plagiarism
Follow the guidelines given here and you will avoid being guilty of plagiarism.
Plagiarism of Language
1. Use your own words and sentence structures when writing your paper, even when you
are writing about the ideas of others. A good way to start is to put things in your own
order.
2. When paraphrasing (putting someone else's ideas into your own words), avoid using any
words from the original unless they are essential technical terms.
3. If you use any of the original words from a source, you must acknowledge them by
enclosing them in quotation marks. It is still regarded as plagiarism if, without quotation
marks, you use some of the original words and phrases from a sentence and change
others. Also, it is still regarded as plagiarism if you keep the sentence structure of the
original and change all the words to synonyms.
Plagiarism of ideas
1. Acknowledge all ideas taken from other writers, either in a parenthetical reference or as a
part of the sentence describing the ideas. This does not apply to ideas and information
that can be considered common knowledge.
In order to get a better understanding of these guidelines, study the following examples of
how a text might be used.
Original text from P. Copperman (1980). The Literary Hoax, p. 65:
65
The second problem would have guaranteed the failure of the new math even if the first
problem had not existed. The overwhelming majority of elementary-school teachers have
had inadequate training in mathematics, and thus did not understand what they were
expected to teach. A program that attempts to transmit knowledge not possessed by the
teacher is doomed to fail. As this fact became clear to curriculum directors and textbook
publishers across the country, they compounded their error by attempting to make the
new math teacher-proof. This involved developing self-explanatory materials and
mechanical, repetitive techniques which were based on underlying mathematical
principles. Unfortunately, the new techniques were far more complicated than the old
ones had been, the teachers still didn't understand what was going on, and an entire
generation did not learn how to compute.
Possible uses of this text:
Example 1
A program that attempts to transmit knowledge not possessed by the teacher is doomed to
fail.
This is PLAGIARISM (This should be in quotation marks and followed by a reference.)
“A program that attempts to transmit knowledge not possessed by the teacher is doomed to
fail” (Copperman, 1980, p. 65).
This is NOT PLAGIARISM (The quotation marks and reference make this an acceptable use
of the original).
Example 2
A course that attempts to transmit knowledge not possessed by the teacher will never
succeed.
This is PLAGIARISM (This is patchwork plagiarism; a few words are paraphrased but most
are from the original, the sentence structure is also from the original, and there is no
reference.)
Success is impossible for a course “that attempts to transmit knowledge not possessed by the
teacher” (Copperman, 1980, p. 65).
This is NOT PLAGIARISM (The quotation marks around words from the original; the changed
sentence structure for words that are not quoted; and the citation/reference make this
acceptable.)
Example 3
A course that tries to convey understanding not held by the teacher is fated to be
unsuccessful.
This is PLAGIARISM (This is plagiarism because the original sentence structure has been
kept, even though the writer has used synonyms to replace most words, and there is no
reference.)
If the instructor does not have the knowledge that the student is meant to learn from a course,
then the course will never succeed.
This is NOT PLAGIARISM (This is acceptable because it is a full paraphrase with original
words and sentence structure changed, of an idea that is common knowledge in the field of
education.)
66
Example 4
Without the first problem, the second one would still have been enough to stop the new math
from working.
This is PLAGIARISM (Even though this is a full paraphrase, it is plagiarism of Copperman's

idea from the first sentence of the extract because the idea is not common knowledge in the
field of maths education, and there is no reference.)
Copperman claims that, without the first problem, the second one would still have been
enough to stop the new math from working (1980, p. 65).
This is NOT PLAGIARISM (This is acceptable because it is a full paraphrase and

Copperman's own idea has been clearly attributed to him.)
Without the first problem, the second one would still have been enough to stop the new math
from working (Copperman, 1980, p. 65).
This is NOT PLAGIARISM (This is acceptable because it is a full paraphrase and

Copperman's own idea has been clearly attributed to him.)
For guidelines on how to document and refer to sources you should see the official APA style
manual. A brief summary has been included in the Quick APA Reference offered in Appendix
2 of this manual. If you have any doubts about this matter, talk to your instructor. He or she
will be able to guide you in the appropriate use of sources.
Seminar 5
In this class:
• You will discuss your assignments with your teacher and your peers,
• You will identify common mistakes and problems using the marking sheet,
• You will correct the mistakes.
You will have the opportunity to discuss your on-going research and to ask for help if you
need it.
Using the provided marking sheet, you will read and evaluate the assignment of a classmate.
67
All students should do the following in preparation for Seminar 6:
1. Study the Example Report on Blackboard
An example of a successful student report has been posted in the Blackboard course. Take a
look at the report, compare it to what you now know about APA style reports and be prepared
to discuss it in your seminar.
2. If you haven’t already: Start Writing Your Report
If you have paid close attention to the structure of an APA Style Report, you will have noticed
that it is very similar to your Research Plan (both start with a clear statement of the question,
followed by an explanation of research methods. If you have a solid research plan (and you
should) you basically have the first two sections of your report (Introduction and Methods)
ready to go.
If you haven’t started your Research Report yet, now would be a good time. Open up a blank
document and put in the different headings (Executive Summary, Introduction, Methods, etc.)
and, using your research plan, add whatever you can to the first two sections. If you have
enough desk research results, you may even want to go further.
Print and bring whatever you have to class for in-class peer review and discussion.
3. Contemplate your Conclusion
While you work on your report, think about what you’ve learned about analysing information
and reaching a conclusion. Then, look over the information you’ve gathered and see if you
can spot any patterns or trends. Do you think you see the answer to your question? If so,
write it down and make a reasoning tree to test it out for logic.
Week 6
Lecture: Finalising Your Findings

In this class:
• You will review some of the most important aspects of report writing
• You will learn how to summarise your research
• You will learn how to package your report professionally
A. Writing Results vs. Discussing them
One part of report writing that many students struggle with is understanding the difference
between writing about their RESULTS and writing a DISCUSSION of those results. As such,
here is a little review:
The Results
The trick to writing up your results section is to focus on

the results themselves. You just want the facts – what
information did you find in your desk research? If you did
interviews, what did people say about this subject? Don’t
interpret those results or make any analysis of what they
mean yet. All you have to do is report them.
Some researchers like to structure their results by

research (sub-)question. Others prefer to do it by method.
You have to find a system that works for you, but please
remember the following golden rule:
1. Be brief
You do not need to describe everything you found in great detail. You don’t have time for that
and neither does your reader. The goal is to summarise what you learned. You will want to
include some quotations, preferably those related to the most important or interesting
information in your report, but you have to keep it simple.
Do not, for example, include full transcripts of interviews and detailed statistical information in
your report unless it’s absolutely necessary. You can and should include these in the
appendices, but not in the report itself. A simple summary and a few quotes will suffice.
For Example:
If you were doing market research to find out if a new product is going to succeed, you would
probably have used several methods. You would have used desk research to find out how
much it would cost, etc. and to get a better idea of the market (are there competitors, etc.).
You would probably also do some surveys, focus groups and interviews to determine whether
or not people would be interested in the product.
The results section of the report would look something like this:
Results
Desk research on the logistics of making the product have shown that it would cost 10 million
euros per year to produce the product, etc., etc.
Research of the marketplace has shown that there are two similar products available, etc.,
etc.
In the surveys that were conducted among potential consumers, 100 people were surveyed.
Fifty were men and fifty were women. All were between the ages of 20 and 30, within the
target market for this product. Twenty percent of the respondents said they would buy this
product and 80% said they would not buy it.
Eight of the ten potential customers who tested the product in a focus group said they would
not buy it because they did not like the colour.
In a personal interview, John Smith, an expert in this field, said, “this product is not going to
work. Your asking price is too high” (John Smith, personal interview, date).
Please note that you do NOT need to report back on every single detail of what you’ve found
in your research. Instead, you should SUMMARISE the most important findings. This is easy
to do if you keep the following in mind:
1. Use your Research Questions to guide you
The whole point of doing research is to answer your questions, so when looking over your
results, you will want to find the information (in your desk research and interview) that is most
useful towards answering those questions. Then, summarise that information and put it into
your report.
It’s a good idea to go through all your notes and your interview transcript and HIGHLIGHT
anything that is relevant towards answering your questions.
2. Use your Annotated Bibliography
If you’ve done your job right, your Annotated Bibliography should have summaries of all your
sources, making it easy to locate the information you need.
The Discussion
Once you have found and organized all of your results together, you are ready to analyse
your data by going over all the information you have found and looking for the following:
1. Relationships between what different sources said
Do multiple desk research sources say the same thing? Did your interview(s) provide
a similar point of view?
2. Contradictions between what different sources said
Are there different points of view? Do your sources have different arguments on your
question? Is there a difference between what you found in your desk research and
what you found in your field research? If so, what does that difference mean?
3. Things that stand out
Did any of the information strike you as especially interesting or useful? Did you learn
something new? Did you encounter anything odd?
Then, all you have to do is briefly report on what you think is interesting or significant about
your results, keeping in mind that this Discussion section is usually shorter than the Results
section. This is a necessary step towards reaching a conclusion, but before you do,
remember your CRITICAL THINKING skills and ask yourself one more question:
Is this information trustworthy and logical?
70
Example
Continuing with the above example, the Discussion section of the market research report
might look something like this:
Discussion
The cost of manufacturing this new product are rather high, which forces the company to
charge a high price for the product in stores. However, expert John Smith’s assessment that
the price will drive customers away demonstrates problems with this pricing strategy.
One fact that stands out from the research is that while the focus group participants said they
did not like the product’s colour, the 100 potential customers who were surveyed were
positive about the product’s appearance, which suggests that the focus group, as a small
sample, does not present the most accurate picture of what all consumers think. The colour
might not be a problem.
etc.
Coming to conclusions
Once you have provided an overview of your information and have analysed it carefully with
your research questions in mind, you’re ready to take the next step and MAKE UP YOUR
OWN MIND. Decide what you think based on the information, evidence and examples you
have in front of you and ANSWER YOUR RESEARCH QUESTION(S).
In this example the conclusion would likely be something like this:
Conclusion
Based on the analysis of the information obtained for this research, the answer to the central
question of this report is that the product is not likely to succeed in its current form and should
not be brought to market yet.
Once you’re able to reach a conclusion and some recommendations, you’re almost ready to
finish the report. But, before you run your victory lap, there is ONE MORE STEP: You need to
evaluate your conclusion using Critical Thinking skills to make sure that it is really supported
by your information.
To test this conclusion, you could make a reasoning tree to check the logic All you have to do
is start with your conclusion and see if it’s supported (“My conclusion is X because…”) It is
highly recommended that you try this on your final report.
Using this example, the reasoning tree would look like this:
Product X is not
likely to succeed
Because experts Because focus

Because the costs think the price will groups did not like
are too high.
deter customers. the colour.
71
B. What to do if things don’t add up
Research is a complex and scientific process but in science, there are no guarantees and no
absolutes (especially since, as you know now, there is no objective truth but only
argumentation that can be critically evaluated) and so it is important that you be aware that
sometimes, even after you’ve made the best possible research plan and gathered good,
credible sources, sometimes you may not be able to find a clear answer. What should you do
about that?
The first thing to do is to remember not to panic. Sometimes, despite the best intentions and
preparations, you may find that you cannot (completely) answer your research questions.
That is not necessarily a bad thing. Sometimes that’s the answer. Even in mathematics,
where you expect an answer, it isn’t always possible to find one, and research is the same.
So, what do you do when you can’t find an answer?
That’s easy: try to figure out what went wrong and make that part of your conclusions and
recommendations. It is quite common, for example, for a research paper to end with the
recommendation that further research needs to be done.
Be aware
If, in your research for this course, you are unable to reach a satisfactory conclusion, you will
not be penalized if you can demonstrate that you know what went wrong and why, and that
you can reach some kind of conclusion and recommendation, even if that recommendation is
for further research.
And while we’re at it, one more tip: it’s quite possible that when you do research, you don’t get
the answer you were hoping for. Try not to be disappointed and see the research for what it
is: a valuable learning experience that will send you in a different direction to get new ideas.
Research, like science, is a process of trial and error.
C. Summarising made simple
As you know, the last thing you write when finishing a research report is an Executive
Summary, a short summary/abstract that should summarise your entire research paper in
such a way that a person could read the summary and know everything they need to know
without reading any further, just like the lead paragraph of a news story. It takes some writers
a while to get used to the idea, but this is the standard. In the professional world, many
people simply don’t have time to read the whole report. Decision makers often rely on the
summaries.
This may make you wonder: why do I have to write the report then? But hopefully, the answer
will be clear if you think about it for a few minutes. You have to write the report because you
have to document your research as you go, just like showing your work on a math test, and
without it, you wouldn’t know what to say in your summary.
Criteria for an Executive Summary
An executive summary of a research paper must always include:
• A statement of the purpose of the report

• The central question and its importance
• A summary of the research methods
• A summary of the findings
• An indication of the conclusion & recommendations (if appropriate)
Often, a summary will read like this:
72
Executive Summary
The purpose of this report is to document research conducted at University X on the topic of
………………….., because of …………………… The central question of this research is/was:
“…………………………………………..?” In order to answer this question, different research
methods were chosen. ……..(description of the methods)…… The research showed that
……..(summary of the findings)……. This led to the conclusion that ……..(conclusion)….. and
it is recommended that…….(recommendations).
D. Proper formatting for your Research Report (or any assignment at HHS) in Microsoft
Word
All assignments at The Hague University of Applied Sciences are typed and formatted
according a few general standards. These standards help you to hand in professional and
presentable work that is attractive to look at and easy to read. If you’re using Microsoft Word,
here are some tips on how to set up your document – but please keep in mind that different
versions of Word look different and some buttons are different locations.
General Rules for Fonts and Spacing
There are a few rules and guidelines for the text in your assignments.
• Use fonts that are 11 or 12 points in size

• Use 1.5 line spacing
• Justify the text
In general, everything you write should be typed, using an attractive font in size 11 or 12 –
smaller fonts are hard for teachers to read and larger fonts waste ink and paper. It’s best to
avoid fonts with serifs (the curly tips on some letters) like Times New Roman because they
are harder to read. A sans-serif font like Ariel, Helvetica or Calibri is easier on the eyes. You
can set the font by going to the FORMAT menu in Word or by using one of the toolbars at the
top:
Once you have selected your font, you need to set the Line Spacing (the amount of blank
space between lines of text). Microsoft Word and other Word Processers use single-spacing
as a default, but this is not a good idea for your report. Single-spaced writing is harder to read
and – especially important for your teachers – it’s
harder to write on. Set your document to 1.5 spacing.
It leaves enough room for comments and is easy on
the eyes, but isn’t wasteful (double-spacing wastes
paper).
You can set the line spacing by going to the FORMAT

menu in Word and selecting the PARAGRAPH
options, but many versions of Word have special
button for this in the HOME toolbar.
Another important thing to do, if you want your assignment to look nice, is to justify the text.
By default, Word lines text up on the left, but it also allows you to centre your text, to line it up
to the right, or to Justify it. The last option is best, because it makes the text line up nicely on
both sides and gives the document a professional look.
73
You can justify text by going (once again) to the FORMAT menu
and selecting the PARAGRAPH option, but most versions of Word
have a button for this on the main toolbar.
Example Texts
This is an example of a ‘bad’ text for an assignment. It is letters that are hard to read and they are too small. Plus, to make
matters worse, it’s single spaced, which is hard on the eyes. You want your assignments to be easy to read and easy to grade.
Keep your teachers in mind. It’s not justified either, and that gives the text a jagged edge that is unattractive.
This text is much easier to look at and to read. It’s a smooth font, in a good size for
reading on paper and the 1.5 line spacing gives people’s eyes a break – while also
making it easier for your teacher to write comments. Justifying the text also has a
nice effect – by stretching it smoothly across the page, it makes the text more
attractive.
Please note that when you use this style, the text runs from side to side without any
tabs, so it is important to put a blank line between paragraphs (simply hit ENTER
twice) so that it’s clear when one paragraph begins and the other ends.
Using Word to Create an Automatic Table of Contents

Research reports, like many documents, almost always have a Table of Contents. These
used to be difficult to make, because you’d have to count by hand, but Word will do it for you
automatically if you follow a couple of simple steps.
• Create headings
• Insert a Table of Contents
In order to make a Table of Contents, you need to make sure your document is divided into
sections and each of these sections should start with a heading. In an APA Style report, use
the title of each section (Introduction, Methodology, Results, Discussion, Conclusion, etc.) as
headings.
You can create a heading by highlighting the text and selecting the kind of heading you want
from the STYLE menu that is part of the HOME toolbar in most versions of word, or you can
use the text options in the standard toolbar.
If you play around with the options, you can change the style of the headings. You can also
create sub-headings and other labels for your document.
74
Once you have headings, making the Table of
Contents is easy. Simply go to the part of your
document where you want the table and then
insert it by using the Table of Contents Button,
which is located under REFERENCES in
some versions of Word and under
DOCUMENT ELEMENTS in others. There are
different styles to choose from.
This will give you an automatic Table of Contents.
It’s important to remember that the Table does not update itself automatically. If you make
changes to your document, you need to update the Table of Contents by clicking on it and
choosing the UPDATE FIELD option.
Microsoft Word and APA Style Referencing

Microsoft Word can also help you with your in-text referencing. You can use the programme
to:
• Insert and Edit Citations
• Create a Bibliography
The buttons are slightly different in different versions of the programme, but all recent
editions of Word include referencing tools. On most Windows versions, there is a
REFERENCES tab on the ribbon. On a Mac, you’ll find it under DOCUMENT ELEMENTS.
Windows
Apple
75
Once you find the tool, you open a field where you can enter information about the author,
publisher, website, etc. Then Word will automatically add a citation.
You can edit that citation by clicking on it, so

that you can add the page number, or, in
some cases, remove information you don’t
want to show.
You only have to enter the information once

and if you use the source again, you can
simply insert the citation from your list and
then edit the citation to add the page number.
Once you have added all your sources and citations, you can ask word to make your List of
References for you by simply inserting a Bibliography from the REFERENCING tab
(DOCUMENT ELEMENTS on a Mac). It will generate a full list with minimal effort.
Please note that, because you are making an Annotated Bibliography for this course, you will
still need to do some editing.
These tools will not do your thinking for you but they can make your life easier. Whether you
use them or not is up to you, but no matter what, you must always include in-text referencing
and a list of references in any writing assignment.
76
If you need help
Microsoft Word has dozens of useful functions you might not be familiar with, and it will take
you awhile to truly get to know the programme. Luckily, the internet is full of free tutorials for
Word and similar programmes. If you run into difficulties, or if you need help figuring out how
to do something in Word, the fastest way to get help is to do a little ‘desk research’ – Just
Google “How do you xxxxxxxxxxx in Word?” – sometimes it helps to mention the specific
version you’re using.
C. Practice Formatting and time for questions
If you brought your work with you on a computer or tablet, you will have the opportunity to
practice formatting your report properly.
77
Seminar 6
In this class:
• You will talk about coming to conclusions
• You will discuss your Research Papers with your Classmates and Teacher to
get useful feedback.
• Your teacher will provide instructions about handing in the final assignment.

A. Contemplating Conclusions
In your homework, you were asked to begin analysing your information to see if you are close
to a conclusion. Be prepared to show your analysis to your teacher and class and to discuss
your possible conclusion.
B. Peer Review of your (draft) paper
All students should have written (some of) their research paper and should have it in class. In
pairs or small groups, you will review each other’s work using the marking sheet. Your
teacher will check your work, answer questions and provide feedback where needed.
Focus your review on the following points:
1. Structure and Formatting – Is the paper structured in APA Style with the
appropriate sub-headings and format?
2. Introduction – Does the report have a clear introduction that states the central
research question and relevant sub-questions?
3. Methodology – Does the report explain the research strategy, including an

explanation of what choices were made and why? Is it specific?
4. Body – Is the body organized by subquestions? Do the subquestions contain:

hard facts, arguments from the literature and an original argument that answers
the main research question?
5. Conclusion – does it clearly spell out your answer to the research question? Does
it discuss the relevance of the answer? Does it acknowledge limitations?
6. APA Referencing – Have the three simple steps (signal phrase, quote or
paraphrase & citation) been used to introduce and reference examples and
quotes? Are all three steps present, including signal phrasing?
7. Annotated Bibliography – Is there an Annotated Bibliography? Does is included

proper APA references for each source? Does it summarise and evaluate the
credibility of each source?
As you know from this course, the peer review process is an essential aspect of real-life
research. Before any major scientific finding can be published, it must first be tested by other
scientists to prove its validity and credibility.
But that’s not the only reason to conduct peer reviews. Showing your work to someone else is
an incredibly valuable step in the process for a number of reasons, not the least of which is
that another pair of eyes might see something you’ve missed. They might discover problems
in your work that can be corrected before you hand it in, or they might have suggestions for
how to improve the report. A second reader is able to TAKE DISTANCE and look at things
from a neutral point of view, which is difficult to do with your own work, especially if you have
invested a lot of time and energy in it.
At the same time, you can learn a lot by reviewing someone else’s work. You might pick up
some useful ideas for yourself, and, at the same time, if you practice finding errors in
someone else’s work, it will get easier to find them in your own work.
Doing Peer Reviews with Research Drafts
For this peer review exercise, each student must should pair up with another and exchange
drafts. Do not discuss or explain your draft, simply exchange them and begin reviewing them
immediately. Answer the questions below and then, when both teams have had enough time
to review the work thoroughly, give the papers back together with your comments.
Your teacher will come round to observe the discussion.

When receiving feedback on your paper, resist the urge to defend yourself against criticism.
Simply listen and consider that feedback carefully. Then, look over your work and see if you
feel that the feedback was useful.
B. Preparation for final deadline and Course Criteria
Your teacher will discuss your assignment deadline and final preparations. Your teacher will
review the grading criteria and may ask you to apply it to the papers you’ve just reviewed.
C. General conclusions to this Course
By now, you have studied and practiced the techniques involved in basic academic research,
step by step. In upcoming semesters, you will learn even more research techniques, but the
process itself is largely the same. Learn it well, because you will need to do this over and over
again throughout your education and career, and remember that it consists of several simple
steps:
1. Deciding what want to know and why.

2. Choosing the best way to find the information.
3. Finding the information and making sure it’s trustworthy, logical and useful.
4. Analysing what you’ve found and answering your questions.
5. Decide what to do next and make recommendations.
It really is that simple.
Your teachers wish you luck on your assignment and we hope you will use this manual
for reference in the future.
79
Appendix 1: Quick APA Reference
APA style was covered in detail in this course. A detailed APA Style manual is available on
Blackboard and students are encouraged to use that manual (and this one) for reference, or
to consult www.apastyle.org, but here is a quick reference guide summarizing the main
points.
APA Formatting
An APA style report consists of the following elements:
1. Title Page – A cover sheet that includes the

title of the report, as well as the names (as well as class and ID numbers) of its
authors and the title of the course.
2. Executive Summary (Samenvatting) - A section that

offers an overview of the whole report, including its conclusions and
recommendations. Can be automatically generated in Microsoft Word
3. A Table of Contents – a short list of each section that provides the page
numbers. Can be automatically generated in Microsoft Word.
4. Introduction (Introductie) - Where the topic is introduced, as are the central

research question(s) and hypotheses (if applicable).
5. Methods (Methodiek) - A section in which the

methods used to find information to answer the research questions are
explained and justified.
6. Results (Resultaten) - A section in which the

information gathered through research is reported factually, without any
observation or interpretation. It’s usually organized by sub-question or by
research method.
7. Discussion/Analysis (Discussie) - A section in

which the findings are analyzed.
8. Conclusions (Conclusie) - A section detailing the

answer to the research question(s) based on the information analyzed. (What
we know now).
9. Recommendations (Aanbevelingen) - Though not

always included as a separate section from conclusions, this is where advice is
given based on those conclusions.
10. List of References (Referenties/Bronnen) - This

refers to an APA style list of all the sources used in the report in Alphabetical
order.
11. Appendices – Any information or documents that

do not fit in the body of the report (like an interview transcript or raw data) can be
included as an appendix.
Documenting Sources
APA style also includes rules for referring to sources and documenting information that is
used in research reports and essays. The specific rules are explained in the school's APA
guide (an in this course manual), but here is a brief summary of how in-text references should
work. Whenever a report writer wants to use words or ideas from someone or somewhere
else, they must follow three steps:
1. Use a signal phrase - Writers must introduce borrowed words or ideas by using
phrases like "According to" or "As Smith said," in order to be clear about where the
information came from.
2. Quote or paraphrase - The signal phrase should be followed by a direct quote (in
quotation marks - "quote") or a close paraphrase (a summary in different words from
the original).
3. Citation - The quote or paraphrase must be followed by an APA citation in
parenthesis. This parenthesis should not include detailed information, just the author
of the source, the date, and the page or paragraph number. If no author is known, the
citation should include the title, date and page or paragraph number.
For Example
According to author Bob Smith, "APA style is easy to use" (Smith, 2011, p. 12).
Or:
According to Smith, APA style is not difficult (Smith, 2011, p. 12).
Or:
As author Bob Smith (2011) said, "APA style is easy to use" (p. 12).
Please note that punctuation comes after the citation.
For more detail, review the course and see the APA Style Guide or consult
www.apastyle.org.
81

Intro To Research Student Manual 2017-2018

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Intro To Research Student Manual 2017-2018

Uploaded by

Copyright:

Available Formats

INTRODUCTION TO RESEARCH SKILLS

How to use this Manual .................................................................................................... 3

One of the great challenges to researchers is the difficulty of interpreting information –

The Research Process

Relationship to other skills

Assessment and Feedback

If possible, example reports will be posted on Blackboard for reference.

Topics for European Studies

Suitable topic areas include:

Political/economic/social issues – issues related to political philosophies and parties, national

Here are some more specific examples of suitable research topics:

2. The influence of globalization (in terms of business, media or other factors) on

6. The political/cultural/economic effects of immigration on The Netherlands/Any other

Lecture: Introducing Research

Lecture Outline & Reference Material

Step 1. Figure out what you want to know and why

Step 2. Select research methods

Step 4. Analyse Your Data

Step 5. Make Conclusions (and Possibly Recommendations)

3. Reasons for Researching

1. Essays and Presentations

2. Policy Documents, Proposals and Plans

4. Research Papers in APA Style

An APA Style report includes:

1. An Introduction – in which the topic is introduced and the central research

5. Getting Started: Creating a central question

6. Getting Ready to do Research

Formulating the Research Question

Now you’re ready to do some real research.

7. Introducing Desk Research

In Applied Research, it is common to do a lot of desk research to find a ‘theoretical

Tips for getting started with Desk Research

1. Use your notes

2. Consult general reference sources first.

Here are some general sources worth considering:

Encyclopaedia Britannica is one of the most widely read

About.com is a popular North-American website that provides

What about Wikipedia?

It is true that YOU CANNOT TRUST EVERYTHING YOU FIND

Either way, you should know that IT IS POSSIBLE TO USE

How to use Wikipedia in a responsible and academically acceptable way

2. Look for references and links.

4. Use Wikipedia to find keywords and follow through on links.

You’ll also find a ‘References’ section that might include useful

Many articles even recommend ‘Further Reading’ to point your

1. Google is not the Information business. It’s in the Advertising business.

2. Google is not capable of finding most of the content on the Internet.

Risks associated with search engines

If you were to search for the same keywords on

This is because each search engine uses different

1. Use Meta-Search Engines

There are several Meta-Search Engines out there, including Metacrawler

2. Consult the media

3. Use Search Engines more wisely

If you’ve done your library training,

It might seem a little overwhelming at first,

8. Reading Articles & Information Sources

Encyclopaedias and Wikis

General information sources like Wikipedia are a

Most informative websites follow the same format,

News articles are a good source of general information

Academic Journal Articles