Critically evaluating information (Social Sciences)

Name, Libraries’ Learning Development Team (Information Skills)

University of Nottingham Libraries
Open Programme 18/10/22
 Learning outcomes

By the end of this session, you should be able to:

▪ explain what is meant by critical evaluation
▪ comprehend why critically evaluating all information you read is important for
your academic work
▪ recognise that a given scholarly work may not be the only (or most)
authoritative opinion and that differing viewpoints are central to academic
debate
▪ approach published material with a sceptical stance, open mind, and
awareness of biases, both personal and in the literature
▪ employ critical skills to assess the authority, reliability and validity of published
literature
What is critical evaluation and
why is it important?
 What do we mean by ‘critical evaluation’?

In a HE / research setting, critical evaluation can be

defined as:
“the structured assessment of the strengths
and weaknesses of a paper, thus allowing you to
make an assessment of its quality and ... how it
helps you to answer your research question”
(Helen Aveyard, 2019, p.102)

Aveyard, H. 2019. Doing a literature review in health and social care: a

practical guide (4th ed.). McGraw-Hill Education/Open University Press.
Why is critical evaluation important?

It enables you to:

▪ Make informed judgements based on the
strengths and weaknesses you identify
▪ Develop your understanding of research
methodologies and their appropriate use
▪ Identify potential bias, including your own
▪ Assess if research is valid and reliable
▪ Comprehensively assess a whole paper
▪ Relate what you are reading to your topic or
research
 Critical evaluation and your degree!

Bachelor’s Degree with Honours

▪ To critically evaluate arguments, assumptions,
abstract concepts and data … to make
judgements

Master’s Degree
▪ To evaluate critically current research and
advanced scholarship in the discipline … and
develop critiques

University of Nottingham. The Quality Assurance Agency Qualification

Framework [Online]. [Accessed 23 September 2021].
 Battling bad science

Dr Ben Goldacre: Medical doctor, academic, author, broadcaster

“Real science is about critically appraising the evidence…” (see transcript 01:30)
How do you critically evaluate
information?
 Critically evaluating information

Whether information has been published in a

book, a journal, or on a website, ask yourself
these questions as you read:

▪ Relevance?
▪ Who?
▪ Where?
▪ When?
▪ Why?
▪ How?
 Relevance?

▪ What are the requirements of your Requirements

assessment? Is what you are reading
relevant?
Detail
▪ Is the information too detailed, too specialised,
too general, or too simple for your needs?
Relevance
▪ Does it relate to countries, areas, subjects,
contexts etc that you’re not interested in?
Context
▪ Look at the title, abstract, keywords and
descriptors to assess relevance
Review title,
abstract and
Keywords
 Who?

▪ Is the author(s) an expert in their field? How do Expert

you know?
▪ Are they affiliated to a reputable organisation? Affiliation
▪ Is their first publication or have they published
regularly in this area?
Who
▪ Has their research been cited by other authors?
Was this for good reasons or bad? Cited
▪ How was the author(s) funded? Could this
have introduced bias?
Funding
 Who? What to look for…

Author affiliations are

often
listed at the
beginning or end
of an article

Do a quick author search on

NUsearch. Browse the results –
simi lar topic? Cited by others?
Look for the ‘Citing this’ button:

Funding or conflict of interest information may be at In the full text press

the beginning, at the end, or within a paper CTRL + F and search
for fund or sponsor
 Where?

▪ Has the research been published in a Recognised

journal
recognised journal?
▪ Is it a peer reviewed journal? Peer
reviewed

▪ What is the publication’s ranking within the

discipline?
Where
▪ What credentials do the editors have?
Ranking
▪ Has the research been disseminated beyond
the publication? Has it made the news?
Made the
news?
 Where? What to look for…

NUsearch will
show if an article
is in a peer reviewed journal.
Look for this icon:

You can often find a journal’s

impact factor and ranking on its
own ‘About the journal’ page

https://www.tandfonline.com/journals/rics20
 When?

▪ What’s the date of publication?

Current
▪ Is the research current? research
Or has it been superseded and can now be
regarded as out of date? Seminal
Was there a significant gap between the research
research undertaken and publication?
▪ If older research, is it regarded as seminal When
research within the discipline?
Scholarly
▪ How does the research fit within scholarly conversation
conversations to date?
▪ Has the research been cited by others? For Citations
good reasons or bad?
 When? What to look for…

Where does the research sit in relation to other

evidence? Refer to the introduction or literature review

Use multiple search bars

on Web of Science to see
how many times a paper
has been cited

You can access

Web of Science via
NUsearch
 Why?

▪ Has the rationale or purpose for the Purpose /

Rationale
research been made clear?

▪ In your view, are these reasons justified? Justification

Why?

▪ Has the research been sponsored by Why

anyone? If so, what kind of bias does this
bring? How can you tell?
Sponsored

▪ Has the publication been invited? (e.g.

guest editorial, a paper for a conference? If
Instigation
so, has this introduced bias?
 Why? What to look for…

Are the aims, objective, rationale,

purpose of the
research clearly stated?

You can usually find this in the

abstract and introduction

If funded, what role does the funder

play?
Does their involvement add a conflict of
interest or potential bias?
 How?

▪ What methodology was employed? Was it

appropriate? Methodology

▪ What was the sampling criteria? Was the

sample size appropriate? How was the Sampling
sample group selected / recruited? criteria

▪ Was the research conducted ethically?

▪ Have any limitations been discussed? If so, How
do these have an impact on findings?
▪ What kind of analysis has been undertaken? Ethics /
Limitations /
Are findings valid and reliable? Bias

▪ What evidence has been used to underpin

Analysis of
arguments or results? What sources have findings
been cited?
 How? What to look for…

Find information about

methodologies, ethics,
sample groups,
data analysis, findings
and limitations
in the Methods,
Results and
Conclusion /
Discussion sections

Tip!
In the full text press
CTRL + F
and search for e.g.
method, analysis,
ethics, sample or
limitations
Research papers in the Sciences

Introduction
• Purpose, aim, rationale and justification of research

Methods
• Research methods, sampling criteria, ethical concerns, validity and reliability

Results
• Findings, methods of data analysis

Discussion
• Implications for practice / future research, limitations, bias
Resources to help with critical
evaluation
 Studying Effectively: Evaluating information

Visit the Studying Effectively webpage.

 Critical evaluation checklists: Research papers

Cornell University: Critical Critical Appraisal Skills

Appraisal Programme (CASP)
 Critical evaluation checklist for the web: CCOW
Credentials
▪ Who created the information?
▪ Are they a trustworthy source?
Claims
▪ Is the information they are providing accurate and up-to-
date?
▪ Use experts and lateral reading to find out more
information
Objectives
▪ Why was this information created– to inform, convince or
sell?
Worldview
▪ Consider the author’s worldview and how this can
influence the information presented?
▪ Consider how your own worldview can also influence your
perception of this information
Tardiff, A. (2022). Have a CCOW: A CRAAP alternative for the internet
age. Journal Of Information Literacy, 16(1), 119-130. doi:10.11645/16.1.3092
 Reading critically: Online resource

Go to:
Reading critically
 Finding help online

▪ Browse your subject guides to discover key resources in your area and develop
your literature searching skills
▪ Masters students: sign up to On Course for Your Masters on Moodle
▪ Join a Librarian for Literature Searching Support

▪ Support from Learning Development (Academic Skills)

▪ Watch out for our Live chat pop-up on Library web pages

▪ Email library-helpline@nottingham.ac.uk with any enquiry

 Online information skills sessions

Easily browse and instantly join teaching sessions on:

▪ Using library resources
▪ Literature searching
▪ Database searching
▪ Critical evaluation
▪ Systematic reviews

No need to pre-book. All sessions are taught live online via MS Teams, by the
Library Teaching and Learning Support Team, on a rotating schedule designed
to fit your timetable!

https://www.nottingham.ac.uk/library/studying/training.aspx#info
Thank you for joining us!