You are on page 1of 124

Communicating Your Research:

Introduction

Christine Espin
Maria Sherwood Smith
Leiden University

Communicating Your Research September 8, 2020

Discover the world at Leiden University


Outline
• Introduction to Writing and Presenting
• 4 Knows – and No-no’s – of writing and presenting
• Writing an introduction to an article

Discover the world at Leiden University


4 “Knows” and “No-no’s” of
Writing and Presenting

• Know yourself w i ng
no ence,
• Know your “stuff” o : N
t k
o udi
’s uff, a e or
- n t t
No self, s to wri
• Know your audience r
you or how resen
t
p
• Know how to write /
how to present

Discover the world at Leiden University


Know yourself

Discover the world at Leiden University


What is your writing / presentation
style?

Work within your


style
Discover the world at Leiden University
Regardless of your style: A few tips
for writing and presenting . . .
• Tell a story: Communicate / teach
• Use figures / illustrations / pictures to help you
tell your story
• Keep your audience interested
• Writing: Make introduction coherent; make
transitions clear; use examples
• Presentation: Use anecdotes, humor, video/audio
clips, examples, activities

Discover the world at Leiden University


Know your “stuff”

Discover the world at Leiden University


Know your “stuff”:
• What you did
• Why you did it that way
• Why you did not do it a different way

• What you found


• What you did not find
• If this is what you expected to find or
not find

• What you could have / should have


done differently and . . .
• What you will do differently in the
future (based on all of the above)
Discover the world at Leiden University
Know your audience

Discover the world at Leiden University


To whom are you writing /
speaking?
• What do they know or not know?
• What do they believe or not believe?
• What research have they done that might:
• support your conclusions?
• contradict your conclusions?
• What are they likely to
• say in the reviews?
• ask during the presentation?

Discover the world at Leiden University


To whom are you writing /
speaking?
• What do they know or not know?
• What do they believe or not believe?

Discover the world at Leiden University


What do they know / not know,
believe / not believe?
• Organization of your introduction /
presentation
• Amount of detail provided and information
emphasized or deemphasized
• Use of terms / vocabulary
• Amount and types of explanations given

Discover the world at Leiden University


Examples

Discover the world at Leiden University


How much does your “audience”
know?

Audience: Readers, reviewers, listeners

Discover the world at Leiden University


Audience knows a lot about topic: little
information needed
• Curriculum-Based Measurement (CBM):
• System used to monitor the progress and evaluate the
effects of instruction for students with learning disabilities

Discover the world at Leiden University


Audience knows a little about topic: more
information needed
• Curriculum-Based Measurement (CBM):
• System used to monitor the progress and evaluate the
effects of instruction for students with learning disabilities
• CBM involves the frequent (weekly) monitoring of student
performance using brief measures such as reading aloud
from text or maze selection
• Scores from the measures serve as global indicators of
performance in areas such as reading

Discover the world at Leiden University


Audience knows nothing about topic: more
information needed
• Curriculum-Based Measurement (CBM):
• System used to monitor the progress and evaluate the
effects of instruction for students with learning disabilities
• CBM involves the frequent (weekly) monitoring of student
performance using brief measures such as reading aloud
from text or maze selection
• Scores on the probes serve as global indicators of
performance in areas such as reading
• Scores are placed on graphs that display progress
• Graphs: guide decision-making about instruction
• Students not progressing: modify instruction
• Students progressing: raise goal
• Examples
Discover the world at Leiden University
Sam’s CBM Progress Graph in Reading

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph in Reading

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph
Instruction phase 4

(Deno,
Discover the world at 1985;
Leiden2003)
University
Characteristics of CBM measures / scores

l Practical
l Inexpensive
l Easy to use
l Time efficient
l Repeatable
l Easy to understand
l Technical (scores)
l Reliable
l Valid
What measures / scores? l Sensitive to change
in performance

(Deno, 1985; 2003)


Discover the world at Leiden University
Characteristics of CBM measures / scores
Global indicators of performance in an academic area

l Practical
l Inexpensive
l Easy to use
l Time efficient
l Repeatable
l Easy to understand
l Technical (scores)
l Reliable
l Valid
What measures / scores? l Sensitive to change
in performance

(Deno, 1985; 2003)


Discover the world at Leiden University
Examples: CBM Reading Measures
Reading aloud
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another one: he graduated from high school. Dressed in his black cap and gown, he
shook hands with commencement speaker, ex-Viking, and state Supreme Court Justice Alan
Page. Then he took his diploma from Principal Linda Nelson.

Score: words read correctly Time: 1 minute

Maze selection
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another was / eat / one: he graduated from high school. Dressed be / in / or his black
cap and gown, he shook / carts / south hands with commencement speaker, ex-Viking, will /
and / buy state Supreme Court Justice Alan Page. Hole / Said / Then he took his diploma
from Principal Linda Nelson.

Score: correct choices Time: 2 minutes

Discover the world at Leiden University


What does your “audience” believe?

Audience: Readers, reviewers, listeners

Discover the world at Leiden University


Audience “beliefs” about what CBM
reading measures actually measure
• Curriculum-Based Measurement
• System used to monitor the progress and evaluate the
effects of instruction for students with learning disabilities
• CBM involves the frequent (weekly) monitoring of student
performance using brief measures such as reading aloud
from text or maze selection
• Scores on the probes serve as global indicators of
performance in areas such as reading

Discover the world at Leiden University


Audience beliefs about what CBM measures
actually measure . . .
Reading aloud: Measure of technical reading (word reading and reading
fluency), not comprehension
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another one: he graduated from high school. Dressed in his black cap and gown, he
shook hands with commencement speaker, ex-Viking, and state Supreme Court Justice Alan
Page. Then he took his diploma from Principal Linda Nelson.
Score: words read correctly Time: 1 minute

Maze selection: Measure of reading comprehension, not technical reading


Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another was / eat / one: he graduated from high school. Dressed be / in / or his black
cap and gown, he shook / carts / south hands with commencement speaker, ex-Viking, will /
and / buy state Supreme Court Justice Alan Page. Hole / Said / Then he took his diploma
from Principal Linda Nelson.

Score: correct choices Time: 2 – 3 minutes

Discover the world at Leiden University


“Global indicators” of performance

Scores increase overall reading performance is improving

Discover the world at Leiden University


How beliefs affect audience views of your
article/ presentation
• Article: Teachers’ use of CBM maze measures to
monitor the progress of students with reading
difficulties

Discover the world at Leiden University


How beliefs affect audience views of your
article/ presentation
• Article: Teachers’ use of CBM maze measures to
monitor the progress of students with reading
difficulties

• Audience Beliefs:
• CBM reading aloud is a measure of technical reading
(word-reading and fluency); cannot possibly be an indicator
of comprehension: Study is dumb!!

Discover the world at Leiden University


How beliefs affect audience views of your
article/ presentation
• Article: Teachers’ use of CBM maze measures to
monitor the progress of students with reading
difficulties

• Audience Beliefs:
• CBM maze is a measure of reading comprehension: Study is
good!!

Discover the world at Leiden University


How beliefs affect audience views of your
article/ presentation
• Article: Teachers’ use of CBM maze measures to
monitor the progress of students with reading
difficulties

• Audience Beliefs:
• CBM maze is a measure of reading comprehension: Study is
good!!
• BUT . . . Research has shown that maze is not a good
measure of reading comprehension: Study is dumb!!

Discover the world at Leiden University


How beliefs affect audience views of your
article/ presentation
• Article: Teachers’ use of CBM maze measures to
monitor the progress of students with reading
difficulties

• Audience Beliefs: Important to . . .


• Present evidence supporting claim that CBM scores valid
(good) indicators of general reading performance
• Works if audience is convinced by empirical evidence
• Otherwise: Study is dumb!!

Discover the world at Leiden University


To whom are you writing /
speaking?
üWhat do they know or not know?
üWhat do they believe or not believe?
• What research have they done that might:
• support your conclusions?
• contradict your conclusions?
• What are they likely to
• say in the reviews?
• ask during the presentation?

Discover the world at Leiden University


What research have they done?

Supports conclusions: Yaay!!

Contradicts conclusions: Blaah!!

Discover the world at Leiden University


What research have they done?

Supports conclusions: Yaay!!

Contradicts conclusions: Yaay!!

Discover the world at Leiden University


Contradicting results move us forward

• Why are there contradictions (methods,


population, etc.)?
• The answer is NOT:
• that the other research is poor / wrong / dumb

• How can we address these contradictions in


the future?

Discover the world at Leiden University


What are they (audience) likely to say in
the reviews / ask in the presentation?
• Question Challenge: As you write or before you
present
• Ask yourself EVERY question that is likely to be asked
. . and have the answer
• When appropriate
• Foreshadow (anticiperen and vooruitzien) the question
in your introduction / presentation
• Be honest . . . be open . . . don’t be defensive
• Science is a public affair!!

Discover the world at Leiden University


What are they (audience) likely to say in
the reviews / ask in the presentation?
• Question Challenge: As you write or before you
present
• Ask yourself EVERY question that is likely to be asked
. . and have the answer
• When appropriate
• Foreshadow (anticiperen and vooruitzien) the question
in your introduction / presentation
• Be honest . . . be open . . . don’t be defensive
• Science is a public affair!!

Discover the world at Leiden University


Example of foreshadowing
Although researchers and practitioners alike often
view reading aloud as a measure of technical
reading, and maze as a measure of reading
comprehension, within CBM, scores from both
are used as global indicators of general reading
performance, not as “measures of” technical
reading or reading comprehension.

Discover the world at Leiden University


4 “Knows” and “No-no’s” of
Writing and Presenting

üKnow yourself w i ng
no ence,
üKnow your “stuff” o : N
t k
o udi
’s uff, a e or
- n t t
No self, s to wri
üKnow your audience r
you or how resen
t
p
• Know how to write /
how to present
• Remainder of course

Discover the world at Leiden University


4 “Knows” and “No-no’s” of
Writing and Presenting

üKnow yourself w i ng
no ence,
üKnow your “stuff” o : N
t k
o udi
’s uff, a e or
- n t t
No self, s to wri
üKnow your audience r
you or how resen
t
p
• Know how to write /
how to present
• General tips

Discover the world at Leiden University


General writing tips: Bem’s article
• Which article should you write?
• The article that makes the most sense now that you have
seen the results
• How should you write?
• Accuracy
• Clarity: write simply and directly
• Shape of an article: Hourglass
• Move from broad to specific to broad (more later)
• For whom should you write?
• Less specialized readers: Your grandparents / parents /
neighbors (unless they are academics)
• Write and revise . . . write and revise . . . write and revise . . .
write and revise . . . write and revise . . .
Discover the world at Leiden University
Additional general writing tips
• ELIMINATE WORDS! (‘nuf said.)
• Use first person, “I” or “we” (don’t overdo it)
• Reduce language bias
• Avoid sexist language (use plural form)
• Call people what they want to be called (e.g., African American, Asian American,
etc.)
• People are nouns, their attributes are adjectives (e.g., students with learning
disabilities rather than learning disabled students)
• Avoid metacomments (“Now that we have talked about
metacomments, we can move onto parallel contructions.”)
• Use repetition and parallel construction
• Avoid jargon (unnecessary specialized terms)
• Use active voice rather than passive

Discover the world at Leiden University


Additional general writing tips (cont.)
• Use correct tense: much (not all) of introduction and methods
in past, and much (not all) of results and discussion in present.
• Avoid animosity and ad hominem arguments (personal
attacks on other researchers)
• Make your writing flow smoothly! Be:
• Clear
• Concise
• Consistent
• Be prepared to write and rewrite and rewrite and rewrite and .
. . rewrite!
• every sentence should say precisely what you want it to say
• every sentence should flow smoothly into the next
• Give your paper to others to read and review

Discover the world at Leiden University


Ethical and legal standards in publishing
• Do not fabricate or falsify data
• Data should be available for inspection
• Do not duplicate publications or do piecemeal publications of data. Make clear
to editor (and in paper):
• If study is a reanalysis of data;
• If data are from a larger data set;
• What other publications have come from that data set
• Do not plagiarize others or yourself
• Limited exceptions: descriptions or details of an instrument or analytic approach
• Do protect rights and confidentiality of research participants
• Do disclose conflict of interests
• Do protect intellectual property rights: authorship determination
• Authors: person making substantial contribution to work and who accept
responsibility for work

Discover the world at Leiden University


4 “Knows” and “No-no’s” of
Writing and Presenting

üKnow yourself w i ng
no ence,
üKnow your “stuff” o : N
t k
o udi
’s uff, a e or
- n t t
No self, s to wri
üKnow your audience r
you or how resen
t
p
üKnow how to write /
how to present
üGeneral tips

Discover the world at Leiden University


Outline
üIntroduction to Writing and Presenting
• 4 Knows – and No-no’s – of writing and presenting
ØWriting an introduction to an article

Discover the world at Leiden University


Discover the world at Leiden University
Writing an introduction
to an article

Discover the world at Leiden University


Writing the introduction
• Structure of research articles
• Content of introduction: CARS
• Writing the opening statement
• Putting it all together: Your assignments
• Example

Discover the world at Leiden University


Structure of research articles
• Title page • Results
• Title • Descriptions of results
• Authors, affiliations • Descriptive / inferential statistics
• Running head • Tables / figures
• Abstract • Discussion
• Introduction • Relation between stated purpose
• Opening statement and results
• Literature review • Theoretical/methodological
• Present study contribution
• Method • Future directions
• Participants • Ending sections
• Materials / instruments • References
• Procedures • Author notes
• Footnotes
• Tables
• Figures captions
• Figures

Discover the world at Leiden University


Writing the introduction
üStructure of research articles
ØContent of introduction: CARS
• Writing the opening statement
• Putting it all together: Your assignments
• Example

Discover the world at Leiden University


CARS: Create-a-Research Space
Move 1
General background
Specific background
Previous research

Move 2
Establish your niche

Move 3
Research purpose and
aims/questions

Swales, 1990; Swales & Feak, 2012

Discover the world at Leiden University


CARS: Create-a-Research Space
• Move 1: Establish a research territory
• Show that general research area is important,
central, interesting, problematic, or relevant
Introduce and review previous research
• Move 2: Establish a niche
• Indicate gap in previous research or need to
extend previous research in some important way
• Move 3: Occupy the niche
• Outline purpose / state nature of current research
• Present research questions / hypothesis
• Outline structure of article (only in some cases)
Discover the world at Leiden University
Across all “moves” . . .
• Demonstrate logical continuity
• One section should lead to the next, which should
lead to the next, etc.
• Connect ideas with transition statements!

Discover the world at Leiden University


Move 1 to 2: Review previous research
and establish niche
• Make opening statement
• Overview of the three moves in one paragraph
• Establish the importance of the research
• Describe what is known: Research relevant to the specific topic
• Avoid nonessential details
• Emphasize major conclusions
• Refer to reviews or meta-analyses
• Describe what is not known
• Point out gaps in research or describe need to replicate / extend research
• Make the case for your study
• Your study is the one that logically has to be done (establish niche)

Discover the world at Leiden University


Move 2 to 3: Establish and occupy niche
• Describe approach taken in current study
• How will this research address the gap / replicate and
extend the research
• State purpose of study and/or research questions
• State hypotheses

Discover the world at Leiden University


Writing the introduction
üStructure of research articles
üContent of the introduction: CARS
ØWriting the opening statement
• Putting it all together: Your assignments
• Example

Discover the world at Leiden University


Opening Statement
• A brief overview of information in 3 moves
• States the what and why of the article
• Why does the problem deserve new research?
• Layout of opening statement: hourglass
• Start with people, not researchers (broad; sparks
interest)
• Use examples to clarify statements and concepts
• Add specifics
• Tell why study is important
• State purpose of study (general terms)
• Purpose relates to “core message” of your
presentations (more later)

Discover the world at Leiden University


Start with
Opening Statement people, not
researchers

Students with reading difficulties (RD) face significant challenges in school


and in their daily lives.
Broad, sparks
interest

Discover the world at Leiden University


Opening Statement
Students with reading difficulties (RD) face significant challenges in school
and in their daily lives. They are at risk for poor academic outcomes, for
dropping out of school, and for lower-levels of employment upon leaving school
(Reschly, 2010; Smart et al., 2017; Willcutt, et al., 2007). They struggle to do
“simple” tasks such as reading menus, emails, and instructions for medicines.

Use
examples to
clarify

Discover the world at Leiden University


Opening Statement
Students with reading difficulties (RD) face significant challenges in school
and in their daily lives. They are at risk for poor academic outcomes, for
dropping out of school, and for lower-levels of employment upon leaving
school (Reschly, 2010; Smart et al., 2017; Willcutt, et al., 2007). They struggle
to do “simple” tasks such as reading menus, emails, and instructions for
medicines. Improving the reading performance of students with RD is essential
if they are to lead successful and productive lives. To improve their reading
performance, students with RD need interventions that effectively address their
individual learning needs; yet, evaluating the effectiveness of interventions for
individual students is a challenge.

More specific:
What is the
topic?

Discover the world at Leiden University


Opening Statement
Students with reading difficulties (RD) face significant challenges in school
and in their daily lives. They are at risk for poor academic outcomes, for
dropping out of school, and for lower-levels of employment upon leaving
Why is the2010;
school (Reschly, study important?
Smart et al., 2017; Willcutt, et al., 2007). They struggle
Highlights gap in research (Move 2).
to do “simple” tasks such as reading menus, emails, and instructions for
medicines. Improving the reading performance of students with RD is essential
if they are to lead successful and productive lives. To improve their reading
performance, students with RD need interventions that effectively address their
individual learning needs; yet evaluating the effectiveness of interventions for
individual students is a challenge. Curriculum-Based Measurement (CBM;
Deno, 1985) can be used to evaluate interventions for individual students.
However, recently, questions have been raised about the extent to which scores
from the CBM maze measure reflect reading comprehension.

Discover the world at Leiden University


Opening Statement
Students with reading difficulties (RD) face significant challenges in school
and in their daily lives. They are at risk for poor academic outcomes, for
dropping out of school, and for lower-levels of employment upon leaving school
(Reschly, 2010; Smart et al., 2017; Willcutt, et al., 2007). They struggle to do
“simple” tasks such as reading menus, emails, and instructions for medicines.
Improving the reading performance of students with RD is essential if they are
to lead successful and productive lives. To improve their reading performance,
students with RD need interventions that effectively address their individual
How
learning needs; yetstudy addresses
evaluating thegap (Moves
effectiveness of interventions for individual
2 and 3).
students is a challenge. Curriculum-Based Measurement (CBM; Deno, 1985)
can be used to evaluate interventions for individual students. However, recently,
questions have been raised about the extent to which scores from the CBM
maze measure reflect reading comprehension. These previous studies have used
correlational methods. In this study, we examine the extent to which CBM
reading scores reflect reading comprehension, but do so by directly
manipulating the “comprehensibility” of CBM maze texts.

Discover the world at Leiden University


Writing the introduction
üStructure of research articles
üCARS: Process for writing an introduction
üWriting the opening statement
ØPutting it all together
ØYour assignment
ØProcess for writing the introduction
ØExample

Discover the world at Leiden University


Putting it all together . . .

Discover the world at Leiden University


Your assignment
• Write a “condensed” introduction to an article
• Condensed: Shortened version of an introduction
for an article
• Length: approximately 800-1000 words; approximately
2-3 pages (A4, double spaced)
• Outlines the logic of your study
• Includes:
• Opening statement
• Three moves
• Does not include in-depth review and description of
previous research
• From your outline, create a ”lightning
presentation”

Discover the world at Leiden University


Process for writing introduction
• Brainstorm ideas / themes: Make a mind map
• Organize ideas and themes into outline using CARS
• Present your outline to group
• “Lightning presentation”
• Write your opening statement
• Write rest of introduction

Discover the world at Leiden University


Brainstorm ideas / themes: Make a mind map

Related ideas /
themes

Related ideas / Related ideas /


themes themes

Main theme /
core message

Related ideas / Related ideas /


themes themes

Related ideas /
themes

Discover the world at Leiden University


Put your mind map into outline
form using CARS
Move 1
General background
Specific background
Previous research

Move 2
Establish your niche

Move 3
Research purpose and
aims/questions

Discover the world at Leiden University


Use outline to make a “lightning
presentation” for class

Discover the world at Leiden University


Write opening statement

Discover the world at Leiden University


Example of process for writing
introduction

Discover the world at Leiden University


Brainstorm ideas / themes: Make a mind map

What is CBM?
-Purpose
What is known
What does the about scores
answer to the from CBM
question mean measures in
for practice? reading?

Do CBM maze
scores reflect
reading
comprehension?

Why is this
What is reading question
comprehension? important?

Where did this


question come
from?

Discover the world at Leiden University


Make a mind map: Group ideas / themes
What is
CBM?
-Purpose What is
What does the known about
answer to the -Why
important? scores from
question CBM
mean for measures in
practice? reading?

Do CBM maze
Why are there
How will we scores reflect questions
determine reading about whether
whether comprehension scores reflect
scores reflect (RC)? RC?
RC?

So why is it
important to What is RC
know if scores anyway?
reflect RC??

Discover the world at Leiden University


Use mind map to create outline of
introduction using CARS
Move 1
General background
Specific background
Previous research

Move 2
Establish your niche

Move 3
Research purpose and
aims/questions

Discover the world at Leiden University


Use mind map to create detailed outline
What is
CBM?
-Purpose What is
What does the known about
answer to the -Why
important? scores from
question CBM
mean for measures in
practice? reading?

Do CBM maze
Why are there
How will we scores reflect questions
determine reading about whether
whether comprehension scores reflect
scores reflect (RC)? RC?
RC?

So why is it
important to What is RC
know if scores anyway?
reflect RC??

Discover the world at Leiden University


Detailed outline: Background (Move 1)
• General background:
• Why is problem addressed in study important?
• Students who struggle to read need more intensive, individualized interventions
• It is important to determine if those intensive, individualized interventions are
effective for the student
• What is purpose of CBM – how does it address problem?
• Specific background
• What is CBM
• Define CBM generally
• Define CBM reading: Describe the CBM maze measure
• Previous research
• What is known about CBM maze: Validity of CBM maze scores in reading
• Why are there questions about validity of CBM maze scores for measuring
reading comprehension?
• Emphasize correlational research
• What is reading comprehension?
• Is this necessary? Should it go here?

Discover the world at Leiden University


Use mind map to create outline of
introduction using CARS
Move 1
General background
Specific background
Previous research

Move 2
Establish your niche

Move 3
Research purpose and
aims/questions

Discover the world at Leiden University


Use mind map to create outline
What is
CBM? What is
What does the -Purpose known about
answer to the scores from
question CBM
mean for measures in
practice? reading?

Do CBM maze
Why are there
How will we scores reflect questions
determine reading about whether
whether comprehension scores reflect
scores reflect (RC)? RC?
RC?

So why is it
important to What is RC
know if scores anyway?
reflect RC??

Discover the world at Leiden University


Detailed outline: Establish your niche –
what is not known?? (Move 2)
• Do CBM scores reflect reading comprehension?
• Is this question important for CBM implementation? (Not
really?)
• Why, then, is it important to know whether scores reflect
reading comprehension?
• Question cast doubts on validity of entire system of CBM for
measuring progress in reading
• How will we determine whether scores reflect
reading comprehension?
• Previous research: correlational research
• Our research: Direct manipulation of comprehensibility of
CBM maze texts

Discover the world at Leiden University


Use mind map to create outline of
introduction using CARS
Move 1
General background
Specific background
Previous research

Move 2
Establish your niche

Move 3
Research purpose and
aims/questions

Discover the world at Leiden University


Use mind map to create outline
What is
CBM? What is
What does the -Purpose known about
answer to the scores from
question CBM
mean for measures in
practice? reading?

Do CBM maze
Why are there
How will we scores reflect questions
determine reading about whether
whether comprehension scores reflect
scores reflect (RC)? RC?
RC?

So why is it
important to What is RC
know if scores anyway?
reflect RC??

Discover the world at Leiden University


Move 3
• Research purpose
• To determine if CBM maze scores reflect reading comprehension
• How will we determine this?
• Compare a “standard” maze to a “scrambled” maze in which sentences are put
in random order
• Scrambled maze distorts coherence of text
• If readers build a mental model of the text while they are completing the maze
(are working to comprehend the texts, thus):
• Scores on standard maze should be significantly higher than scores on scrambled
maze
• Research question:
• Are there differences in scores on a standard and scrambled CBM maze task?
• Hypothesis and rationale:
• Scores on the standard maze task will be significantly higher than scores on a
scrambled maze task
• The scrambled maze task will distort the coherence of the text
• Because maze completion requires text-level comprehension – and not just word-level or
sentence-level comprehension – when the sentences are placed in a random order in the text,
scores will decrease because the text will be more difficult to understand

Discover the world at Leiden University


Write your opening statement
• May want to try writing your opening
statement before preparing your lightning
presentation
• Then revise it after doing lightning
presentation and getting feedback

Discover the world at Leiden University


Using outline / opening statement, create
a “lightning presentation”

Discover the world at Leiden University


What is a “lightning presentation”?
• Short presentation (3-5 min) addressing a
specific topic
• Guidelines for lightning talks:
• Make your point and make it quickly (core
message)
• Don’t “sweat the small stuff”
• Details not necessary at this point
• Purpose of lightning talks for CYR:
• Present the “story” that will be conveyed in your
introduction
• Get feedback from peers / instructors:
• Is your message clear, coherent and logical?
v (More on presentations in general next week from Maria)
Discover the world at Leiden University
Example lightning presentation

Discover the world at Leiden University


Do CBM maze scores reflect reading
comprehension?

Sample Lightning Presentation: CYR 2020

Discover the world at Leiden University


Main Question: (Core Message)
Do Curriculum-based
Measurement (CBM) maze scores
reflect more than just fluency and
sentence-level comprehension?
Do they also reflect text-level
comprehension? Probably

Discover the world at Leiden University


What is Curriculum-Based Measurement
(CBM; Deno, 1985)?
• Continuous progress-monitoring system
designed to:
• Measure students’ growth over time in an
academic area
• Guide teachers’ instructional decision-
making
• Evaluate a student’s response to iterations of an
instructional program

Discover the world at Leiden University


Sam’s CBM Progress Graph in Reading

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph in Reading

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph

(Deno,
Discover the world at 1985;
Leiden2003)
University
Sam’s CBM Progress Graph
Instruction phase 4

(Deno,
Discover the world at 1985;
Leiden2003)
University
Intervention team celebrates!!!

101

Discover the world at Leiden University


Characteristics of CBM measures / scores

l Practical
l Inexpensive
l Easy to use
l Time efficient
l Repeatable
l Easy to understand
l Technical (scores)
l Reliable
l Valid
What measures / scores? l Sensitive to change
in performance

(Deno, 1985; 2003)


Discover the world at Leiden University
Characteristics of CBM measures / scores
Global indicators of performance in an academic area

l Practical
l Inexpensive
l Easy to use
l Time efficient
l Repeatable
l Easy to understand
l Technical (scores)
l Reliable
l Valid
What measures / scores? l Sensitive to change
in performance

(Deno, 1985; 2003)


Discover the world at Leiden University
CBM Reading Measures
Reading aloud
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another one: he graduated from high school. Dressed in his black cap and gown, he
shook hands with commencement speaker, ex-Viking, and state Supreme Court Justice Alan
Page. Then he took his diploma from Principal Linda Nelson.

Score: words read correctly Time: 1 minute

Maze selection
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another was / eat / one: he graduated from high school. Dressed be / in / or his black
cap and gown, he shook / carts / south hands with commencement speaker, ex-Viking, will /
and / buy state Supreme Court Justice Alan Page. Hole / Said / Then he took his diploma
from Principal Linda Nelson.

Score: correct choices Time: 2 – 3 minutes

Discover the world at Leiden University


CBM Reading Measures

Maze selection
Larry Fryklund has marked many milestones in his 86 years of life. On Wednesday night, he
marked another was / eat / one: he graduated from high school. Dressed be / in / or his black
cap and gown, he shook / carts / south hands with commencement speaker, ex-Viking, will /
and / buy state Supreme Court Justice Alan Page. Hole / Said / Then he took his diploma
from Principal Linda Nelson.

Score: correct choices Time: 2 – 3 minutes

Discover the world at Leiden University


Results of validity studies on CBM
CBM maze scores demonstrate that . .

CBM maze scores are


valid as global
indicators of reading
performance.

Discover the world at Leiden University


But wait . . .

Isn’t maze a
measure of reading
comprehension?

Discover the world at Leiden University


Guthrie, J.T., Seifert, M., Burnham, N.A., & Caplan,
R. (1974). The maze technique to assess, monitor
reading comprehension. Reading Teacher (28), 161-
168.

Discover the world at Leiden University


. . . and moreover . . . isn’t it true
that . . .

Researchers
criticize the maze
as a measure of
reading
comprehension?

Discover the world at Leiden University


More specifically, isn’t it true
that . . .
CBM maze scores reflect just
fluency and sentence-level
comprehension. . .
. . . and not text-level comprehension . . .
Probably

Discover the world at Leiden University


For example: Muijselaar, Kendeou, de
Jong, & van den Broek (2017)

What does the CBM-maze test measure?

• CBM maze scores reflect technical reading skills


(word reading, reading fluency), but not text
comprehension

Discover the world at Leiden University


OK, but . . . does it really matter?

You told us
that . . .

Discover the world at Leiden University


Within CBM . . .
• Maze is not a measure of reading (comprehension)
• Maze scores: global indicators of reading performance
(fluency, decoding, and comprehension)

• If . . .
• Maze scores serve as valid indicators of performance and
progress in reading, then. . .
• Does it really matter if there are criticisms of the maze as a
“measure of” reading comprehension?

Discover the world at Leiden University


Maybe it does matter . . .
• Researchers and practitioners alike:
• often do refer to CBM maze as a measure of reading
comprehension
• Research criticizing CBM maze:
• casts doubts on use of CBM maze for progress
monitoring and instructional decision-making in
reading

Discover the world at Leiden University


Conclusion of Muijselaar et al.
(2017)
• “… the findings raise an important question, namely,
whether the wide use of the CBM-Maze test, when used as a
formative assessment to influence instructional focus (vs.
more simply using as an efficient predictor), can adequately
evaluate the strengths and weaknesses of students in reading
comprehension. Previous studies also suggested that such
cloze tests measure primarily lower level comprehension
processes (e.g., sentence comprehension; e.g., Gellert &
Elbro, 2013).” (pp. 128)

Discover the world at Leiden University


Reason enough to examine issue
more closely . . .

Discover the world at Leiden University


Question Addressed in Research
Do CBM maze scores reflect more
than just fluency and sentence-
level comprehension. . .

. . . do they also reflect text-level


comprehension?

Discover the world at Leiden University


And how do you propose to do address this
question?

Previous
research:
correlational

Discover the world at Leiden University


Our research . . .

Directly manipulate
“coherence” of maze
text by scrambling
sentences, and
examining effects on
CBM maze scores

Discover the world at Leiden University


Research question
• Are scores on a scrambled (not coherent) CBM maze
lower than scores on a standard (coherent) CBM maze
passage?
• If yes, implies that:
• text-level comprehension is an important aspect of CBM
maze completion, and
• that CBM maze scores reflect reading comprehension

Discover the world at Leiden University


Results of the study will answer. . .
Do CBM maze scores reflect more
than just fluency and sentence-
level comprehension. . .

. . . do they also reflect text-level


comprehension?

Discover the world at Leiden University


Discover the world at Leiden University
Outline
üIntroduction to Writing and Presenting
ü4 Knows – and No-no’s – of writing and presenting
üWriting an introduction to an article

Discover the world at Leiden University


Conclusion
• Remember that
writing is also
thinking . . .
• And it’s hard to do,
even when you are a
native speaker . . .
• But it’s also
rewarding

Discover the world at Leiden University

You might also like