Assignment 1B: Constructs, Variables, and Hypotheses

Chapters 4 and 5 in the course textbook describe the most important elements of a research
study and provide practice with reading (“dissecting”) research reports. (Chapter 5 is especially
important—you may need to reread it.) Now you can apply your new learning to the Ardern &
Henry (2019) research report by answering the following questions.

These researchers sidestep the complexities of measuring a construct by repeatedly referring to

“performance.” Yet student performance reflects underlying traits or abilities (that is,
constructs). One could argue that the construct investigated by these researchers is “learning,”
“achievement,” “writing skill,” or some such label.

1. Remember that abstract traits like intelligence, motivation, anxiety, and sociability are
constructs. They are labels “constructed” to refer to some “in the head” dimension or
characteristic, whereas operational definitions are the means for measuring or
observing the abstraction. (The Stanford-Binet IQ score, GPA, heart rate, and number of
reported friends may be operational definitions for intelligence, motivation, anxiety, and
sociability, respectively.) Keep in mind that some constructs (like intelligence) are
incredibly complex, and some of their underlying operational definitions are incredibly
simple.

Describe how the researchers translated “performance” into something measurable,

namely, scores. Remember, an operational definition describes the rule for “putting
numbers next to names.” [3 points]

Performance as an abstract ability cannot be measured directly. This construct has to be

defined in a way that makes it observable and measurable as concrete operational
definitions. In this study, the researchers measured performance in the form of scores
of paper-and-pencil or handwritten tests such as: multiple choice tests, short and long
answer open-ended assessment, and hands-on performance assessments covering
reading, writing, science, and math skills.

2. In a published report, researchers often provide a clear statement of their research

hypothesis (or research question) in the last sentence, just above the “Design” or
“Method” section. How would you translate the researchers’ statement of purpose into
a clear If A, then B format for the research hypothesis? [2 points]

If the students use computer as the medium of test administration, they will score
higher on multiple-choice, short-answer and extended writing test items than others
who use paper and pencil.

3. These researchers also had reason to believe that students who are accustomed to
using a computer for writing will probably underperform if they are tested with paper
and pencil as opposed to a computer (but their assessment using a multiple-choice
 format will not be affected). It’s a complex hypothesis. Researchers are often not
committed to the direction of a difference (or relationship), leaving open the possibility
that students may perform better or worse in a particular condition. These researchers
may have had reason to question the validity of paper-and-pencil tests in the age of
computers, hence their hypothesis might be stated in this “underperformance” manner.
They are, though, a bit fuzzy in this regard. Sometimes, it is easier to simply ask a
research question opposed to using awkward language in a hypothesis. How would you
recast the study’s purpose as a research question? [1 point]

How does the medium of administration affect student performance on short and long-answer
questions, as well as performance assessments?

4. Scientific research never exists within a vacuum. Describe how this study “fits” into
existing knowledge. [3 points]

From their literature review, the researchers found out that the most commonly used open-
ended test has been hand-and-pencil writing test. Also, as more and more students start using
computers to write digitally, there’s a lack of explanation on how the medium of administration
may not necessarily reflect the authenticity of those tests. The knowledge gap here is that it is
not clear how handwriting test will affect performance for students who are accustomed to
writing via computer.

5. After carefully reviewing the four major types of variables that researchers work with,
how would you identify the following? [4 points]
1. The independent variable or variables and levels or categories of variation.

Independent variable: medium of test administration

Categories of variation: writing the test by hand-and-pencil OR digitally using computers

2. The dependent variable or variables (Hint: Focus on the major dependent

variables, that is, all three of them.)

Dependent variables:

- Student performance/scores for the National Assessment of Educational Progress

- Student performance/scores for the open-ended questions
- Student performance/scores for the long-answer writing test

3. Gender as an attribute variable was only included as an afterthought in the

study,
and not in the original design. Did they indicate a focus on any other attribute
variable or variables?
Attribute variables:
- Length of writing
- Student motivation
- Different ability levels of students

4. One extraneous variable that was controlled by these researchers.

- Student

6. Researchers go to great effort to avoid confounding (study this difficult idea carefully in
Chapter 5 of the text). Now describe how these researchers, if they were very careless,
might have introduced a confounding. State what is confounded with what. [2 points]

- Rt
-

7. Good researchers attend to the concept of alternative hypotheses, those influences that
might explain the findings in contrast to the hypothesized cause (the independent
variable). Describe one alternative hypothesis that concerned these researchers. (Hint:
They discuss it closer to the end of the report.) [1 point]

Alternative hypothesis:
- Students score higher on the performance tests because they have a greater extent of
experience working on computers.

8. Finally, the null hypothesis plays a pivotal role during the data analysis phase of
research. Provide one example of a null hypothesis that was tested by these
researchers. (Hint: You’ll want to focus on the Overall Results section for this.) [1 point]

Null hypothesis:
- In a population of students, there is no difference in student performance of those who
use computers and those who use pencil and paper to answer the tests.