This article was downloaded by: [Colorado College]

On: 09 December 2014, At: 12:14

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Science

Education
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/tsed20

Construct Validation of the Physics

Metacognition Inventory
a b
Gita Taasoobshirazi & John Farley
a
Department of Secondary & Middle Grades Education , Kennesaw
State University , Kennesaw , USA
b
Department of Physics , University of Nevada , Las Vegas , USA
Published online: 05 Dec 2012.

To cite this article: Gita Taasoobshirazi & John Farley (2013) Construct Validation of the Physics
Metacognition Inventory, International Journal of Science Education, 35:3, 447-459, DOI:
10.1080/09500693.2012.750433

To link to this article: http://dx.doi.org/10.1080/09500693.2012.750433

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 International Journal of Science Education, 2013
Vol. 35, No. 3, 447–459, http://dx.doi.org/10.1080/09500693.2012.750433

Construct Validation of the Physics

Metacognition Inventory
Gita Taasoobshirazia∗ and John Farleyb
a
Department of Secondary & Middle Grades Education, Kennesaw State University,
Kennesaw, USA; bDepartment of Physics, University of Nevada, Las Vegas, USA
Downloaded by [Colorado College] at 12:14 09 December 2014

The 24-item Physics Metacognition Inventory was developed to measure physics students’
metacognition for problem solving. Items were classified into eight subcomponents subsumed
under two broader components: knowledge of cognition and regulation of cognition. The
students’ scores on the inventory were found to be reliable and related to students’ physics
motivation and physics grade. An exploratory factor analysis provided evidence of construct
validity, revealing six components of students’ metacognition when solving physics problems
including: knowledge of cognition, planning, monitoring, evaluation, debugging, and information
management. Although women and men differed on the components, they had equivalent overall
metacognition for problem solving. The implications of these findings for future research are
discussed.

Keywords: Metacognition; Physics education; Construct validity; Factor analysis

Physics Problem Solving

The ability to correctly set up and solve physics problems is critical for success in
introductory, intermediate, and advanced level physics courses. Most of the work stu-
dents complete in high school and college level physics courses in class, for homework,
and on tests involves setting up and solving problems (Chi, 2006; Davidson, 2003;
Williams & Noyes, 2007). Physics problem solving is important because it facilitates
students’ learning of the concepts and how they relate to one another. Problem solving
also plays a vital role in discovery and invention in physics. These discoveries and
inventions have been of great benefit to humanity and, consequently, a major

∗
Corresponding author. Department of Secondary & Middle Grades Education, Kennesaw State
University, 1000 Chastain Road, Kennesaw, GA, USA. Email: gita.tshirazi@gmail.com

# 2013 Taylor & Francis

 448 G. Taasoobshirazi and J. Farley

educational goal in all countries is to develop the problem-solving skills of students

learning physics (Chi, 2006).
Metacognition has been found to be a critical factor in contributing to students’
learning of physics (Schunk, 2008; Zimmerman, 2006; Zimmerman & Campillo,
2003). For example, Koch (2001) taught college-level physics students how to
think metacognitively about problems in their physics textbooks. She found that
those students, compared to students who did not receive the instruction, had
greater understanding of the physics concepts. Less research, however, has examined
the impact of metacognition on physics problem solving. The research that has been
conducted indicates that students who are more metacognitive during physics
problem solving are more likely to correctly solve the problems (e.g. Neto &
Valente, 1997; Rozencwajg, 2003).
One potential reason for the lack of research examining the role of metacognition on
Downloaded by [Colorado College] at 12:14 09 December 2014

physics problem solving is the absence of an inventory that measures metacognition

for science problem solving. The research that has examined metacognition for
problem solving in science has done so using primarily verbal interviews or a small
set of researcher-developed items (e.g. Rozencwajg, 2003). The purpose of the
present research was to develop a valid, reliable, objective, and convenient tool that
researchers and instructors can use to assess students’ metacognition for solving
physics problems.

Theoretical Framework
Metacognition is a ‘cognitive activity that takes as its object, or regulates, any aspect of
any cognitive enterprise’ (Flavell, 1985, p. 104). The research on metacognition tends
to distinguish between two components of metacognition: knowledge of cognition and
regulation of cognition (e.g. Alexander, 2008; Dinsmore, Alexander, & Loughlin,
2008; Veenman, 2005; Veenman, Van Hout-Wolters, & Afflerbach, 2006). Knowledge
of cognition refers to what individuals know about their cognition and has been con-
sidered to include at least three different kinds of metacognitive knowledge including
declarative, procedural, and conditional knowledge (Schraw, 2001; Veenman, 2005;
Veenman et al., 2006). Declarative knowledge refers to knowledge about oneself as a
learner or problem solver. For example, a physics student may know his or her
strengths and weaknesses when it comes to solving physics problems. Procedural
knowledge refers to knowledge about how to perform a task or activity. This includes
knowledge about how to apply strategies and procedures. For example, a physics
student may know how to apply an effective strategy to successfully solve a
problem. Conditional knowledge refers to knowledge about when and why to use
declarative and procedural knowledge. This includes knowledge about when and
why to select and apply strategies that are most appropriate for a problem. For
example, a physics student may know when and why to apply a particular strategy
to successfully solve a problem.
The second component of metacognition, regulation of cognition, refers to actions
that help students regulate or control their learning and problem solving (Dinsmore
 Construct Validation 449

et al., 2008; Veenman et al., 2006). Regulation of cognition includes at least three
different kinds of metacognitive regulation including planning, monitoring, and evalu-
ation (Schraw, 2001; Schraw, Crippen, & Hartley, 2006). Planning refers to planning
and goal setting prior to performing a task. For example, a physics student may plan
how he or she will solve a problem before beginning to work on the problem. Moni-
toring refers to the periodic assessment of one’s goals, work, and performance. For
example, a physics student may periodically evaluate his or her work while solving a
problem. Evaluation refers to the appraising of one’s work after it is completed. For
example, a physics student may look back at a solved physics problem to evaluate
his or her procedures and final answer.
In addition to the three commonly discussed subcomponents of regulation of cog-
nition, two additional subcomponents have been reported in the research on metacog-
nitive regulation in science (Schraw & Dennison, 1994). This includes debugging and
Downloaded by [Colorado College] at 12:14 09 December 2014

information management. Debugging refers to the strategies used to correct learning and
problem-solving errors. For example, a student who is having difficulty solving a
physics problem may seek help from peers or the instructor. Information management
refers to specific strategies used by a student to help him or her solve problems more
effectively. One important example in physics would be creating and using free-body
diagrams when solving physics problems. There is a large body of research emphasiz-
ing the importance of the use of free-body diagrams in physics to help students cor-
rectly set up and solve problems (Heckler, 2010; Kohl & Finkelstein, 2006).
Physics textbooks, high school and college physics instructors, and researchers
emphasize the importance of these diagrams because they allow the problem solver
to determine which approach to the problem is appropriate, to identify the forces
and energies at work, and to reduce the amount of information that must be attended
to at one time (e.g. diSessa, 2004; Hewitt, 2006). Such diagrams are also important
for problem solving in other sciences such as in chemistry problem solving (often
called chemical structures or molecular diagrams rather than free-body diagrams).
The literature on the metacognitive components provided a foundation for opera-
tionalizing the construct of metacognition for physics problem solving and developing
the Physics Metacognition Inventory. A construct such as metacognition could be
conceptualized by students either as a single component or one with several subcom-
ponents (Glynn, Taasoobshirazi, & Brickman, 2009). In addition, students’ concep-
tualizations of a construct may differ somewhat from how experts conceptualize it and
describe it in the literature (Donald, 1993). Our goal was to determine how students
conceptualized their metacognition for physics problem solving.

Present Study
The goal of the present study was to establish the construct validity of the Physics
Metacognition Inventory through an exploratory factor analysis. The Physics Meta-
cognition Inventory can provide objective information about a student’s metacogni-
tive activity during physics problem solving and identify where students may be
failing to understand and regulate their problem solving. This information can help
 450 G. Taasoobshirazi and J. Farley

instructors and researchers understand and provide support to enhance metacogni-

tion and, as a result, problem-solving success.

Method
Participants
Subjects included a total of 505 introductory-level college physics students (261
women and 244 men) from six classes at four different universities in Nevada and
Georgia (one of the universities was located in Nevada and three were located in
Georgia). In order to recruit participants, physics instructors at universities who
taught introductory-level physics (trigonometry or calculus-based introductory
physics) were contacted by email and invited to participate.
Downloaded by [Colorado College] at 12:14 09 December 2014

Overall, a majority of the students participated in the study (87%). Students’ par-
ticipation was voluntary rather than compulsory, as specified by the guidelines for
research with human participants by the institutional review board. The students par-
ticipated towards the latter part of the 15-week semester. Regarding ethnicity, 345
students were Caucasian (68%), 76 were Asian (15%), 49 were African American
(10%), and 35 were Hispanic (7%).

Physics Metacognition Inventory

Following guidelines by Pett, Lackey, and Sullivan (2003), the Physics Metacognition
Inventory (see Appendix for the 24 items) was developed. These guidelines, described
in detail in the texts, include reviewing the relevant research, identifying latent vari-
ables, and developing empirical indicators of the latent variables.
The components and items included knowledge of cognition: declarative knowledge
(items 5 and 6), knowledge of cognition: procedural knowledge (items 7 and 11), knowledge
of cognition: conditional knowledge (items 12 and 13), regulation of cognition: planning
(items 1, 14, 19, 20, and 24), regulation of cognition: monitoring (items 2, 15, 16,
and 21), regulation of cognition: evaluation (items 8, 9, and 17), regulation of cognition:
debugging (items 3 and 22), and regulation of cognition: information management (items
4, 10, 18, and 23). Students responded to each of the randomly ordered items on a
five-point Likert-type scale ranging from 1 (never true of myself) to 5 (always true of
myself).

Materials and Procedures

Students were administered a packet that included a demographics inventory and two
questionnaires. First, demographic information, including information about course
grade, gender, and ethnicity, was collected. This reported course grade included stu-
dents’ grade in the course with the exclusion of the final exam (scored as 5 ¼ A,
4 ¼ B, 3 ¼ C, 2 ¼ D, 1 ¼ F). Second, students were administered the Physics Meta-
cognition Inventory. Finally, students were administered the Physics Motivation
 Construct Validation 451

Questionnaire (PMQ) (Glynn & Koballa, 2006; Glynn et al., 2009). The PMQ
includes 30 items that assess six important components of student motivation in
physics including intrinsically motivated physics learning, extrinsically motivated
physics learning, relevance of learning physics to personal goals, self-determination
for learning physics, self-efficacy in learning physics, and anxiety about physics assess-
ment. Students responded to each of the 30 randomly ordered items on a five-point
Likert scale ranging from 1 (never) to 5 (always) from the perspective of ‘when learn-
ing physics.’ The anxiety about physics assessment items were reverse scored when
added to the total, so that a higher score on this component meant less anxiety. Pre-
vious findings (Glynn & Koballa, 2006) indicate that the PMQ is reliable as measured
by coefficient alpha (a ¼ 0.93). Convergent and discriminant validity for the PMQ is
established by Glynn and Koballa (2006). For this study, internal consistency was
found to be (a ¼ 0.90). For the present study, self-reported course grade and
Downloaded by [Colorado College] at 12:14 09 December 2014

physics motivation were used as criterion variables.

Results
PASW (Predictive Analytics SoftWare), version 18.0 was used to analyze the data. For
the Physics Metacognition Inventory, steps used by Glynn et al. (2009) and Pett et al.
(2003) were followed closely to determine the reliability of students’ scores, assess the
adequacy of the matrix of correlations among the items, extract the factors, rotate
them, examine the factor loadings, and interpret the factors.

Gender and Ethnic Group Differences, Reliability, and Criterion Validity of Total Scores
The reliability (internal consistency) of the 24 items was 0.90, indicating that 90% of
the variance of the total scores on the Physics Metacognition Inventory can be attrib-
uted to systematic variance. The students’ total scores on the 24 items correlated sig-
nificantly with their reported physics course grade (r ¼ 0.27, p , 0.001) and physics
motivation (r ¼ 0.56, p , 0.001), providing evidence of criterion-related validity.
No significant differences were found between women and men or the different
ethnic groups in total scores on the Physics Metacognition Inventory. The men
(M ¼ 102.15, SD ¼ 16.79), however, had higher motivation than the women
(M ¼ 95.33, SD ¼ 15.42), t(503) ¼ 4.76, p , 0.001, Cohen’s d ¼ 0.42. Across
the different ethnic groups, Caucasians (M ¼ 4.00, SD ¼ 0.82) had higher course
grades than African Americans (M ¼ 3.63, SD ¼ 1.01), p ¼ 0.03. However, this
last difference should be interpreted with caution given the discrepancy in the
sample size of the two groups.

Exploratory Factor Analysis

Inter-item correlations for the 24 items were obtained and it was determined that the
resulting matrix of correlations was appropriate for factor analysis as determined
 452 G. Taasoobshirazi and J. Farley

by Bartlett’s test of sphericity, x2 ¼ 7,077.36, df ¼ 276, p , 0.001, and the

Kaiser-Meyer-Olkin measure of sampling adequacy, KMO ¼ 0.88.
To extract factors, a principal component analysis was performed on the 24 items.
Using the Kaiser2Guttman rule, six factors that had eigenvalues greater than 1 were
identified. Together, these six factors accounted for 69.46% of the total variance. A
scree plot was also used and supported the six-factor solution obtained using the
Kaiser2Guttman rule. To further check the stability of the six-factor solution, the
data were subjected to principal axis factoring and maximum likelihood extraction.
The same number of factors was extracted and the items loaded similarly across all
the extraction techniques, attesting to the stability of the six-factor solution.
The six factors were then rotated, turning their reference axes about their origin. A
promax rotation was used; a varimax rotation was also used and produced similar
results.
Downloaded by [Colorado College] at 12:14 09 December 2014

The factor loadings from the principal components analysis with the promax sol-
ution are in Table 1. All but one of the items met the criterion of loading at least
0.35 on their respective factor (Tabachnick & Fidell, 2000). The eigenvalue associ-
ated with each factor, the percent of variance explained by each factor, the cumulative
percent of variance explained, and the Cronbach coefficient alpha for each factor are
in Table 2. The Cronbach coefficient alphas for Factors 1 –5 range from acceptable to
excellent, but the one associated with Factor 6 is less than acceptable, so at least some
items in Factor 6 need revision.
An examination of the six factors indicated that they were related to the eight meta-
cognitive components reported in the research. Factor 1 contained 6 items: each of
the two items from the knowledge of cognition components (declarative, procedural,
and conditional knowledge). For simplicity, therefore, we labeled this factor knowledge
of cognition. This six-item factor was the most important of the six factors because it
explained 30.09% of the total amount of variation in students’ responses to the ques-
tionnaire items. We interpret this finding to mean that the students perceived the
different knowledge of cognition components to be so closely related that they con-
ceptualized them as a set.
Factor 2 contained the four Regulation of cognition: Information management items.
These items included those that refer to regulating one’s cognition through the use
of free-body diagrams to assist in physics problem solving. This factor was the
second most important of the six, explaining 11.42% of the total variation in the stu-
dents’ responses to the questionnaire items.
Factor 3 contained the four Regulation of cognition: Monitoring items. The items
accounted for 9.64% of the total variation in students’ responses to the questionnaire
items.
Factor 4 contained only three items, but the loadings were strong, accounting for
6.70% of the total variation in students’ responses to the questionnaire items. The
items included the Regulation of Cognition: Evaluation items.
Factor 5 included the two Regulation of Cognition: Debugging items. The items
accounted for 5.94% of the total variation in students’ responses to the questionnaire
items. Although the two items have strong loadings, psychometrically, it is typically
 Construct Validation 453

Table 1. Factor loadings on the Physics Metacognition Inventory

Item Factor
no. loading Item

Factor 1. Knowledge of cognition: declarative, procedural, and conditional

11 0.87 When solving a physics problem, I know how to apply a strategy to
successfully solve the problem (procedural)
13 0.87 When solving a physics problem, I know when to use a particular strategy
(conditional)
12 0.87 When solving a physics problem, I know why I am using a particular
strategy (conditional)
6 0.81 When solving physics problems, I know how I work best (declarative)
7 0.79 When solving physics problems, I have a specific purpose for each strategy
I use (procedural)
Downloaded by [Colorado College] at 12:14 09 December 2014

5 0.76 I am a good judge of how well I solve physics problems (declarative)

Factor 2. Regulation of cognition: information management (free-body diagrams)
23 0.96 I draw free-body diagrams for the physics problems I am solving
10 0.95 I use free-body diagrams to help me solve physics problems
4 0.92 I draw free-body diagrams to help me solve physics problems
18 0.71 I know why free-body diagrams are important for physics problem solving
Factor 3. Regulation of cognition: monitoring
15 0.96 While solving a physics problem, I ask myself questions about how well I
am doing
16 0.91 While solving a physics problem, I periodically evaluate how well I am
doing
21 0.87 While solving a physics problem, I ask myself if I am meeting my goals
2 0.71 While solving a physics problem, I ask myself periodically if I am meeting
my goals
Factor 4. Regulation of cognition: evaluation
9 0.93 After solving a physics problem, I double check my answer
8 0.89 I go back and check my work after solving a physics problem
17 0.62 After solving a physics problem, I look back to see if I did the correct
procedures
Factor 5. Regulation of cognition: debugging
3 0.96 I ask for help when I do not understand a physics problem
22 0.93 I seek help when I do not understand the physics problems that I am
solving
Factor 6. Regulation of cognition: planning
1 0.93 I think about what a physics problem is asking before I begin to solve it
20 0.52 Before solving a physics problem, I identify all the important parts of the
problem
24 0.49 Before solving a physics problem, I eliminate information in the problem
that I do not need
14 0.39 Before solving a physics problem, I think about what a reasonable value for
the answer would be
19 0.30 Before I start solving a physics problem, I plan out how I am going to solve
it
 454 G. Taasoobshirazi and J. Farley

Table 2. Eigenvalue, percent of variance explained, and Cronbach’s coefficient alpha for each
factor

Factor Eigenvalue Percentage of variance Cumulative percent Cronbach’s alpha

Factor 1 7.58 31.59 31.59 0.90

Factor 2 2.74 11.42 43.01 0.91
Factor 3 2.32 9.64 52.65 0.87
Factor 4 1.61 6.70 59.35 0.78
Factor 5 1.43 5.94 65.29 0.92
Factor 6 1.00 4.17 69.46 0.68

Notes: Factor 1 is knowledge of cognition, Factor 2 is information management, Factor 3 is monitoring,

Factor 4 is evaluation, Factor 5 is debugging, and Factor 6 is planning.
Downloaded by [Colorado College] at 12:14 09 December 2014

desirable to have more than two items in a factor. It would be helpful to develop more
items to assess this component. Additional items, for example, could include: ‘I ask
my instructor for help when I don’t understand a physics problem,’ and ‘I seek tutor-
ing when I don’t understand the physics problems that I am solving.’
Factor 6 included the five Regulation of Cognition: Planning items. The items
accounted for 4.17% of the total variation in students’ responses to the questionnaire
items. Two of the items, however, were problematic. Item 19 had a loading of less than
0.35 (0.30) on the factor; item 14 had a loading of 0.39 on the planning factor and a
loading of 0.40 on the monitoring factor. Item 14 was placed with the planning factor
because, conceptually, it fit better with this factor. However, both items 19 and 14
should be revised so that the loadings of the items to their respective factor are
clear and follow a simple structure. For example, item 14 could be revised to read
‘I think about what a reasonable answer for the problem might be before I even
begin solving the physics problem’ or ‘I think about what a reasonable value for the
answer might be before I even begin solving the physics problem.’ It is not completely
clear why item 14 loaded on both the planning and monitoring factors. Students
might consider a potential reasonable answer to the problem while they are solving
the problem. However, this is particularly important during the planning phase
before even beginning to solve the problem. The item can be revised to emphasize
that a potential answer is being taken into consideration ‘before I even begin
solving the problem.’ Item 19 could be revised to read ‘I plan out how I’m going to
solve a physics problem before I even begin to solve it’ to emphasize the ‘before I
even begin to solve it’ planning aspect of problem solving.
Having identified six factors, we examined the correlations among the factors.
These correlations are reported in Table 3. The factors correlated significantly with
each other.

Ethnic Group and Gender Differences on the Six Factors

There were no significant differences in total scores on the Physics Metacognition
Inventory due to ethnicity or gender; however, gender differences were found
 Construct Validation 455

Table 3. Correlations among the factor-based scales and the criterion variables

F1 F2 F3 F4 F5

F1
F2 0.33
F3 0.34 0.21
F4 0.34 0.24 0.41
F5 0.13 0.29 0.13 0.18
F6 0.57 0.35 0.53 0.42 0.16

Note: All correlations are significant, p , 0.01.

across three of the six factors. The scores on the knowledge of cognition items were
Downloaded by [Colorado College] at 12:14 09 December 2014

higher among the men (M ¼ 21.20, SD ¼ 4.58) than the women (M ¼ 20.05, SD
¼ 4.44), t(503) ¼ 2.89, p ¼ 0.004, Cohen’s d ¼ 0.25, suggesting that the men are
more likely to understand their problem solving strengths and weaknesses, how to
apply strategies, and when and why to apply those strategies. The scores on the infor-
mation management (free-body diagrams) scale were higher among the women (M ¼
16.91, SD ¼ 2.85) than the men (M ¼ 15.98, SD ¼ 3.38), t(503) ¼ 3.36, p ,
0.001, Cohen’s d ¼ 0.30, suggesting that the women are more likely to integrate
free-body diagrams in their physics problem solving. The scores on the debugging
scale were higher among the women (M ¼ 7.58, SD ¼ 1.94) than the men (M ¼
6.81, SD ¼ 2.30), t(503) ¼ 4.06, p , 0.001, Cohen’s d ¼ 0.36, suggesting that
women are more likely to seek help when having difficulty solving physics problems.

Discussion
The goal of the present study was to establish the construct validity of the Physics
Metacognition Inventory with physics students in an introductory-level physics
course. The students’ total scores on the Physics Metacognition Inventory were
found to be reliable and related to students’ course grade and physics motivation.
This questionnaire, designed to provide knowledge about students’ metacognition
when solving physics problems, can be useful to physics instructors and physics edu-
cation researchers in understanding and supporting students’ problem-solving
success, and in turn, achievement.

Dimensions of Metacognition in Physics Problem Solving

There were no significant differences in total scores on the Physics Metacognition
Inventory due to gender, indicating that the women and men were equivalent in
their overall metacognition for physics problem solving. There were gender differ-
ences, however, on the dimensions of knowledge of cognition, information management,
and debugging. The men reported more knowledge of their cognition whereas women
were more likely to draw free-body diagrams and seek help when solving physics
 456 G. Taasoobshirazi and J. Farley

problems. These findings provide new information about the role that gender plays in
metacognition for physics problem solving. Future research should explore these
gender differences and their impact on quality of free-body diagrams, problem-
solving strategy use, and problem-solving achievement.

Directions for Future Research

The six factors identified by the exploratory factor analysis were interpreted as dimen-
sions by which students perceive their metacognition for physics problem solving.
Some of the items need to be revised to better fit the dimensions, and some dimen-
sions need additional items. Revising items and developing new ones are common
procedure in the process of construct validation, which occurs over a series of
studies (Glynn et al., 2009; Pett et al., 2003). When items have been revised and
Downloaded by [Colorado College] at 12:14 09 December 2014

new ones developed, the next step is to cross-validate the instrument on new
samples of physics students using the method of confirmatory factor analysis
(Glynn et al., 2009; Kline, 2005; Schumacker & Lomax, 2004).
The instrument can be used to assess students’ metacognition for problem solving
in other sciences, such as chemistry, where problem solving also plays a significant role
in understanding and achievement. In such a case, the word chemistry is substituted
for the word physics. The information management (free-body diagrams) component
items were designed to focus on the domain of physics. Although diagrams are impor-
tant for problem solving in other sciences as well (e.g. molecular diagrams for chem-
istry), they are not referred to as free-body diagrams. When administering the
instrument to chemistry students, for example, the information management items
can be excluded or revised to state ‘molecular diagrams’ rather than ‘free-body
diagrams.’
Another direction for future research includes developing and testing models—
using structural equation modeling—that examine how metacognition interacts
with other variables critical for problem-solving success such as motivation, strategy
use, and conceptual knowledge (Williams & Noyes, 2007). Such models will help
describe how these variables influence each other and quantify the relative contri-
butions of each variable. The knowledge that results from studying these variables,
and how they interact, can be used to improve the teaching and learning of physics
problem solving by informing instructors and researchers about how these variables
impact one another and which variables have the largest impact on problem-solving
success. In addition, research can examine how the various components of metacog-
nition (e.g. monitoring and evaluation) interact with and impact problem solving,
which can help provide specific information about the relative contributions of the
various metacognitive components on problem-solving success.

Conclusion
The ability to correctly set up and solve physics problems is critical for success in
introductory, intermediate, and advanced level physics courses. Physics education
 Construct Validation 457

researchers and instructors can contribute to students’ problem solving by assessing,

understanding, and fostering students’ metacognition for physics problem solving. In
order to do this they need good assessment tools. Besides for the present study, to
date, there is no instrument specifically designed to assess metacognition for
science problem solving. This significantly limits research and instruction on meta-
cogntion. For example, one potential reason for the lack of research examining the
role of metacognition on physics problem solving is the absence of an inventory
that measures metacognition for science problem solving. The goal of the present
study was to develop an objective, valid, and reliable instrument that could be used
to assess metacognition for physics problem solving. The instrument can also be
used to assess metacognition for problem solving in other sciences as well, such as
chemistry, where problem solving also plays a critical role in understanding and
achievement.
Downloaded by [Colorado College] at 12:14 09 December 2014

The present study is an important one because it confirmed the overall reliability
and construct validity of the Physics Metacognition Inventory and examined how stu-
dents perceived dimensions of their metacognition. The instrument can be used by
researchers to study student metacognition for physics problem solving. For
example, the instrument can be used as a pre-post measure to assess the effectiveness
of instructional interventions designed to improve metacognition among physics
students.
The instrument can also be used by researchers and instructors to understand and
support students’ metacognition for physics problem solving. For example, physics
instructors can administer the instrument to their physics students a few weeks into
a physics course. After administrating the inventory, the metacognitive components
can be described to the students, and students’ individual results can be provided
to them so that they can understand their metacognitive strengths and weaknesses.
Instructors can then use group and individual results to provide advisement sessions
and/or instructional strategies such as modeling to improve students’ metacognition.

References
Alexander, P.A. (2008). Why this and why now? Introduction to the special issue on metacognition,
self-regulation, and self-regulated learning. Educational Psychology Review, 20, 369 –372.
Chi, M.T.H. (2006). Two approaches to the study of experts’ characteristics. In N. Charness, P.J.
Feltovich, R.R. Hoffman, & K.A. Ericsson (Eds.), The Cambridge handbook of expertise and
expert performance (pp. 21– 30). New York, NY: Cambridge University Press.
Davidson, J.E. (2003). Insights about insightful problem solving. In J.E. Davidson & R.J. Sternberg
(Eds.), The psychology of problem solving (pp. 149– 175). New York, NY: Cambridge University
Press.
Dinsmore, D.L., Alexander, P.A., & Loughlin, S.M. (2008). Focusing the conceptual lens on meta-
cognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20,
391– 409.
Donald, J.G. (1993). Professors’ and students’ conceptualizations of the learning task in physics
courses. Journal of Research on Science Teaching, 30, 905– 918.
Flavell, J.H. (1985). Cognitive development (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
 458 G. Taasoobshirazi and J. Farley

Glynn, S.M., & Koballa, T.R., Jr. (2006). Motivation to learn in college science. In J.J. Mintzes &
W.H. Leonard (Eds.), Handbook of college science teaching (pp. 25–32). Arlington, VA: National
Science Teachers Association Press.
Glynn, S.M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Con-
struct validation with nonscience majors. Journal of Research in Science Teaching, 46(2), 127–146.
Heckler, A. (2010). Some consequences of prompting novice physics students to construct force
diagrams. International Journal of Science Education, 32, 1829– 1851.
Hewitt, P.G. (2006). Conceptual physics. Boston, MA: Prentice Hall.
Kline, R.B. (2005). Principles and practice of structural equation modeling. New York: Guilford Press.
Koch, A. (2001). Training in metacognition and comprehension of physics texts. Science Education,
85, 758 –768.
Kohl, P.B., & Finkelstein, N.D. (2006). Effects of representation on students solving physics pro-
blems: A fine-grained characterization. Physical Review Special Topics: Physics Education
Research, 2, 1 –12.
Neto, A.J., & Valente, M.O. (1997, March). Problem solving in physics: Towards a synergetic metacog-
Downloaded by [Colorado College] at 12:14 09 December 2014

nitively developed approach. Paper presented at the annual meeting of the National Association
for Research in Science Teaching. Retrieved from http://www.eric.ed.gov/ERICDocs/data/
ericdocs2sql/content_storage_01/0000019 b/80/16/59/d1.pdf
Pett, M.A., Lackey, N.R., & Sullivan, J.J. (2003). Making sense of factor analysis: The use of factor
analysis for instrument development in health care research. London: Sage.
Rozencwajg, P. (2003). Metacognitive factors in scientific problem-solving strategies. European
Journal of Psychology of Education, 18, 281–294.
Schraw, G. (2001). Promoting general metacognitive awareness. In H.J. Hartman (Ed.), Metacogni-
tion in learning and instruction: Theory, research and practice (pp. 3–16). Boston: Kluwer.
Schraw, G., Crippen, K.J., & Hartley, K. (2006). Promoting self-regulation in science education:
Metacognition as part of a broader perspective on learning. Research in Science Education, 36,
111– 139.
Schraw, G., & Dennison, R.S. (1994). Assessing metacognitive awareness. Contemporary Edu-
cational Psychology, 19, 460 –475.
Schumacker, R.E., & Lomax, R.G. (2004). A beginner’s guide to structural equation modeling (2nd
ed.). Mahwah, NJ: Erlbaum.
Schunk, D.H. (2008). Learning theories: An educational perspective. New York, NY: Prentice-Hall.
diSessa, A.A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition
and Instruction, 22, 293 –331.
Tabachnick, B.G., & Fidell, L. (2000). Using multivariate statistics (4th ed.). Boston: Allyn & Bacon.
Veenman, M.V.J. (2005). The assessment of metacognitive skills: What can be learned from multi-
method designs? In C. Artelt & B. Moschner (Eds.), Lernstrategien und Metakognition: Implika-
tionen fur Forschung und Praxis [Learning strategies and metacognition: Implications for
research and practice] (pp. 75–97). Berlin: Waxmann.
Veenman, M.V.J., Van Hout-Wolters, B., & Afflerbach, P. (2006). Metacognition and learning: Con-
ceptual and methodological considerations. Metacognition and Learning, 1, 3 –14.
Williams, D.J., & Noyes, J.M. (2007). Effect of experience and mode of presentation on problem
solving. Computers in Human Behavior, 23, 258–274.
Zimmerman, B.J. (2006). Development and adaptation of expertise: The role of self-regulatory pro-
cesses and beliefs. In N. Charness, P.J. Feltovich, R.R. Hoffman, & K.A. Ericsson (Eds.), The
Cambridge handbook of expertise and expert performance (pp. 705 –722). New York, NY: Cam-
bridge University Press.
Zimmerman, B.J., & Campillo, M. (2003). Motivating self-regulated problem solvers. In J.E.
Davidson & R.J. Sternberg (Eds.), The psychology of problem solving (pp. 233–262).
New York, NY: Cambridge University Press.
 Construct Validation 459

Appendix
Physics Metacognition Inventory (PMI)
In order to better understand how you solve problems in physics, please respond to
each of the following statements from the perspective of:
When solving physics problems. . ...

Never True of Rarely True of Sometimes True of Usually True of Always True of
Myself Myself Myself Myself Myself

1 2 3 4 5

1. I think about what a physics problem is asking before I begin to solve it_ 1 2 3 4 5
Downloaded by [Colorado College] at 12:14 09 December 2014

2. While solving a physics problem, I ask myself periodically if I am 1 2 3 4 5

meeting my goals_______________________________________________
3. I ask for help when I don’t understand a physics problem____________ 1 2 3 4 5
4. I draw free-body diagrams to help me solve physics problems________ 1 2 3 4 5
5. I am a good judge of how well I solve physics problems_____________ 1 2 3 4 5
6. When solving physics problems, I know how I work best_____________ 1 2 3 4 5
7. When solving physics problems, I have a specific purpose for each 1 2 3 4 5
strategy I use___________________________________________
8. I go back and check my work after solving a physics problem_________ 1 2 3 4 5
9. After solving a physics problem, I double check my answer__________ 1 2 3 4 5
10. I use free-body diagrams to help me solve physics problems_________ 1 2 3 4 5
11. When solving a physics problem, I know how to apply a strategy to 1 2 3 4 5
successfully solve the problem_____________________________
12. When solving a physics problem, I know why I’m using a 1 2 3 4 5
particular strategy_______________________________________________
13. When solving a physics problem, I know when to use a 1 2 3 4 5
particular strategy_______________________________________________
14. Before solving a physics problem, I think about what a reasonable value 1 2 3 4 5
for the answer would be__________________________________
15. While solving a physics problem, I ask myself questions 1 2 3 4 5
about how well I am doing__________________________________________
16. While solving a physics problem, I periodically evaluate how well 1 2 3 4 5
I am doing_______________________________________________
17. After solving a physics problem, I look back to see if I did the 1 2 3 4 5
correct procedures_____________________________________________
18. I know why free-body diagrams are important for physics problem 1 2 3 4 5
solving_________________________________________________
19. Before I start solving a physics problem, I plan out how I’m 1 2 3 4 5
going to solve it______________________________________________
20. Before solving a physics problem, I identify all the important parts 1 2 3 4 5
of the problem________________________________________________
21. While solving a physics problem, I ask myself if I am meeting my goals_ 1 2 3 4 5
22. I seek help when I don’t understand the physics problems that 1 2 3 4 5
I am solving_____________________________________________
23. I draw free-body diagrams for the physics problems I am solving_____ 1 2 3 4 5
24. Before solving a physics problem, I eliminate information in the 1 2 3 4 5
problem that I don’t need_________________________________________