Library & Information Science Research 33 (2011) 261–268

Contents lists available at ScienceDirect

Library & Information Science Research

Re-framing information literacy: Problem-based learning as informed learning

Anne R. Diekema a,⁎, Wendy Holliday b, Heather Leary c
a
Instructional Technology and Learning Sciences, 2830 Old Main Hill, Utah State University, Logan, UT 84322, USA
b
Merrill-Cazier Library, 3000 Old Main Hill, Utah State University, Logan, UT 84322, USA
c
Institute of Cognitive Science, 594 UCB, University of Colorado, Boulder, CO 80309, USA

a r t i c l e i n f o a b s t r a c t

Available online 23 August 2011
This study explores an online information literacy module that uses problem-based learning (PBL). The goal
was to enable students to experience information literacy in a richer way, by moving away from a focus on
locating information sources to one of information use in the construction of knowledge. A content analysis of
the research journals and reflection papers of students (N = 15) in a distance education school library media
administration endorsement program suggests that PBL was an effective approach for some students, but not
others. Some students were motivated by working on authentic problems, and at least half the students
engaged deeply with information and discovered new questions and angles for research during the process.
These students applied more sophisticated evaluation strategies and were more metacognitive in their
thinking, assessing their progress and shifting strategies as they progressed through the module.
© 2011 Elsevier Inc. All rights reserved.

1. Introduction
Information literacy (IL) is defined as “recognizing when infor-
mation is needed and knowing how to locate, evaluate, and use it
effectively” (Association of College & Research Libraries, 2000).
Librarians, especially in the United States, have used this definition
to create standards, learning outcomes, and instructional activities
that focus on information search and evaluation skills and the use of
information technology. Library instruction often emphasizes the use
of tools and techniques. Bruce (2008) critiques this tool-based
approach: “Confining information literacy to such skills denies
learners the rich potential that may be gained from broader attention
to different ways of experiencing information use in the disciplines,
the professions, and community” (p. 5).
Bruce (2008) suggests that the notion of “informed learning,” also
referred to as “using information to learn,” can bring “learner-
centered, experiential, and reflective approaches to the information
literacy agenda” (p. 5). She uses a relational model, which posits that
learning involves an appreciation of different ways of seeing or
experiencing the object of learning, such as a concept or practice.
Educators, then, need to help students use different lenses to see or
approach a concept, problem, or practice. Bruce's research outlines six
frames for informed learning: content, competency, learning to learn,
personal relevance, social impact, and relational. Ideally, these frames
should be balanced. But the competency frame, which defines IL as a
set of discrete competencies or skills, is the dominant lens in IL
⁎ Corresponding author.
E-mail address: anne.diekema@usu.edu (A.R. Diekema).
instruction for kindergarten through 12th grade, and higher education
in the United States. Students become competent in IL skills by
“observing skillful practitioners” and “following predetermined
pathways”(Bruce, 2008, pp. 25–26). While many librarians and
educators employ teaching strategies and models that include
elements of learning to learn and critical thinking, IL practice tends
to privilege easily-defined skills, often in a linear model that
emphasizes the search component of the research process.
2. Problem statement
Academic librarians have expressed concern about student
research skill deficiencies (Asher, Duke, & Green, 2010; Kolowich,
2010). Studies have shown that students have tended to use Google
and Wikipedia for quick information fixes, rather than going deeper
into a question by using academic databases commonly found in
libraries. College students have also tended to find just enough
information to make do for their assignments, and they have rarely
conferred with librarians or other information professionals available
at their institutions (Head & Eisenberg, 2009). Even more seriously,
freshmen have appeared ill-equipped to analyze and synthesize
information, which is a key information literacy skill (Fitzgerald,
2004). Despite the integration of information literacy in educational
standards for kindergarten through 12th grade students (American
Association of School Librarians, 2007) and the widespread adoption
of library instruction at the college level, there is a growing concern
among educators and librarians that students are developing
superficial information literacy skills.
Lupton and Bruce (2010) have described this level of information
literacy as generic. For example, a lesson might introduce students to

0740-8188/$ – see front matter © 2011 Elsevier Inc. All rights reserved.
doi:10.1016/j.lisr.2011.02.002
262 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
searching a database but does not address how to process, analyze,
and apply the information found during that search. While generic
skills are needed, Lupton and Bruce (2010) have suggested informa-
tion literacy should also be taught as situated (a range of social prac-
tices based on disciplinary, work, or community-based contexts) and
transformative (practices used to transform individuals and society).
In an effort to expand beyond the generic information literacy
perspective, the authors designed and evaluated an online course
module for students in a school library media specialist endorsement
program. The study explored the question: How well does the course
module provide a new conception of information literacy that moves
beyond a purely skills-based approach? Specifically, the researchers
investigated problem-based learning (PBL) as a potential method to
reframe current models of information literacy instruction as situated
and informed learning.
PBL is a constructivist pedagogy that enables students to
experience information literacy in a richer way, moving away from
a focus on locating information sources, to one of information use in
the construction of knowledge. PBL is a potentially useful device
because it emphasizes application, synthesis, and using information in
a specific problem context. Recent library and information science
literature suggests that a sociocultural approach might enliven the
study and practice of information literacy (Lloyd, 2010; Steinerova,
2010), but much of this work remains theoretical. This study tested a
practical teaching strategy, problem-based learning, to explore
whether it could help students experience information literacy in a
more engaged, situated, and authentic way, especially in modeling the
vital connection between information seeking and information use.
3. Literature review
3.1. Teaching information literacy
Several scholars have critiqued information literacy practice as
driven by positivist and behavioral educational theories, which
support the mastery of isolated skills (Lloyd, 2007; Montiel-Overall,
2007; Simmons, 2005; Tuominen, Savolainen, & Talja, 2005; Webber
& Johnston, 2000). IL practice in the United States, especially as
represented in the Association of College and Research Libraries
(ACRL) Information Literacy Competency Standards for Higher
Education (ACRL, 2000), has been particularly criticized for a
tendency to reinforce a mechanistic approach to information literacy,
one that emphasizes information location and a generic recipe for
library-based research skills (Johnston & Webber, 2003). Grafstein
(2002) suggested that librarians' emphasis on information as a
discrete thing to be accessed and transmitted tended to isolate
information retrieval from the process of knowledge construction in
disciplinary contexts. Instead of a library-centric practice, information
literacy should be a shared endeavor, with librarians teaching the
“enabling skills that are prerequisite to information seeking and
knowledge acquisition across the curriculum” (Grafstein, p. 202).
Much of the critical information-literacy literature focuses on
issues related to the development and deployment of IL standards
since the 1990s. More recent work argues for a closer theoretical
examination of the connection between conceptions of information
literacy and broader assumptions about learning. Lipponen (2010)
argued that the ACRL standards, for example, will be always be limited
because they are based upon an acquisition framework of learning
that views the mind as a container of knowledge, and learning as a
matter of filling the container. For Lipponen, information literacy is
better approached via sociocultural theories of learning. Specifically,
information literacy is a situated and distributed activity, learned in
specific contexts, between people, and through practical activity.
Montiel-Overall (2007) also argued that scant attention has been paid
to the actual pedagogy of information literacy, stating: “a cultural
model of information literacy based on constructivist learning theory
focuses on abilities and thoughtful consideration of information,
rather than on mechanisms used to find and organize it” (p. 59). In her
model, “what is important about developing information literacy is
not information itself, but the connection learners make with
information as a means of constructing knowledge with others”
(p. 63).
The ACRL standards, with their emphasis on individuals and
generic processes, fail to capture the contingent and situated
character of information literacy, especially as it relates to the notion
of competency. Talja and Lloyd (2010) described how sociocultural
theories of learning assume that competency “originates not from the
heads of individual actors but among members of a community
practitioners” (p. xii). Elsewhere, Lloyd (2006) suggested that
becoming information literate is not a matter of mastering a set of
skills, but of learning, through practice, and guided by members of a
community, to navigate an information environment in increasingly
complex ways. Viewing information literacy and learning in this way
calls into question not only information literacy standards them-
selves, but also larger questions of pedagogy and assessment. The
current accountability movement in higher education tends to
privilege learning outcomes that can be easily measured, such as
the ability to search a library database or evaluate a website. This
reinforces a generic skills-based pedagogy of information literacy.
Most assessments of student learning tend to be local initiatives
designed to measure the impact of specific information literacy
programs or instructional approaches. Project Information Literacy
(PIL) stands out as a large, multicampus study designed to explore
how students in the United States approach information in their
academic work and their daily lives (Head & Eisenberg, 2009). PIL is
not an explicit study of learning outcomes. Instead, surveys and
follow-up interviews were conducted with college students to
explore their information behavior and their competencies. They
were especially interested in identifying situations in which students
struggle in finding and using information for college level work.
According to Head and Eisenberg (2009), in their first survey of more
than 2300 students from six colleges, “Students in our study
developed information strategy [sic] that was learned by rote, applied
with dogged consistency, and resulted in respectable grades” (p. 34).
In their most recent study, Head and Eisenberg (2010) reported on a
second survey of more than 8000 students on 25 U.S. college
campuses. They noted that the routine strategies that students
applied to college research assignments were learned in high school
and transferred to college (p. 3). Furthermore, while most students
were confident in locating information, they had difficulties getting
started, and framing academic research inquiry.
Smaller, qualitative studies have sought to explore the relationship
between students' approach to information and their performance on
broader learning outcomes, such as critical thinking. Limberg (2000)
studied the connection between student conceptions of information
seeking, content knowledge, and learning. She found that students fell
into three categories: fact-finding, balancing information to find the
right side, and scrutinizing and analyzing. Students who approached
information seeking as fact-finding tended to develop fragmentary
knowledge of their research topics. Furthermore, students' search and
technical abilities were consistent across groups, suggesting that more
deeply analytical approaches to information are not necessarily
related to search skills. Lupton (2008) found similar results in a
phenomenographic study of information literacy and learning.
Students experienced information literacy in one of three ways:
seeking evidence, developing an argument, or learning as a social
responsibility. Students who approached IL as seeking evidence
tended to locate information only to back up a preexisting argument
or viewpoint. These rich, qualitative studies are hard to generalize, but
they do confirm the PIL surveys and a large body of anecdotal
literature that suggest students are often learning and experiencing
information literacy in a superficial and limiting way.
 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
3.2. Problem-based learning
Problem-based learning is a constructivist pedagogy that might
enable students to experience information literacy in a richer way. PBL
began in the 1960s in the medical school at McMaster University
(Barrows, 1996). Today it is used in many disciplines and educational
contexts (Savery, 2006; Savery & Duffy, 1995; Walker & Leary, 2009).
Barrows (2002) defines PBL as learning with authentic, ill-structured
problems to solve, where the process of solving the problem takes
priority over the answer. It is a student-centered approach in which
students gradually assume more responsibility for their learning by
identifying and carrying out the direction of the learning and instructors
taking the role of a tutor or facilitator guiding the learning. PBL uses
authentic, real-world problems that are cross-disciplinary, with stu-
dents working in small groups. In face-to-face classrooms, PBL has
shown positive gains in cognitive student outcomes, as compared to
traditional learning (Albanese & Mitchell, 1993; Dochy, Segers, Van den
Bossche, & Gijbels, 2003; Gijbels, Dochy, Van den Bossche, & Segers,
2005; Vernon & Blake, 1993; Walker & Leary, 2009). PBL also emphasizes
the development of critical reasoning skills and self-directed learning.
PBL facilitates the recursive use, evaluation, and synthesis of
information. It requires that students seek out the information,
determine its value, and then synthesize that information with
information provided by other group members. This can be a very
messy process that invariably produces more questions, requiring
students to seek out additional information to solve their problem.
Students often shift the focus of the problem through the process, so
they must be iterative and reflective in their information seeking,
evaluating, and synthesizing.
Problem-based learning has been explored as a potentially useful
instructional approach for information literacy. It aligns well with
sociocultural theories of learning, especially in the emphasis on
collaborative learning through practical activity. PBL can be more
engaging than traditional demonstration-based library instruction
(Spence, 2004) and it addresses various learning styles (Hsieh &
Knight, 2008). PBL provides a means to introduce information literacy
in a more complex way, especially to students who think they already
know how to search for information effectively (Macklin, 2001). It is
collaborative and promotes student interaction and peer learning
(Snavely, 2004). PBL also closely integrates information literacy with
disciplinary content, enabling students to learn subject matter,
information seeking, evaluation, and synthesis skills, and critical
thinking all at the same time. PBL can be integrated within a course or
an entire curriculum, and can provide students opportunities to
practice and further refine their information literacy skills (Snavely,
2004). Macklin (2001), Munro (2006), and Cheney (2004) provide
models of how to incorporate PBL into the traditional one-shot library
session. Little empirical research comparing PBL to traditional
approaches has been done, but one small experimental study suggests
that engineering students performed better with PBL-based library
instruction (Hsieh & Knight, 2008).
4. Procedures
The researchers in this study created an online information literacy
module that could be used as either a stand-alone course or as an
assignment or tutorial in an existing class. The course is available on
the Utah State University Digital Commons site and is licensed under a
Creative Commons site license (http://digitalcommons.usu.edu/
ocw_itls/3). The course was designed using PBL in order to assess
whether authentic scenarios can promote information literacy skills
within an academic setting.
The module introduces students to the practice of information
literacy through a scaffold that models the process of researching
health issues that possibly result from cell phone use. The model
shows how an individual goes through the process of making a
263
decision about cell phone use, including synthesizing differing
viewpoints on the issue. Students are then given a list of four
authentic scenarios (e.g., whether to eat only organic food). They can
choose one of these scenarios, or they can research their own
questions. Students are encouraged to use the cell phone case as a
model for researching their own scenario through prompting
questions that ask them to reflect on what the cell phone researcher
encountered at various stages. Students are also provided basic
information on information sources and search strategies. Finally,
they are provided a series of reflective questions to guide them
through the process of developing research questions, conducting
preliminary searches, reflecting on what they found, and then
synthesizing and applying information to make a final decision.
The steps in the module were loosely based on the Big6
information skills model consisting of task definition, choosing
information seeking strategies, location and access of information,
use of information, synthesis, and evaluation of the process (Eisenberg
& Berkowitz, 1990). The researchers decided to use the Big6 in order
to introduce the students—all future school librarians—to this
prominent information literacy model, and because of the model's
emphasis on reflection, information synthesis, and use.
4.1. Participants
Study participants were students (N = 15) in a distance education
school library media administration endorsement program. Approx-
imately half the students were graduate students and the other half
undergraduates. All of the students were female. The Information
Literacy module was a required assignment for one of the courses in
the program. Researchers asked permission of each student to use
their materials in the study. Participation status was not tied to
student grades and the instructor did not know who agreed to
participate in the study and who did not.
4.2. Data collection and analysis
Research journals and reflection papers were collected after
completion of the module. Please see Appendix A for a full description
of the journal and reflection papers. Each student's journal and paper
were kept together as a pair and analyzed together. All identifying
information was stripped from the documents to ensure anonymity. A
content analysis of the student journals and papers was conducted to
categorize the stages of the students' research processes, and to
identify emergent themes, such as learning breakthroughs or
problems (Clandinin & Connelly, 1994). Themes were identified in
the data through an iterative and inductive process through which the
researchers discussed divergent codes and came to agreement on the
final coding scheme.
In the first step of data analysis, the authors mapped out the
research process for each participant according to the Big6 stages
(task definition, information seeking strategies, location and access of
information, use of information, synthesis, and evaluation). The
authors were interested in determining the extent to which students
were superficially following the course module and completing their
search in a linear fashion, or whether they were authentically engaged
in their own scenario and using metacognitive strategies (as modeled
in the cell phone case) to refine their questions, search strategies, and
evaluate and apply information.
The researchers also coded the information choices that each student
made. Some students were assignment-driven and searched the same
sources in the same order (Google first, and then the database Academic
Search Premier) as outlined in the module and the cell phone case.
Others made more independent information choices, such as searching
a library database that they were already familiar with.
From the initial coding, three important elements emerged. Some
students described a highly iterative process, and exhibited
264 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268

metacognitive strategies to refine that process. They reflected on the
information from their initial searches, applied it to their problem or
scenario, reformulated their questions, and then began the search
process anew. They made information choices that were informed by
their new questions. Others followed the module steps more literally,
only searching for information in the tools suggested in the
instructional text and cell phone model. Using these three elements
(iteration, metacognitive strategies, and information choices), the
researchers developed six typologies, recoded the data using these
typologies, and assigned each student to one of the types, as outlined
in Table 1. Then the researchers analyzed the data further, looking for
correlations between common themes, such as information evalua-
tion, and the six typologies.
4.3. Limitations of the study
The relatively small sample size (N = 15) and the particular
endorsement program the students were enrolled in limits the
generalizability of the study to student populations as a whole.
5. Findings
Problem-based learning did appear to facilitate a new conception
of information literacy for many students, but not all. Many students
still approached their research assignment as one of finding sources or
evidence, often for a pre-existing opinion or decision. For some
students, however, PBL helped them reframe information literacy as a
process of using information to learn and make an informed decision.
Seven students displayed metacognitive strategies that enabled
them to conduct a more extensive research process. In their journals
and reflective papers, these students tended to pause and evaluate
their progress at various points. They took more iterative steps,
reflecting on what they found, and redefining their questions and
search strategies multiple times, rather than only once. They also
evaluated their understanding and reflected on the usefulness of the
various sources they encountered. One student, for example, initially
had difficulty understanding scholarly information on the topic, so she
re-assessed her search strategy and went to sources written in lay
language:
This search was the most useful in finding information I wanted. A
lot of that had to do with going to a student search site—
information more on my level. This search helped me solidify my
conclusions. The scientific journals and university reports were
often beyond my [level] of background understanding [Student 10 -
journal].
I didn't find it too difficult to sort through the information I found
after I selected which ones to print. It was harder selecting which
items to click on to look at. However, after reading a few, I became
a littler smarter in what bits of information were given with the
site. I could begin to tell if a site may be a possible good site
[Student 14 - journal].
Only three students followed the module in a strictly linear
fashion, which was a surprise, given that students can be strongly
focused on assignment requirements. Twelve students redefined their
task, probably because the cell-phone case and the prompting
questions asked if they discovered new questions along the way.
More students were assignment-driven in their information choices
than independent (9 versus 6), but as students exhibited greater
metacognition; they made more independent information choices (4
versus 3).
Three features of the module emerged from student journals and
reflective papers as potentially crucial to developing these more
robust approaches to information seeking and use: the use of
authentic scenarios, emphasizing research as an iterative process,
and the requirement to make a decision as the final assignment task.
All of these features highlight the value of a PBL-based approach for
promoting deeper engagement with information.
5.1. Authentic experience
Twelve students mentioned personal motivation in conjunction
with their scenario choice. Initial interest in their chosen topic helped
students maintain personal motivation when the work became
difficult. Most of the student noted that the connection with real life
in the scenario they chose was helpful. One participant described the
link between scenarios depicted in video games and their effects on
children's behavior after playing the games, and her own personal and
professional life:
I chose this [scenario] for a couple of reasons. First, when I was a
young mother this was a concern for my children. This gave me
the opportunity to reflect on my decisions as a parent and the
choices I made for my children. Secondly, because I work with
children every day, I see an increase in inappropriate behavior.
The question that came to my mind was, “Is it media influence or
just the day and age of insensitivity?” I wanted to assure myself
that it isn't lack of parenting or parental care [Student 4 -
reflection paper].
Another student was motivated to learn more in order to inform an
ongoing debate she was having with her daughter about organic food:

Another student wrote about how it became easier to find relevant I chose this scenario because my daughter and I have had an
information after she had learned more about her topic. She learned to ongoing conversation about this topic. She is convinced that
read for relevance as she went along. organic food is superior in all ways to conventionally-grown food,

Table 1
Typologies of student search processes.

Type Description Student ID # Total

Linear and assignment-driven
Linear and independent
Iterative and assignment-driven
Iterative and independent
Metacognitive and assignment-driven
Metacognitive and independent
Conducted research in a linear fashion and did not iterate or redefine their task as they used information. 8, 9 2
Information choices were driven by the assignment.
Conducted research in a linear fashion and did not iterate or redefine their task as they used information. 13 1
Made independent information choices.
Conducted a more iterative research process and redefined tasks part of the way through the process as 3, 6, 12, 15 4
they used information. Information choices were driven by the assignment.
Conducted a more iterative research process and redefined tasks part of the way through the process as 1 1
they used information. Made independent information choices.
Evaluated their search process throughout and made changes to information choices and tasks in a 2, 4, 16 3
nonlinear way. Information choices were driven by the assignment.
Evaluated their search process throughout and made changes to information choices and tasks in a 5, 10, 11, 14 4
nonlinear way. Made independent information choices.
 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
while I think there may be some benefits, but I think it is mostly a
marketing technique. Neither of us have been informed enough to
provide solid data to back up our opinions—now I do [Student 10 -
reflection paper].
One participant, a school teacher, linked her scenario selection to
her experiences as a teacher trying to motivate her students to learn:
“I really liked being able to choose my topic. As a teacher I noticed that
kids always worked harder and produced better results when they
could choose what they wanted to research” [Student 11 - journal].
Another student described authenticity as central to her engagement.
Knowing that her research might inform actual questions from
parents motivated her to look at information more critically:
A new question, which never occurred to me before, but does
now, is if I have parents ask me as an educator what I know about
video game playing and children's behavior, can I give them
reliable information? I think I can at least direct their search in the
right direction now [Student 14 - journal].
Enjoyment was also tied to persistence in the search process.
According to one student, “I enjoyed the topic that I was researching
and that made me want to find more information” [Student 12 -
reflection paper].
5.2. Research as an iterative process
One of the goals of the online module was to teach students that
research is not a simple process of selecting a topic, finding a few
information sources, and summarizing them in a paper. Rather,
research is cyclical, with new questions and issues emerging
throughout the research process as a result of reading and learning
more about a topic. The cell phone model included a section in which
the researcher identified new issues that emerged from what she had
read. This was intended to model to students that research is
question-driven and iterative. Good research means reading infor-
mation critically not only for answers, but for questions and new
angles.
Beginning with questions and uncertainty was difficult for some
students. More than half the students commented on the difficulty of
the research process. Three students referred to the cell phone model,
but most described their own experiences. Student 6 (reflection
paper) appeared to enjoy the early phase:
I enjoyed the process in the beginning of the process. I found
information about organically grown foods versus conventionally
grown, and pretty quickly found two sources of information that
supported the claim that organically grown food was no healthier
than conventionally grown.
The same student grew weary in the middle and frustrated by the
open-endedness that new questions evoked: “This became tedious,
because no matter how much information I find to some of my
questions, more questions will arise” [Student 6 - journal].
Other students found the cell phone model reassuring. Student 2
(journal) struggled with the beginning, but the cell phone model
helped her understand that confusion is a normal part of the early
stages of research:
“The first step is knowing what you need to know.” I copied this
quote from the learning module and it came in very handy when I
began my search. It took me several tries before I realized what I
was looking for.
One student even found the discovery of new questions highly
motivating:
265
I now understand how cyclical this research process is. There is so
much information; new questions are raised, needing further
research. I could never read up on everything. In fact the more I
searched the more I wanted to search. I would find something and
that would lead to me look further. I honestly think I could have
searched for months and still find a need to continue searching.
New information comes available and there is a lot of information
all ready (sic) out there [Student 15 - journal].
Nearly half of the students used new questions to create a better
understanding of their subject and to help them synthesize what they
were finding: “I also have two articles, viewpoint/opinion articles. I
loved both of these articles because they cleared up some points for
me and helped me to make a connection I had been missing prior to
reading” [Student 1 - journal]. Student 14 (journal) identified a new
issue that entirely shifted the way in which she evaluated the rest of
her research:
A big new question which was raised I hadn't thought of before
was does the personality of the children make a difference on
whether playing video games makes a difference in behavior.
That's kind of like a duh! I'd never thought about that before. But
guess what? In about at least three of the sources I found, they all
indicated that, from their research, that playing video games for
long periods of time does not necessarily cause negative behavior
in children. One study found that children with aggressive
personalities were aggressive after playing. However, nonaggres-
sive personalities were not. Now that, to me is significant.
Some students used new questions to shift the entire focus of their
research. They had not understood the most central issues for their
scenario initially, and research helped them identify more relevant
and productive questions. Finally, others used new questions to help
them formulate criteria for making an informed decision. According to
Student 10 (reflection paper):
I used the information by comparing and contrasting the data. I
narrowed down my topic to eliminate animal products, which
made the topic [unwieldy]. I focused on the nutritional benefits,
since there was no question about the financial differences. I did
include some information about environmental effects since that
was important to me in making a final decision.
5.3. Decision-making and engagement with information
Some students demonstrated deeper engagement with the
research process in many ways. They discovered new questions and
angles of research from reading through their preliminary search
results; they were more comfortable with ambiguity and conflicting
information; and they synthesized and applied information to make a
final decision. These students also tended to apply more sophisticated
evaluation strategies, and were more metacognitive in their thinking
about their questions and progress as they worked through their
scenario.
“Evaluation of information” was the most common code in the
analysis, occurring 88 times. Evaluating information for quality and
relevance is a major information-literacy learning outcome, and the
module was designed, through the cell phone modeling scaffold and
the use of PBL, to help students learn to evaluate information in the
context of problem-solving and decision-making. The module was
specifically designed to move students beyond a superficial checklist
approach to information evaluation. This approach provides students
with a set of criteria, such as authority and currency and critics of this
approach have argued that students engage with information
superficially (Meola, 2004). The module provides a series of
266 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
prompting questions which goal is to promote deeper thinking about
information, such as who is funding research.
The module was successful in getting some students to develop
more sophisticated evaluation strategies, but the checklist approach
and superficial evaluation was still a problem for other students.
Three strategies of evaluation emerged from the data. Two students
used a traditional checklist approach. These students tended to
identify elements such as the URL of a source to assess a source's
credibility, or the last site update to assess the currency of the source.
The following illustrates a typical checklist approach:
The copyright on the page is 2009, so this should be some good up
to date information. The last time the information was updated
was July of 2009, so again this should be valid up to day
information. They are members of the Association of Academic
Health Centers (AAHC) and listed as one of America's best
hospitals in 2009–10 [Student 13 - journal].
Students using traditional evaluation strategies also mentioned
new criteria from the cell phone model, such as the source of funding
for research, but they did not apply the new criteria to their own
evaluation process.
Most students identified deeper evaluative strategies, such as
looking at funding sources, to detect potential agendas. Unlike the
students above, they actually applied this question to their own
process. Several students used a more sophisticated strategy of
evaluating information in the context of use. They developed new
questions from information they found initially, and explored the
logic of potential agendas, rather than simply identifying the agendas.
When Student 9 encountered some information about video games
being potentially educational and beneficial, she explored what this
might mean to various research claims:
After reading about the cell phone research, I tried to think about
who my stakeholders would be here. At first glance, one would
assume that those who create and market video games would be
invested in having their games perceived as safe. However, it might
be that some of the more counter-culture games might actually
benefit from being seen as “dangerous.” This seems like something to
consider—would video game designers really want their games to be
viewed as educational or wholesome? [Student 9 - journal]
Students who used deeper evaluation strategies reflected on
whether the information made sense to them. Student 4 kept an open
mind, even though she suspected that video games are bad for
children. Instead of stopping at the answers she agreed with, she
thought carefully about claims and evidence:
The first article I read by Ron Kaufman at TurnOffYourTV.com
provided reassurance that I was right. But the more I read, I
thought the information sounded no more factual than my belief.
It seemed that it was a value issue, not factual. Even the links
provided seemed to have little research to validate the claims
[Student 4 - reflection paper].
Students who were more engaged with the information they were
finding throughout the research process also had an easier time dealing
with ambiguous and even conflicting information. Students who had
difficulty tended to identify ambiguity as a problem because the issue had
two sides, each with a bias. After doing some initial research, Student 3
(journal) said, “So, with this initial research, only more conflict ensues.
Both sides have a study, although I haven't looked directly at the studies
yet.” A few students seemed to want clear answers to make a decision, and
could not deal with conflict. Student 16 wrote in her reflection paper:
I started my research feeling quite confident that I would find tons
of information against video games, which I did to an extent.
There was a lot of information that was against violent video
games, but there didn't seem to be any information that was
conclusive. It was frustrating to me to find this because I wanted
there to be a flat out “YES! Violent video games cause your child
to be aggressive and you should never ever let them play!” Much
to my dismay the findings were more like “Well there is ‘pretty
good evidence’ that video games can have an adverse effect on
your children but there is no real conclusion.”
The students who were more comfortable with ambiguity tended
to be the same students who used complex evaluation strategies,
identified new questions and issues to further their understanding,
and applied new questions to their criteria for making a decision.
Student 14, for example, encountered a surprise early in her search for
information on video games:
The conflicting information (I believe) which surfaced is what is
touted as positive impact of video game playing. I believe this
conflict is over knowledge, and possibly values. What one person
values as a positive attribute, may not be what experts in the field,
or others tout. I would be interested in reading more research on
the positive effects and what constitute [sic] a positive effect
[Student 14 - journal].
Instead of identifying two biased sides, she identified how
different individuals could use the same information to reach a
different conclusion, based on a variety of criteria. This seeming
conflict energized her search, rather than frustrating it.
The use of problems helped several students apply the information
they found to make a decision. Again, these were the students who
seemed the most deeply engaged with information and the research
process. These students often identified criteria for making a decision
based on the information they found, rather than trying to find
information that matched their preconceived ideas. Student 14
changed her initial opinion about video games in her journal, for
example: “I am definitely not the over-zealous, concerned grandma I
was before I began my search. I feel very comfortable in letting the
synthesis of all the information I gleaned guide my decision. I feel it is
an informed and important decision.”
In the end, students were able to apply information to make a
decision about their scenarios, none of which had clear answers or
solutions. According to one student:
I feel much more confident with the information I received later
than the first information I found. However, I'm glad I found the
two different ways to search. Having the first batch of information
actually made the second batch more relevant. In other words, the
contract [sic] between the two is what really made me think,
analyze, critique, and make conclusions [Student 14 - journal].
6. Future work
The authors identified three areas for further study: motivation,
facilitation, and group work. While a subset of the students were
intrinsically motivated to pursue information in more depth (Anderson,
Greeno, Reder, & Simon, 2000), many students followed an easier route
consistent with what is known in literature on information-seeking
behavior in digital environments. These students demonstrated
horizontal information seeking, power browsing, and cross-checking
(University College London, 2008). Students skimmed one or two pages
of a website and never returned to it, read through titles, tables of
contents, and abstracts of journal articles without ever reading in depth,
and bounced around a few websites in quick succession to perform
cross-checks on information or facts. How can librarians get more
students, then, to increase their motivation to learn and engage with
 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
information, rather than collecting information superficially and from
the most convenient sources?
In the current study, students did not work in groups, as is
common in PBL. The organization of the module is set up to provide
guidance to the students during the different phases of the
information seeking process, but there is no facilitator who provides
feedback on the process itself, nobody to push the students to go
deeper. The authors want to investigate whether incorporating a
facilitator would change the learning outcome for the students.
Another avenue for exploration is whether making the module into a
group assignment would produce different learning outcomes. Recent
research suggests that the social aspect of learning information
literacy is important (Lloyd, 2005, 2007).
7. Conclusion
Much of the recent information literacy literature argues for using
a sociocultural approach, one that views information seeking and use
as situated in specific contexts. Much of this work is theoretical and
based on contexts such as the workplace or a community of practice.
Yet most formal information literacy instruction takes place in school.
Students' information behavior is influenced, sometimes in counter-
productive ways, by predominant skills-based and generic ap-
proaches to IL instruction. Grafstein (2002) and Simmons (2005)
argue that information literacy is more effectively taught when
integrated within disciplinary contexts.
This exploratory research provides evidence that problem-based
learning might offer another, less discipline-centered strategy for
introducing students to a more situated experience of information
literacy. It suggests that making decisions about authentic problems
might focus the learner's attention in new ways, and help shift students'
conception of information literacy from finding sources to using
information to learn. The authenticity of the scenarios deepened
students' engagement with the process, possibly because the stakes
were real and, therefore, higher than in a typical college essay
assignment. Problem-based learning seemed to provide a productive
context for supporting a view of research as iterative and question-
driven, and for supporting deeper engagement with information itself.
The cell phone scenario modeled the research process, providing a
scaffold to show students the connection between information seeking
and use. Some students were able to summarize the information they
found, assess its logic and validity in context, and then apply it to adapt
their research strategy and create a better understanding. In the end,
they were able to apply information to make a decision about their
scenarios, none of which had clear answers or solutions.
This research suggests that PBL might help students situate
information seeking and use in a personal and problem context. A
problem context focused less on a discipline or profession can help
librarians who often have to teach information literacy in a more
generic, “stand-alone” way because of particular institutional de-
mands. PBL can afford librarians an opportunity to teach a situated
information literacy without having to rely on disciplinary faculty as
teaching partners or on any particular instructional model, such as
course-integrated or stand-alone credit-bearing IL courses. Something
as simple as making a decision might provide librarians with a
practical, flexible, and extensible instructional strategy for transform-
ing IL practice from a “click here” approach to one that engages
students in social and transformative processes of making meaning.
Appendix A. Student instructions
Research journals and reflection papers were collected after
completion of the module.
Students were encouraged to use a public blog or an editable pdf
(provided for them) to keep their journals and were given journal
267
prompts for each module activity. Students were also given guidelines
for writing their reflection papers.
Journal prompts
Activity 1: In the journal record what you think of information
literacy. Use the following questions to guide your writing:
• How important is the skill of information literacy?
• Why is it important to understand that there are various types of
information that can be found in different ways, using different
tools?
• Why is it important to have good searching skills?
Activity 2: In your journal record your thoughts about choosing
one of the scenarios. Feel free to add any thoughts you have on the cell
phone case example. Use these questions to guide your thoughts:
• What do you already know about the scenario you selected?
• What are you uncertain about?
• What are some specific questions that you have about the issue?
• What do you need to know to understand the issues better?
Use these questions to write down some keywords you can use to
begin your searching.
Activity 3: In your journal, record your thoughts on your initial
search for information about your scenario. Use the following
questions to help guide your writing (you do not need to answer
each question).
• Where did the information come from?
• Are reporters from news organizations writing about only one or
two issues related to your scenario?
• Do any of your information sources seem to have an agenda?
• Did you find any research about your topic?
• Who is doing research about your topic? What kind of expertise do
they have?
• Who is funding research on your topic?
• What issues emerged as most important or newsworthy from an
initial scan of the information about the scenario?
• How did your search compare with that of the cell phone
researcher?
• Where did the cell phone information come from?
• Who funded the various research studies on cell phone use?
• Did all of her questions get answered?
• Did the blog commenters raise any new questions or concerns?
Activity 4: In your journal, record your thoughts on sorting
through information. Use the following questions to help guide your
writing (you do not need to answer each question):
• What did you learn about your scenario generally? Summarize your
"working knowledge" of your scenario. William Badke defines a
working knowledge of a topic as the ability to "talk about it for one
minute without repeating yourself." (William Badke, Research
Strategies, 2nd ed., iUniverse, 2004).
• Do you have more questions now? Are there still questions you feel
are unanswered?
• Is there conflicting information that needs more clarification? Is this
a conflict over facts or knowledge or is it conflict over values?
• Did you find enough information to make an informed decision?
• How difficult is it to sort through information? What was your
process for doing so?
Activity 5: In your journal, record your thoughts on doing more
searching, reading, and sorting. Use the following questions to help
guide your writing (you do not need to answer each question).
• How successful were you in finding more information?
• How did this new information contribute to your understanding?
268 A.R. Diekema et al. / Library & Information Science Research 33 (2011) 261–268
• How does this information compare to the information you found
initially?
• Compare using library search versus the Internet. Did you find
different kinds of information?
• Did you have to search in a different way?
Activity 6: In your journal, record your thoughts on synthesizing
information. Use the following questions to help guide your writing
(you do not need to answer each question).
• What solution did you come to?
• What were your steps for reaching this decision?
• Write a paragraph or two that specifically answers the questions
raised in your scenario:
o Are you going to approach your sibling about the impact of watching
violent television or video games? What are you going to tell her?
o Are you going to continue to eat organic food? Why or why not?
o Are you going to support the local recycling program? Or are you
going to advocate another solution that involves re-use?
o Are you going to advocate for manned or unmanned space
exploration? Why?
Guidelines for the reflection paper: (no more than 2 pages)
• Write down the scenario you chose.
• Describe why you chose this scenario.
• Describe your experience overall in looking for information to
answer this scenario.
• Describe how you used the information you found to formulate an
answer.
• Describe problems you had in coming up with an answer for the
scenario.
• Describe what you found most valuable in this process.
References
Albanese, M., & Mitchell, S. (1993). Problem-based learning: A review of literature on
its outcomes and implementation issues. Academic Medicine, 68(1), 52–81.
American Association of School Librarians (2007). Standards for the 21st century learner.
Chicago: American Association of School Librarians.
Asher, A., Duke, L., & Green, D. (2010, May). The ERIAL Project: Ethnographic research in
Illinois academic libraries. Academic Commons. Retrieved from http://www.
academiccommons.org/commons/essay/erial-project
Anderson, J. R., Greeno, J. G., Reder, L. M., & Simon, H. A. (2000). Perspectives on
learning, thinking, and activity. Educational Researcher, 29(4), 11–13.
Association of College and Research Libraries. (2000). Information literacy competency
standards for higher education. Retrieved from http://www.ala.org/ala/mgrps/
divs/acrl/standards/informationliteracycompetency.cfm
Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief
overview. New Directions for Teaching and Learning, 1996(68), 3–12.
Barrows, H. S. (2002). Is it truly possible to have such a thing as dPBL? Distance
Education, 23(1), 119–122.
Bruce, C. (2008). Informed learning. Chicago: Association of College and Research Libraries.
Cheney, D. (2004). Problem-based learning: Librarians as collaborators and consultants.
portal: Libraries and the Academy, 4(4), 495–508.
Clandinin, D. J., & Connelly, F. M. (1994). Personal experience method. In N. K. Denzin, &
Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 413–427). London: Sage.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based
learning: A meta-analysis. Learning and instruction: The journal of the European
Association for Research on Learning and Instruction, 13, 533–568.
Eisenberg, M., & Berkowitz, R. E. (1990). Information problem-solving: The Big Six Skills
approach to library & information skills instruction. Norwood, N.J.: Ablex.
Fitzgerald, M. A. (2004). Making the leap from high school to college: Three new studies
about information literacy skills of first-year college students. Knowledge Quest, 32(4).
Gijbels, D., Dochy, F., Van den Bossche, P., & Segers, M. (2005). Effects of problem-based
learning: A meta-analysis from the angle of assessment. Review of Educational
Research, 75(1), 27–61.
Grafstein, A. (2002). A discipline-based approach to information literacy. Journal of
Academic Librarianship, 28(4), 197–204.
Head, A. J., & Eisenberg, M. B. (2009). Lessons learned: How college students seek
information in the digital age. Seattle, WA: Project Information Literacy. Retrieved
from http://projectinfolit.org/pdfs/PIL_Fall2009_finalv_YR1_12_2009v2.pdf
Head, A. J., & Eisenberg, M. B. (2010). Truth be told: How college students evaluate and use
information in the digital age. Seattle, WA: Project Information Literacy. Retrieved
from http://projectinfolit.org/pdfs/PIL_Fall2010_Survey_FullReport1.pdf
Hsieh, C., & Knight, L. (2008). Problem-based learning for engineering students: An
evidence-based comparative study. The Journal of Academic Librarianship, 34, 25–30.
Johnston, B., & Webber, S. (2003). Information literacy in higher education: A review
and case study. Studies in Higher Education, 28, 335–352.
Kolowich, S. (2010, September 29). Searching for better research habits. Inside Higher Ed.
Retrieved from http://www.insidehighered.com/news/2010/09/29/search
Limberg, L. (2000). Is there a relationship between information seeking and learning
outcomes? In C. Bruce, P. C. Candy, & H. Klaus (Eds.), Information literacy around the
world: Advances in programs and research (pp. 193–218). Wagga Wagga, New South
Wales: Center for Information Studies, Charles Sturt University.
Lipponen, L. (2010). Information literacy as situated and distributed activity. In A. Lloyd,
& S. Talja (Eds.), Practising information literacy: Bringing theories of learning, practice,
and information literacy together (pp. 51–64). Wagga Wagga, New South Wales:
Center for Information Studies, Charles Sturt University.
Lloyd, A. (2005). Information literacy: Different contexts, different concepts, different
truths? Journal of Librarianship and Information Science, 37(2), 82–88. doi:
10.1177/0961000605055355.
Lloyd, A. (2006). Information literacy landscapes: An emerging picture. Journal of
Documentation, 62, 570–583.
Lloyd, A. (2007). Recasting information literacy as sociocultural practice: Implications
for library and information science researchers. Information Research, 12, 1–13.
Retrieved from http://informationr.net/ir/12-4/colis/colis34.html
Lloyd, A. (2010). Framing information literacy as information practice: Site ontology
and practice theory. Journal of Documentation, 66(2), 245–258.
Lupton, M. (2008). Evidence, argument and social responsibility: First-year students'
experiences of information literacy when researching an essay. Higher Education
Research & Development, 27, 399–414.
Lupton, M., & Bruce, C. (2010). Windows on information literacy worlds: Generic,
situated and transformative perspectives. In A. Lloyd, & S. Talja (Eds.), Practising
information literacy: Bringing theories of learning, practice, and information literacy
together (pp. 3–27). Wagga Wagga, New South Wales: Charles Sturt University.
Macklin, A. S. (2001). Integrating information literacy using problem-based learning.
Reference Services Review, 29, 306–314.
Meola, M. (2004). Chucking the checklist: A contextual approach to teaching
undergraduates web-site evaluation. portal: Libraries and the Academy, 4, 331–344.
Montiel-Overall, P. (2007). Information literacy: Toward a cultural model. Canadian
Journal of Information and Library Science, 31(1), 43–68.
Munro, K. (2006). Modified problem-based library instruction: A simple, reusable
instruction design. College & Undergraduate Libraries, 13(3), 53–61.
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions.
The Interdisciplinary Journal of Problem-based Learning, 1(1), 9–20.
Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and
its constructivist framework. Educational Technology, 35(5), 31–38.
Simmons, M. H. (2005). Librarians as disciplinary discourse mediators: Using genre
theory to move toward critical information literacy. portal: Libraries and the
Academy, 5(3), 297–311.
Snavely, L. (2004). Making problem-based learning work: Institutional challenges.
portal: Libraries and the Academy, 4, 521–531.
Spence, L. (2004). The usual doesn't work: Why we need problem-based learning.
portal: Libraries and the Academy, 4, 485–493.
Steinerova, J. (2010, March). Ecological dimensions of information literacy. Proceedings
of the Seventh International Conference on Conceptions of Library and Information
Science, Vol. 15(1).
Talja, S., & Lloyd, A. (2010). Integrating theories of learning, literacies, and information
practices. In A. Lloyd, & S. Talja (Eds.), Practising information literacy: Bringing
theories of learning, practice, and information literacy together (pp. iv–xx). Wagga
Wagga, New South Wales: Center for Information Studies, Charles Sturt University.
Tuominen, K., Savolainen, R., & Talja, S. (2005). Information literacy as a sociotechnical
practice. The Library Quarterly, 75, 329–345.
University College London (2008). Information behaviour of the research of the future. CIBER
briefing paper, 9. London, UK: University College London. Retrieved from http://www.
jisc.ac.uk/media/documents/programmes/reppres/gg_final_keynote_11012008.pdf
Vernon, D. T., & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis
of evaluative research. Academic Medicine, 68(7), 550–563.
Walker, A., & Leary, H. (2009). A problem based learning meta analysis: Differences
across problem types, implementation types, disciplines, and assessment levels.
Interdisciplinary Journal of Problem Based Learning, 3(1), 6–28.
Webber, S. A. E., & Johnston, B. (2000). Conceptions of information literacy: New
perspectives and implications. Journal of Information Science, 26(6), 381–397.
Anne Diekema is an assistant professor in the Department of Instructional Technology
& Learning Sciences in the Emma Eccles Jones College of Education and Human
Services at Utah State University. Her research interests include digital libraries,
information retrieval, information organization, and human information behavior. She
holds a master of library science and a Ph.D. from Syracuse University.
Wendy Holliday is coordinator of library instruction at Utah State University, where she
oversees efforts to integrate information literacy across the curriculum. She holds a master
of library science from the University of Illinois and a Ph.D. from New York University.
Heather Leary is a research associate in the Institute of Cognitive Science at the
University of Colorado at Boulder. She has participated in multiple teacher professional
development workshops helping teachers and school librarians incorporate technol-
ogy and online resources in their teaching. Her research interests include problem-
based learning, online educational resources, information literacy, and open access.
She holds a master of education and a Ph.D. from Utah State University.