Int J Digit Libr
DOI 10.1007/s00799-008-0036-y
REGULAR PAPER
The use of online digital resources and educational digital libraries
in higher education
Flora McMartin · Ellen Iverson · Alan Wolf ·
Joshua Morrill · Glenda Morgan · Cathryn Manduca
© Springer-Verlag 2008
Abstract This paper summarizes results from a national
survey of 4,678 respondents, representing 119 institutions of
higher education in the United States regarding their use of
digital resources for scholarly purposes. This paper presents
the following results: (1) demographics commonly used in
higher education to categorize populations such as institu-
tion type or level of teaching experience could not reliably
predict use of online digital resources, (2) valuing online
digital resources corresponds with only higher levels of use
for certain types of digital resources, (3) lack of time was a
significant barrier to use of materials while, paradoxically,
respondents indicated that they used them because they save
time, (4) respondents did not tend to intentionally look to the
Internet as a trusted resource for learning about teaching.
Keywords Digital library · Education · User study
1 Introduction
This article summarizes aggregate and summary results
from a national survey of higher education instructors in the
F. McMartin (B)
Broad-based Knowledge, Richmond, CA, USA
e-mail: flora.mcmartin@gmail.com
E. Iverson · C. Manduca
Carleton College, Northfield, MN, USA
A. Wolf
University of Wisconsin—Madison, Madison, WI, USA
J. Morrill
Morrill Solutions, Madison, WI, USA
G. Morgan
George Mason University, Fairfax, VA, USA
United States aimed at understanding the uses, motivations,
and barriers surrounding faculty members’ and instructors’
use of educational digital libraries.1
In addition, it examines discovery and use of educational
digital resources within the context of the growing desire on
the part of faculty members and instructors to use them, and
an increasing difficulty in their ability to find, access and
use them. The National Science Education Digital Library
(NSDL) aptly described the situation faculty members and
instructors face:
The rapid acceleration of information available via the
internet makes locating high-quality, accurate, and truly
useful educational resources challenging for teachers
and learners. Educators, in particular, need efficient
and reliable methods to discover and use science and
math materials that will help them meet the demands
of instruction, assessment, and professional develop-
ment. [39]
A growing movement is evolving to address the needs of
faculty members and instructors wishing to find and use
online digital materials. Since the mid 1990s the National
1 For the purposes of this discussion we use the term “digital library”
broadly, aligning our view with that of the Lynch and Garcia-Molina
definition: [digital libraries are] “systems providing a community of
users with coherent access to a large, organized repository of infor-
mation and knowledge” [31]. This definition is sufficiently broad to
include collections that formally identify themselves as digital libraries
(i.e., the National Engineering Education Digital Library or NEEDS—
The National Engineering Education Digital Library), those that are
associated within the Open Educational Resource movement (e.g.,
OpenCourseWare sites such as MIT or Utah State University) or are
campus-supported repositories or members of multi-campus consortia.
According to this definition, commercial entities may also be a collec-
tion, for example, JSTOR and the growing collection of materials held
in the iTunes University website sponsored by Apple.
123

Science Foundation (NSF) has devoted over 150 million
dollars to support this effort, particularly for the purposes
of improving STEM education at all educational levels. In
addition to the NSF, private foundations such as Andrew
W. Mellon Foundation and the William and Flora Hewlett
Foundation have expended hundreds of millions of dollars in
support of the Open Education Resource movement, which
aims to provide open online access to online digital learning
resources worldwide. This movement, led by the Open Cour-
seware Consortium, has grown to over 120 members in less
than 3 years [29]. Along with the development of such large
projects, there are rising numbers of consortia of institutions
and individual U.S. colleges and universities investing in the
creation and support of “local” educational digital libraries
and repositories in support of teaching and research. These
local efforts, supported by projects like the DSpace Fede-
ration [41] attest to the growing recognition that it will be
necessary to support the development and use of online digi-
tal resources in higher education.
Questions about users and how they use digital materials
have become central given the high cost of the development
and maintenance of educational digital libraries. This in turn
has prompted discussions of sustainability models for these
libraries [23,37]. To justify this investment, the actual use
and impact of the contents of the educational digital libraries
have become increasingly important to stakeholders. For all
these reasons several questions have come to the fore: What
do faculty members do with the online digital resources they
find in educational digital libraries? Do faculty value these
resources? How do they use them for teaching purposes?
What are the barriers to their use of both resources and educa-
tional digital libraries? What, if any, meaningful differences
are there between groups of users based on demographic
variables?
2 Prior work
To date, research attempting to answer questions regarding
the use of educational digital libraries by faculty members
and instructors in higher education in the United States or
internationally, has with a few exceptions, been relatively
small in scale, consisting of case studies or surveys of users
of individual libraries or small samples of non-users within
a specific discipline.
From the United Kingdom there is a growing body of lite-
rature regarding digital repositories and collections, inclu-
ding analyses of the ways that faculty and instructors use
these services. These studies tend to focus on institutional
repositories and the emphasis is on examining behaviors
associated with contributing to repositories rather than loo-
king at usage in teaching or more broadly in regards to aca-
demic careers or workflows. In general terms, these studies
123
F. McMartin et al.
found that there was less use of digital collections that might
have been anticipated, but that use of repositories was posi-
tively associated with the development of an e-learning stra-
tegy on the part of the institution [7]. Respondents’ reported
factors limiting their contributing behavior included,
being concerned about copyright issues and whether or not
the materials they find will work with other technologies,
such as course management systems [44]. Peer review of
materials in a repository was seen as a positive motivational
factor encouraging use [3,4]. These same issues were also
identified by Uijtdehaage, et al. [46] study of medical school
faculty in the United States.
Other research has focused on the use of educational digi-
tal libraries and collections. For the most part these have
examined usage of a single collection or library and were
associated with searching behavior rather than adoption or
adaptation of materials for teaching or other professional pur-
poses. For example, Borgman [9] found that faculty mem-
bers in geography at a large research university conducted
logical, methodological searches for online teaching or lear-
ning materials much like they do when conducting research
in their disciplines. The faculty members in this study were
more able to articulate search strategies for research purposes
than for teaching purposes. In contrast, Chang’s 2004 survey
[12] of post-secondary and K-12 educators who were regis-
tered users of the Bioscience Education Network found that
nearly a third of these respondents discovered the site from
another web page and only 18% reported finding it through
a search engine.
When examining what users are seeking Manduca et al.
[33] found that geoscience faculty members preferred to learn
about teaching within the context of the content they teach,
and that their teaching behaviors were highly influenced by
trusted colleagues. They confirmed their findings by compa-
ring results from a small set of faculty members (8 interviews
and 21 cognitive walkthroughs) using the web to search for
teaching materials, to the findings from a survey of 5,700 U.S.
geoscience faculty members [32]. The evaluation of the MIT
OpenCourseWare collection [11], while similar to the stu-
dies described above, was a large research effort involving
both surveys and interviews to learn more about the use of
MIT-OCW materials. Unlike the educational digital library
projects that hold primarily learning objects that address a
single concept or set of concepts, the OCW site contains the
artifacts associated with an individual course, e.g., the course
syllabus, assignments, lecture slides, etc. By far, the majority
of their users are self-described self learners (49%), 32% are
students and 16% describe themselves as educators. 46% of
the educators who used it reported reusing OCW materials
by adopting or combining them with other materials. 26%
reported using the OCW site for course planning purposes,
to prepare to teach a class (22%) or to enhance their own
personal knowledge about a subject or topic (19%).
Online digital resources and educational digital libraries
Studies of the use of educational digital libraries and
collections often evaluate the quantity of use and usability of
particular libraries. These studies for example, within NSDL
collections, have been conducted mainly for formative pur-
poses and tend to be unpublished [5]. Additionally, many of
these studies depended on usage statistics alone to address
questions such as: how many unique visitors come to a site?
What paths do visitors follow when exploring a site? What
are the most popular times for visiting a site or what are the
geographical locations of users [25,26,36]? While studies
such as these hint at how the library is used they give very
little insight into how the contents are used. Another relati-
vely large set of usage studies is more generally associated
with studies of user interface. These studies tend to examine
how users interact with a site and have been conducted pri-
marily to test the effectiveness of the site design [8,45,47].
Other studies, well described in Bishop, Van House and
Buttenfield’s examination of digital library evaluation prac-
tices [8], tend to rely on the case study method and ethnogra-
phic methods. While important additions to our knowledge
about use of a specific educational digital library, these stu-
dies provide limited insight into larger use patterns geogra-
phically, or by type of user, e.g., faculty member, attorney,
member of a political action group, etc. They do not add
significantly to our understanding of their use by the profes-
soriate in the U.S., nor to this population’s knowledge about
or understanding about educational digital libraries, both of
which, are major factors that motivated the research reported
in this article.
There are studies that examine the use of educational digi-
tal libraries or collections by a larger population of faculty
members and instructors. In 2004, Harley et al., surveyed
faculty members and instructors in the humanities and social
sciences at California universities, colleges and community
colleges to learn about their use of digital resources in tea-
ching and to some extent the rest of their professional lives.
Early in their research process they found when conducting
a series of focus groups, that faculty members and instruc-
tors, though users of a wide variety of digital resources did
not understand the formalized concept of a “collection” or
“digital library”.
This finding was so stark that Harley et al. used a frame-
work for the next phase of the survey research that avoided
the language typically used to describe digital libraries, e.g.,
“collections” or “libraries”. Instead the survey language cen-
tered on the use of online digital resources that they defined,
for example, as images and visual materials to more sophis-
ticated learning objects (e.g., simulations, animations, etc.)
including those considered “free” or “open” and those that
are proprietary. Their results suggest that faculty use a variety
of these resources both to help improve their students’ lear-
ning and as primary sources materials in their teaching. More
interestingly, however, is why they chose not to use digital
resources. Respondents reported as their primary reason for
non-use that digital resources did not support their teaching
approach. Other barriers to use included, lack of time as well
as obstacles such as the inability to “find, manage, maintain
and reuse them in new contexts” [24].
Green [21] conducted a multi-method study of how faculty
at liberal arts colleges in the U.S. found and used digital
images in their teaching. The researchers received survey
responses from 404 faculty members at 33 liberal arts insti-
tutions and completed semi-structured interviews with 296
faculty staff and administrators across all disciplines. Their
findings, which support Harley et al.’s research, revealed
that faculty used digital images either from their own col-
lections or from what they found using a search engine such
as Google. Faculty appeared to have little awareness of digi-
tal image collections or libraries. However they were more
aware of and used educational digital libraries more fre-
quently than they did the licensed collections in their own
institutional libraries. A majority of faculty in the study (over
75%) reported that the use of digital images had changed how
they taught. Faculty perceived a range of barriers to greater
and more effective use of digital images including: inade-
quate tools for aggregating and managing digital images,
issues over copyright, inadequate technical support, lack of
time to locate and learn how to use digital images and insuf-
ficient access to collections of images [21].
3 Study method
In this article, we report on the results of a national survey
of U.S. faculty members and instructors regarding the use
and non-use of online digital resources. The purpose of this
study was to provide a description of use by faculty members
and instructors at the national level and unrestricted to any
one educational digital library or collection. This research
study delved deeply into the barriers to use most frequently
mentioned in the literature [17,23] such as, time and resource
constraints, lack of access to high quality materials, lack of
adaptability of the materials themselves, and intellectual pro-
perty.
Focus groups were used to gather preliminary data to
inform the design of the survey instrument used to survey
US faculty members and instructors. Using grounded theory
procedures [20] we analyzed and identified the underlying
themes that emerged from the focus group data. Because of
the diverse nature of colleges and universities in the sample
for the intended survey, we conducted 11 focus groups with
a total of 60 participants. Two groups were conducted at
one research university, one at a community college, three
were conducted at primarily teaching universities, one group
each at two historically black colleges, one group at a libe-
ral arts college and two groups at the MERLOT International
123

Conference2 whose participants represented a wide variety of
institutions. We sought input from this range of faculty mem-
bers and instructors because we assumed that several factors
would be critical to understanding their need for online digi-
tal resources and their search and usage behaviors, such as,
type of institution, teaching experience, teaching load, type
of courses taught, etc. Analysis of the focus group transcripts
provided some evidence of the importance of these factors
and the survey was designed to further test the value of these
factors in predicting user behavior.
Our focus groups confirmed Harley et al.’s [23] findings
that faculty members did not know what educational digi-
tal libraries were. Furthermore, these faculty members and
instructors did not distinguish between a curated collection,
such as the BEN science network (http://www.bioscienet.
org) where only those items that have been peer reviewed are
made available and that of a simple of list of URLs that might
be found at a colleague’s website. These findings, in addition
to Harley’s, highlight how important it is to use the language
that potential respondents understand when designing survey
instruments. So like Harley, we avoided using language in
the survey questions3 associated with digital libraries, e.g.,
collection, metadata, etc. Instead, in order to improve the
face validity of the instruments, we carefully described the
contents of collections, e.g., scholarly articles, visual images,
historical documents, etc. and asked respondents how they
searched for and used these materials.
The survey instrument consisted of 105 items that inclu-
ded demographic information, questions about motivations
for use of materials, barriers to use and descriptions of use.
To minimize survey fatigue, the survey design employed skip
logic so that respondents were asked details about their use
of materials only after indicating they used them. Questions
covered how an individual faculty member or instructor used
particular kinds of online materials, e.g., animations, simu-
lations, scholarly resources, images, etc. (see Table 3 for a
definition of these materials), if they modified these materials
in any way and their motivations for the use of these mate-
rials. Survey participants were asked to rank their likelihood
of use of a digital collection as compared to other search
engines such as Google or Yahoo.com. Responders were
also asked a series of demographic questions regarding their
teaching experience, type of institution in which they work
and so forth. External validity was determined by pre-testing
the survey with approximately 20 faculty members from
the different types of institutions represented in the sample.
2 MERLOT (http://www.merlot.org) is a free digital library collection
designed for higher education and covers over 15 academic disciplines.
MERLOT holds an annual conference for its users and community
members, attracting upwards to 400 participants.
3 A complete list of the survey questions is available at http://serc.
carleton.edu/facultypart.
123
F. McMartin et al.
Table 1 Description of Survey Sample and US Institutions
Type of institution %Sample % US institution
Associate (2 years) 34 47
Bachelor’s granting (4 years) 29 28
Master’s granting 26 17
Doctoral 9 7
Other/NR 2 0
Testers were also asked to comment on the survey and six
in-depth interviews were conducted to determine if the ques-
tions were interpreted as intended. To ensure for internal
validity, some items were reverse coded and Likert scales
(e.g., very unlikely to very likely, or never used to very fre-
quently used) were used throughout. Cronbach’s Alpha was
run on each subscale discreetly with results of over 0.75 on
each.
3.1 The survey sample
To reach the study’s population, we approached institutions
to aid us in contacting their faculty members and instructors.
Using the Carnegie Foundation 2000 list of US institutions of
higher education, a broad invitation to participate was issued
to the majority of higher education institutions in the United
States. A likely contact was identified at each institution by
visiting the institution’s web site. Contacts were generally
head librarians, heads of faculty development, or academic
deans. Of the approximately 3,500 institutions contacted,4
more than 250 responded, and in the end, 119 institutions
agreed to participate. Table 1 shows how this survey’s sample
of institutions compares to the actual distribution of types
of institutions in the Carnegie list. For the purposes of the
administration of the survey, the sample was not stratified
with regards to institutional type because we felt that attemp-
ting to do so would negatively affect institutional buy-in and
implementation of the survey.
The institutional contacts were asked to send the survey to
their faculty via email with at least one reminder email. Based
on patterns of response timed with our reminders to the ins-
titutional contacts, this requirement was met by all. Faculty
members and instructors in STEM education were the pri-
mary audience for the survey and our goal was to reach and
survey the largest possible number of STEM faculty in the
4 The Carnegie Classification system from 2000 was used to describe
institutional type. We did not include those institutions classified as
“special” in the population of institutions invited to participate in the
survey. For reporting purposes we collapsed the categories into tradi-
tionally identified classes for analysis after testing combined groups for
internal consistency. For the full list of classifications, see: http://www.
carnegiefoundation.org/classifications/index.asp?key=791.
Online digital resources and educational digital libraries

United States at the full spectrum of institutions. However, Table 2 Descriptive statistics of survey sample
because issues of use are not unique to STEM disciplines, we Variable N %
felt that limiting the survey to only those faculty members
and instructors would make it too difficult for institutions Type of institution
to participate in the research. Consequently, we encouraged Associate (2 years) 1,144 26
institutions to include their entire faculty in their invitation Bachelor’s granting (4 years) 960 22
to participate in the survey. The majority of participating ins- Master’s granting 1,345 30
titutions sent the survey to their entire faculty body, though Doctoral 926 21
some sent it to a random sample of their faculty. By the end Other/NR 64 1
of the survey period (September 2006–January 2007) 4,678 College/University appointment
individuals from the 119 participating institutions responded. Adjunct 579 13
Of those respondents, 4,439 instructed students; the bulk of Tenured 1,834 41
the analysis was conducted on this group of instructors. Non-tenured 891 20
The demographics of the survey respondents can be found Permanent non-tenured 283 6
in Table 2. Nearly a third (30%) came from Masters gran- Instructor/lecturer/faculty assistant 531 12
ting institutions, a fourth from two-year or associate degree Librarian 93 2
granting schools (26%), followed by four-year Baccalaureate Other/NR 228 5
or Liberal Arts College or Universities (22%) and, Doctoral Nature of appointment
Granting Institutions (21%). The respondents were also pri-
Full-time 3,624 82
marily tenured faculty (41%) with slightly over ten percent
Part-time 734 17
reporting that they held adjunct status (13%), or were pri-
Other/NR 81 2
marily instructors, lecturers or held other non-tenure track
Discipline
positions (12%). The majority by far, held full-time positions
Life sciences (biological sciences, health sciences) 797 18
(81%) and 40% had tenure. It is likely that these faculty mem-
Physical sciences (chemistry, geoscience, physics) 343 8
bers were over-represented in the sample given that 46% of
Computer science and Engineering 286 6
all US faculty members hold part-time positions [1].
Most of the participating institutions chose to administer Mathematics 258 6
the survey to their entire faculty rather than isolating STEM- Social sciences 833 19
only faculty. When asked to indicate in which disciplines they Humanities and Arts 1,139 26
taught, more than one third of the responses (38%) represen- Professional schools (business, education) 737 17
ted a traditional STEM field (biological sciences, chemistry, Teaching experience
computer science, engineering, geoscience, health sciences, <1 year to 6 years 1,154 26
mathematics, or physics). Approximately 45% represented 7–15 years 1,201 27
the humanities, arts or the social sciences while about 20% >16 years 1,278 29
represented the professional schools, e.g., education, busi- NR 806 18
ness, etc. This breakdown may not reflect the instructors’ Use of course website/online course management
disciplinary training and respondents were allowed to select Yes 2,523 54
multiple disciplines. No 1,279 28
The sample was also made up of faculty who were highly Unsure 71 2
experienced instructors, with only one fourth having less than NR 566 13
7 years of teaching experience. This approximates the age Course type (respondents could be in multiple categories
distribution of higher education instructors with approxima- Face-to-face 3,620 82
tely 65% being older than 45 years old [25]. Slightly over Distance education/online 714 16
half of the sample (54%) reported using course management Hybrid 396 12
systems or had a course website. Almost all (95%) of the
respondents reported teaching face to face courses, though
almost a fifth (21%) reported teaching distance education or
online courses. Only 12% reported teaching hybrid courses, 4 Results
that is, courses that both meet face to face and are conducted
online. If a respondent noted he or she did not instruct stu- 4.1 What kinds of online digital resources do faculty use?
dents, they were skipped to the end of the survey answering
a question on the services offered by collections of digital To examine the relationship between the value of digital
resources, and final demographics. resources and their use more closely, we went back to our

123
 F. McMartin et al.

Table 3 Types of digital resources

Digital resource Definition

Digital images/visual materials and historical documents Drawings, photographs, digital video, art, posters, digitized documents and similar
representations of primary source material
Online simulations/animations Illustrations or programs that are created to present a process or concept. The concept
presentation may include user interaction (i.e. a user may input information to
modify a simulation/animation)
Online datasets Any online educational, business and government datasets, and scientific research
databases
Teaching or learning activities and exercises Materials found online that are used in instructing students (assignments, tutorials,
lab procedures, problem sets, case studies, etc.)
Online scholarly resources Online journals, scholarly articles, and other scholarly discussion groups or sites

Table 4 Frequency and means for use of digital resources

Type of learning material Never Rarely Occasionally Very frequently NR

Freq % Freq % Freq % Freq % Freq % M SD

Digital images, visual materials and historical documents 379 8 617 14 1,499 34 1,930 43 19 0 3.13 0.95
Simulations, animations 1,578 36 994 22 1,187 27 503 11 177 4 2.14 1.05
Datasets 1,161 26 826 18 1,204 27 1,007 23 241 5 2.49 1.13
Teaching, learning activities and exercises 750 17 694 16 1,425 32 1,275 29 295 7 2.78 1.07
Scholarly resources 352 8 382 9 1,102 25 2,265 51 338 8 3.29 0.95
Scale: 1 = Never Use, 2 = Rarely Use, 3 = Use Occasionally, 4 = Very Frequently Use. N = 4,678

focus group data. For although these participants tended to
value digital resources highly, they also failed to distinguish
between the different types of online digital resources, e.g.,
educational digital libraries, web pages, online journals, and
were not conscious of using a digital library specifically [38].
Focus group participants defined educational digital libraries
and digital resources broadly making few distinctions bet-
ween for example, a loose collection of PowerPoint slides
(available from a well known or trusted colleague’s web-
site) and a collection of materials such as MERLOT. Rather
than force a definition of digital library for the survey onto a
group of respondents who most likely not equate educatio-
nal digital libraries or collections as sources for these highly
valued online digital resources, we instead used only lan-
guage describing the digital resources. We then sub-divided
the resources into five categories as described in Table 3.
To examine how faculty members reported using these
materials on the survey, we looked at the “top box” score, i.e.,
the resource which faculty indicated they “very frequently
used”. Survey results indicated that the most popular types of
materials used by faculty members and instructors included
online scholarly resources (51%) and digital images/visual
materials (43%). About a quarter of the respondents (29%)
reported frequent use of teaching and learning activities or
online datasets (23%). Online simulations and animations
were used the least with only 11% of the respondents repor-
ting frequent use.
One hypothesis tested was that use of these various types
of online digital resources would vary based on a number of
the demographic variables, most particularly, type of insti-
tution, type of appointment or level of teaching experience.
To examine this hypothesis an ANOVA was computed and
while there were statistically significant differences, they
were extremely small and held no practical significance. This
was perhaps due to the large sample size, which can some-
times obscure the ability of hypothesis testing to provide
a meaningful picture of significant differences. Therefore
effect sizes [14] were also calculated to look for differences
across groups of institutions, and disciplines. Again, they
were found to be statistically significant, but at such a small
level (less than 0.15) that they hold no practical significance.
So, contrary to the hypothesis, the preferences illustrated in
Table 4 remained steady across all respondents regardless of
the demographic variable. These results call into question a
commonly held belief that more experienced faculty mem-
bers, who tend to be older are therefore less likely to use
technology simply does not hold up.
Table 5 shows that the vast majority of respondents
(approximately 60%) felt digital resources were of “great
value” to their instruction, potentially skewing the sample.
However, since we were most interested in what and how
materials were used, this potential bias was of less concern
at this level of the analysis. The high value of digital resources
did lead us to ask; does the high regard for these resources

123
Online digital resources and educational digital libraries
Table 5 Value of digital resources
Freq
Of great value to my instruction 2,766
Of some value to my instruction 1,608
Of no value to my instruction 65
I do not instruct students 87
No response 152
Total 4,678
translate into higher levels of use of digital resources in
teaching and in turn to the use of educational digital libraries
to find the resources?
To examine the interaction of value placed on digital
resources and level of use, we cross-tabulated the reported
level of use across the five categories of resources against
the level that they valued resources. Those users who respon-
ded that they greatly value these resources reported higher
levels of use than other respondents (Table 6). The most used
resources are digital images with 60% of these respondents
reporting frequent use (Mean = 3.44 on a four point scale: 1 =
Never Use, 2 = Rarely Use, 3 = Use Occasionally, 4 = Very
Frequently Use)5 and scholarly resources, with 65% of these
respondents reporting frequent use (Mean = 3.51). The other
categories of resources do not see this strong trend of frequent
use even by those who report valuing digital resources highly
in their instruction. While the number of individuals who sta-
ted that digital resources were of no value in their instruction
is quite low, the use pattern seems to be logically consistent
within each category. When respondents do not value digital
resources, they, unsurprisingly, tend not to use them. Howe-
ver, almost half of these respondents report at least rare use
of scholarly resources (Mean = 1.92) and 41% at least rarely
use digital images (Mean = 1.58), possibly reflecting that
increasingly, some resources are only available in a digital
format.
4.2 How do faculty members and instructors use digital
resources?
To more fully understand frequency of use, we must also look
to how the respondents used the materials, meaning; did they
use them for teaching purposes, professional development or
other scholarly activities? Respondents who were users of a
particular type of resource were asked to rate their likelihood
5 For questions regarding frequency of use and likelihood of use,
contrast coding (i.e., −2 being never use to +2 being very frequently
use) was used to demonstrate an aggregate predisposition for “one side
of the fence” or the other. Mean differences centered around a zero point
allow the reader to see, for example in Table 6 and Table 8, if there are
“head to head” differences across questions.
of using them in specific teaching or educational situations on
% a four point scale from Very Unlikely to Use to Very Likely
to Use (see Table 7). Respondents reported that they were
59 somewhat or very likely to use only a few of the resources
34 for teaching purposes. These included use of digital images in
1 lectures (Mean = 1.16), for professional development pur-
2 poses (Mean = 0.91), and as student study aids (Mean =
3 0.83). They were also somewhat likely to recommended tea-
100 ching and learning activities to students as study aids (Mean
= 0.94). The type of resource most likely to be used ove-
rall was that of Scholarly Resources (Mean = 1.33), for the
purpose of professional development as a teacher.
Overall, the level of usage of these resources suggests that
there may be barriers to better integrating them in teaching
and scholarly work.
Given the impact of respondents’ valuation of digital
resources on frequency of use, we analyzed likelihood of use
under specific scenarios and found a similar pattern. Exami-
nation of one of the more popular scenarios, presentation of
digital images in lectures (Table 8) shows that the likelihood
of use is highest by those who highly value digital resources
(Mean = 1.42), and conversely, if not valued, there is a fairly
low likelihood of use (Mean = −0.84). What is also apparent
is the difference between likelihood of use between users who
value them highly and those who see some value in their use.
Here too, likelihood of use drops quite considerably (Mean
= 0.67). This type of trend is seen across each of the resource
types, regardless of the frequency of use. For example, for the
least frequently used resource, simulations and animations,
the likelihood of use in lectures varied from a Mean of 0.62 by
those who valued them highly, to a Mean of −0.02 by those
who thought they held some value, and to the low of −1.00
by those who did not value them. There is a consistent pattern
within each resource type; the likelihood of use increased the
more respondents valued digital resources.
4.3 What are the barriers to use?
In order to better understand the use and lack of use of these
resources, we explored what, if anything, deterred faculty
members’ and instructors’ use of digital learning materials.
The results from the survey showed that, by far, the most
commonly reported obstacle respondents reported was time.
These results contrast with those from other studies regarding
barriers to use where time was reported as a barrier, but did
not emerge as the primary obstacle to use [3,21,23]. This also
contradicts other responses in our survey, including highly
ranking the use of these resources as a personal priority, and
that faculty members and instructors use digital resources
because it helps save time. This combined with results from
our focus groups, suggest that this response may be highly
nuanced, and will require deeper probing.
123
 F. McMartin et al.

Table 6 Frequency of use by reported value of digital resources to instruction

Great value Some value No value
M SD M SD M SD

Digital images, visual materials and historical documents 3.44 0.80 2.65 0.93 1.58 0.81
Simulations and animations 2.40 1.06 1.73 0.87 1.11 0.32
Datasets 2.72 1.12 2.13 1.05 1.36 0.80
Teaching, learning activities and exercises 3.05 0.99 2.37 1.04 1.27 0.61
Scholarly resources 3.51 0.80 2.96 1.05 1.92 1.14
Frequency of use scale: 1 = Never Use, 2 = Rarely Use, 3 = Use Occasionally, 4 = Very Frequently Use. N = 4,439

Table 7 Likelihood of use of digital resources for teaching and professional purposes
Type of use of learning material Scholarly Teaching, learning Datasets Simulations, Digital images, visual
resources activities and animations materials, historical
exercises documents

M SD M SD M SD M SD M SD

Presented during my lectures/class 0.52 1.47 0.61 1.40 0.21 1.47 0.44 1.39 1.16 1.17
Linked or posted on my course web- 0.38 1.58 0.53 1.54 0.03 1.56 0.17 1.52 0.50 1.55
site/course management system
Used in tests and quizzes −0.74 1.47 −0.40 1.56 −0.91 1.35 −1.09 1.26 −0.53 1.50
Recommended to students for review 0.70 1.43 0.94 1.27 0.31 1.49 0.59 1.38 0.83 1.34
and/or a study aid
For use in scholarly work, including grant 0.85 1.55 −0.74 1.51 0.29 1.62 −1.00 1.34 −0.18 1.61
writing
For use in my own professional develop- 1.33 1.12 0.40 1.50 0.67 1.42 0.22 1.49 0.91 1.61
ment as a teacher
Presented in the context of an online class −0.30 1.63 −0.29 1.64 −0.47 1.53 −0.38 1.54 −0.13 1.64
discussion
Assigned to students as part of a student −0.77 1.45 0.42 1.50 0.43 1.48 0.04 1.49 0.43 1.47
research project or problem-based lear-
ning assignment
Scale: −2 = Very Unlikely, −1 = Some What Unlikely, +1 = Some What Likely, +2 = Very Likely. N = 4,439

Table 8 Likelihood of use by reported value of digital resources to instruction

Type of use of learning material Digital images, visual materials, historical documents

Great value Some value No value

M SD M SD M SD

Presented during my lectures/class 1.42 1.01 0.67 1.27 −0.84 1.25

Linked or posted on my course web- 0.85 1.43 −0.17 1.54 −1.04 1.34
site/course management system
Used in tests and quizzes −0.25 1.53 −1.06 1.29 −1.48 1.00
Recommended to students for review 1.11 1.24 0.33 1.38 −0.54 1.28
and/or a study aid
For use in scholarly work, including grant 0.03 1.62 −0.59 1.52 −1.20 1.19
writing
For use in my own professional develop- 1.16 1.22 0.44 1.39 −0.40 1.32
ment as a teacher
Presented in the context of an online class 0.18 1.63 −0.71 1.47 −1.24 1.30
discussion
Assigned to students as part of a student 0.72 1.38 −0.13 1.45 −1.00 1.17
research project or problem-based lear-
ning assignment
Likelihood of Use Scale: −2 = Very Unlikely, −1 = Some What Unlikely, +1 = Some What Likely, +2 = Very Likely. N = 4,439

123
Online digital resources and educational digital libraries

Table 9 Barriers to use of digital materials

Item Mean SD

(a) I would use more digital resources if there were more / better training available to me on how to use this 0.42 1.36
technology in my classes (n = 3,740)
(b) I would use more digital resources if I had more time (n = 3,745) 0.94 1.21
(c) Digital resource use is not a priority for me (n = 3,717) −0.70 1.31
(d) I would use more digital resources if it was a greater priority at my institution (n = 3,699) −0.37 1.35
(e) I would use more digital resources if I had more access to technology (computers, classroom equipment, −0.07 1.52
etc.) (n = 3,726)
(f) I would use more digital resources if there were more useful digital resources available (n = 3,723) 0.45 1.35
(g) I would use more digital resources if I had more flexibility in the content/curriculum of my classes (m = −0.75 1.28
3, 719)
(h) I would use more digital resources if the technology were more dependable (n = 3,713) −0.07 1.43
(i) I would use more digital resources if my institution rewarded me for using them (i.e., an award, count 0.05 1.54
toward tenure, etc.) (n = 3,714)
Scale: −2 = Strongly Disagree, −1 = Some What Disagree, 1 = Some What Agree, 2 = Strongly Agree

The lack of availability of useful materials was seen by
a majority of respondents as a barrier. The only institutio-
nal motivator respondents reported as being an important
influence, was the availability of more or better training on
how to use this technology in the classroom (Mean = 0.42 on
a four point scale where −2.0 is Strongly Disagree and +2.0
is Strongly Agree). This finding lends support to the com-
monly held perception that faculty and instructors who use
digital resources in the classroom must by supported with
professional development. What was somewhat more sur-
prising was the ambivalence about institutional or external
motivators, e.g., institutional rewards for use or access to
technology. Responses to these types of motivators (items
e, g, h, and i in Table 9) hovered around 0.0, meaning half
the respondents were on each side of the statement. Two
potential barriers, that the use of digital resources was not a
personal priority, and that instructors’ use would increase if
they had more flexibility in the choice of content or curricu-
lum in their classes were not identified as being barriers by
most respondents.
4.4 How do faculty members and instructors find materials?
How users find materials and what role commercial Internet
search engines such as Google play in this search continues
to be an issue for the educational digital library community
[30]. These communities recognize the importance of sup-
porting exploration in digital resource discovery [16,34,35].
In our research we also appreciated the importance of unders-
tanding not only what is used and how materials are used
but also the role that digital libraries play in locating these
resources.
The results of our focus groups indicated that users relied
more heavily on trusted sources and personal networks for
finding materials than they did educational digital libraries
or general web browsing. When directing them to characte-
rize how they use the web to search for materials they used
in a class, Google (other search engines were rarely mentio-
ned) was the primary starting point for searching. Often they
reported starting with Google to find materials they already
knew to exist, e.g., a particular website attributed to a known
instructor, websites associated with established textbooks,
professional associations (academic and non-academic), or
governmental agencies. This behavior is a common search
pattern [41], and suggests that Google itself has become one
of the trusted sources identified in earlier studies [33,34].
In this way, faculty members and instructors typify behavior
associated with web “orienteering” [10,43], meaning that
they use resources they like as a springboard to explore other
sites linked to the original site. Using the original source as
a starting point supports their ability to re-find the “new”
sources.
Focus group participants may not have recognized the
digital resources they used as being part of an educatio-
nal digital library due to their reliance on Google to locate
resources. The results from the Google searches of these
faculty members and instructors obscured the type of
resource, making it difficult for them to recall if they obtained
resources from web pages, collections, or educational digital
libraries. Participants readily recalled the types of materials
and could describe how they used them, but they were not
conscious of having used an educational digital library or
collection to find them.
With this behavior in mind, the survey asked participants
to rate their tendency to use a web search engine (like Google
or Yahoo), and their tendency to use a collection of digi-
tal resources (like an educational digital library) for a given
situation. The survey instrument aligned the two different
drop-down response menus next to each situation described,
so a respondent could think in terms of one being more likely

123
 F. McMartin et al.

Table 10 Use of DL collection versus search engine for searching

When I want to find … DL collection Search engine
Mean SD Mean SD

(a) Information that provides students with context 0.69 1.49 1.21 1.24
(b) Examples that get students excited about a topic 0.43 1.50 1.23 1.23
(c) Current information (data, event information, etc.) for students 0.59 1.53 1.45 1.06
(d) Something that illustrates a difficult concept for students 0.50 1.51 0.87 1.41
(e) Non-technical background for students −0.02 1.55 1.06 1.68
(f) Primary source material that I can integrate into my course 0.87 1.49 0.60 1.55
(g) Lesson plans, syllabi, or exercised that I can integrate into my instruction −0.19 1.61 0.30 1.65
(h) Information about how to improve my teaching 0.21 1.61 0.06 1.61
(i) Education or pedagogy information 0.37 1.63 0.07 1.61
(j) Information for scholarly or my professional development as a teacher 0.93 1.45 0.43 1.55
(k) Information on an unfamiliar topic in my field 1.08 1.34 1.14 1.26
(l) Information in preparation of a grant proposal 0.63 1.64 0.14 1.66
(m) Information about professional activities, e.g., conferences, workshops −0.01 1.66 1.05 1.43
(n) Stay current on new developments in my field 1.09 1.37 0.56 1.52
Scale: −2 = Very Unlikely, −1 = Some What Unlikely, 1 = Some What Likely, 2 = Very Likely. N = 4,439

than the other. The situations were grouped by those items
that were focused on finding resources specifically for their
students and those that supported course planning.
The survey results reflected the focus group findings of
users’ reliance on search engines such as Google. Generally,
the survey respondents reported being positively inclined to
use both search engines and educational digital library collec-
tions, but there is a clear preference towards search engines
with a few exceptions (see Table 10). These were related
to specific tasks about teaching practice where respondents
were more likely to use a collection of digital resources to find
information about how to improve their teaching and when
they specifically sought out education or pedagogy informa-
tion.
5 Discussion
Lack of information about how faculty members and instruc-
tors use educational digital libraries (and in particular, use
the NSDL, their member digital libraries and the contents of
these collections) in their teaching and professional academic
lives motivated this research. Members of the NSDL digital
library community have long expressed frustration with the
lack of knowledge available regarding use [18] and how this
gap has negatively affected their ability to design and imple-
ment services that meet users’ needs. By surveying faculty
members and instructors at a national level, we attempted
to answer our primary research question: What do faculty
members do with the online digital resources they find (and
by extension how to they use educational digital libraries)?
We also explored several other questions: Do faculty value
these resources? How do they use them for teaching pur-
poses? What are the barriers to their use (of both resources
and educational digital libraries)? What, if any, meaningful
differences are there between groups of users based on demo-
graphic variables?
Through the results from both the survey and focus groups,
we examined the uses, motivations and barriers surrounding
faculty members’ and instructors’ use of educational digital
libraries. We identified factors that would help the designers
of these services align their services with how faculty mem-
bers and instructors work, as well as suggest services and
features that faculty members and instructors might find most
valuable when using an educational digital library. In the fol-
lowing section, we discuss four main themes that emerged
from the survey and focus group results.
5.1 Valuing educational digital resources does not always
translate into higher use
The trends with regards to frequency and likelihood of use
suggest that there is some barrier or set of barriers that limit
use by those who do not value them highly or who are not
highly motivated to use them. The disparity among the dif-
ferent types of resources in combination with the different
uses (teaching, research, or professional development) lends
support to the notion that adoption or use of digital resources
may occur best when the resource is analogous to what Steve
Gilbert describes as a “low thresh-hold application” [19].
Known as LTA’s these applications make use of technology
that is easy to use, fairly ubiquitous, is fairly low cost, and

123
Online digital resources and educational digital libraries

Table 11 Use of materials by

level of experience, institution Group Digital images, Teaching/learning
type and college/university visual materials and activities and
appointment historical documents exercises

Mean SD Mean SD

Level of experience
0–6 years 3.22 0.91 2.88 1.05
7 or more years 3.11 0.96 2.76 1.08
Institution type
Associate (2 years) 3.11 0.96 2.88 1.03
Bachelor’s (4 years) 3.13 0.94 2.65 1.08
Master’s 3.11 0.92 2.85 1.07
Doctoral 3.18 0.97 2.70 1.08
College/University appointment
Adjunct faculty 2.96 1.02 2.83 1.09
Tenured faculty 3.13 0.95 2.68 1.10
Non-tenured faculty 3.20 0.92 2.80 1.07
Permanent non-tenured faculty 3.23 0.90 2.86 0.93
Instructor/lecturer/faculty assistant 3.16 0.94 2.91 1.00
N = 4,439

essential to the discipline. The same idea appears to apply to
digital resources and almost certainly (along with the relati-
vely small size of the item) explains why digital images were
the most frequently used resource for teaching purposes and
is likely a factor in why animations and simulations are the
least used type of digital resource.
These findings also raise questions concerning expecta-
tions around the use of digital resources and the meanings
faculty assign to their use. For example, what does valua-
tion of a resource mean to an instructor? Similarly, what
does it mean that an instructor uses a resource frequently or
infrequently? Expecting an instructor to use any one of these
resources all the time or very frequently during a course or
class period simply may not make good pedagogical sense
nor match an instructor’s teaching or learning goals. Low
usage may only reflect the nature of the need for a resource
or the fact that there may only be one seminal resource that
pertains to a relatively small, but important segment of a
discipline’s population. In this case, it would be a mistake
to equate low, but carefully considered, usage with non-use.
The disparate interpretations associated with this issue indi-
cate the need for further exploration of the issue.
These results suggest that there might be services that
would increase usage in higher education. Knowing that
faculty members and instructors highly value and use digital
images, visual images and historical documents as teaching
aids suggests that educational digital libraries that do not
have these resources might consider developing collections
of images, or “image of the day/week” types of services to
help attract users. Given the interest in use of the materials
for their own professional development, collections might
also consider how to focus more on the professional needs
of faculty to help them use the full range of materials in
educational digital library collections rather than providing
only collections of learning materials focused towards use
with students. Providing the types of resources instructors
are seeking while at the same time offering scaffolding that
will support use of less commonly sought, but highly valued
resources, should help increase the use or value of educatio-
nal digital library collections to non-users.
5.2 Faculty are more alike than different in use of online
digital materials
One of the purposes of conducting this study was to learn
more about how faculty members and instructors differ in
their use or non-use of digital resources and educational digi-
tal libraries. Based on the findings of our focus groups, we
hypothesized that users would differ in their use based on
the following set of variables: type of institution, level of
teaching experience and type of academic appointment. For
the most part, our findings did not support that supposition.
In fact, we found that faculty members and instructors were,
with regard to these variables, more similar in their use of
these resources than they are different. Our examination of
the most popular resources and most clearly related to tea-
ching, digital images/visual materials and historical docu-
ments and teaching/learning activities and exercises, showed
few if any differences with regards to the variables listed
above (see Table 11). Considering that the majority of post-
secondary instructors have been trained in a relatively small
number of institutions and that there are rarely significant

123

changes in that training based on potential career path may
partially explain their behavior. If this is the case, it suggests
that investments in future faculty training in the use of these
materials may have the largest impact.
While the focus of this article has not been on these diffe-
rences, a few disciplinary patterns of use and non-use emer-
ged. At this point in our data analysis, we have found that
no one discipline uses teaching and learning activities signi-
ficantly differently from the others. Yet there is some indica-
tion that discipline may serve to be a predictor of the use of
different types of resources. Digital images demonstrates one
such potential area for further study, for it appears that mem-
bers of the Biological Sciences and Geosciences communi-
ties report using them slightly more than other disciplines,
especially when compared to the social sciences and huma-
nities and arts. This also contrasts with the findings from the
Harley et al. study that reported the social sciences, humani-
ties and arts as being strong users of these same resources.
This discrepancy confirms the need to explore these diffe-
rences and this analysis will be reported in more depth in
future publications.
5.3 Barriers to use cannot be simply defined
Given the disparity between respondents’ high value for digi-
tal resources and actual practices in using them, it is clear
that there are barriers that must be overcome for faculty
members and instructors to become regular users of these
resources. Research related to what motivates faculty mem-
bers and instructors to take on teaching distance education
courses found that the motivations took two forms: intrinsic
and extrinsic. Wolcott and Betts [48] found that for inten-
sive activities like teaching at a distance, intrinsic motivation
distinguished faculty who participated in distance education
from those who did not. Others suggest that faculty need
to be rewarded extrinsically to encourage them to change
their teaching behaviors to adopt an innovation. External
rewards range from support by the campus administration
(e.g., credit towards promotion and tenure for implementa-
tion of innovative teaching methods/scholarship of teaching)
and infrastructure support (e.g., teaching assistants, technical
assistance, adequate or more reliable technology) to training
in how to use the materials [13,22,27,42]. There is certainly
a combination of factors including the difficulty of the task
and the personal interest in the task, which drive what types
of rewards are required.
Other theories of change associated with adoption of new
technologies, most especially the Technology Acceptance
Model—TAM [2,15], focus on the perceived usefulness of
a technology and/or its perceived ease-of-use. These dimen-
sions are particularly important when considering barriers
other than motivational barriers. The TAM model may be of
critical importance when considering how the use of Google
123
F. McMartin et al.
defines how users search for materials, what their expecta-
tions are with respect to finding materials and their awareness
of educational digital libraries. In the focus groups, respon-
dents repeatedly reported that Google was their first choice
for finding information. Survey responses upheld the prefe-
rence for Google. However, educational digital libraries were
rarely described by our focus group participants as being dif-
ficult to use or not useful. In this case, when usability does not
seem to be an issue, this model may not be especially appli-
cable to use of educational digital libraries. However, it does
suggest that digital libraries should seek to leverage search
tools to bring users into their collections. If these libraries
wish to be recognized as the resource provider, the design
of these deep links needs to clearly identify the source of
the content and provide navigation that helps the visitor find
other materials.
In terms of barriers to use of educational digital libraries
and online resources, a large majority of the respondents to
our survey stated that lack of time as being a barrier to adop-
tion. The results from our focus groups contradicted this fin-
ding in the sense that faculty reported a willingness to spend
the time necessary to find the “right” materials. It also contra-
dicts responses from the survey where a similar number of
respondents noted that they used digital resources to save
them time. These results point out the complexity of naming
“time” as a barrier. The danger of including a question that
asks if time is a barrier is that it fails to force the user to reflect
on the ultimate barriers that lack of time reflects. We suggest
that it is not really lack of time, for all of us have finite time,
but rather an issue of priorities. For example, when people
say that they do not have time to perform a task, it may be
a polite way of saying it is not as high a priority for them as
the other tasks that they have to do.
Barriers to use for these respondents do not seem to be one
of lack of motivation, nor does use seem to be closely tied
only to factors associated with the Technology Acceptance
Model. When talking with our focus group participants we
heard many stories indicating the opposite. These instructors
were so invested in helping their students visualize a concept
or learn something that required a digital learning object that
they would spend hours looking through page after page of
Google hits in search of the “right” image or item that they
needed. This kind of commitment to helping students learn,
an important intrinsic motivational force, may surpass all
other types of motivators. Motivation to use these resources
seems to rely on personal characteristics not explored in this
research.
5.4 Learning about teaching is not yet an intentional web
activity
Previous studies have indicated that faculty members and
instructors for the most part prefer to learn about new ideas,
Online digital resources and educational digital libraries
innovations, and how to apply them by working with
colleagues [6,32]. The NSDL and other educational digi-
tal libraries were formed to support and promote innovation
in teaching and learning. Unlike other digital libraries, those
that focus on education are attempting to build high quality
collections of innovative learning materials within a speci-
fic discipline available for reuse with the intention of impro-
ving teaching and learning. Some of these libraries also wrap
around or connect these materials to information about how
to use them in teaching. The results from this survey however,
suggest that fewer instructors than hoped for use the Web for
these purposes, as indicated by low ratings regarding their
likelihood to use a search engine or a digital library to find
materials to improve teaching methods (Table 10h). This fin-
ding requires more exploration as it is at odds with other
research. For example, in the geosciences, recent research
estimates that 25% of these faculty members use Teach the
Earth resources that offer both content and pedagogical sup-
port [35]. This suggests that carefully designing targeted
resources with appropriate materials can be successful.
These results suggest that educational digital libraries
whose goal is to promote innovation in teaching and lear-
ning still face quite a few challenge in achieving this goal.
That said, our findings reveal that faculty members and ins-
tructors tend to prefer educational digital libraries as places
to stay current on new developments in their field and to seek
information for their scholarly or professional development
as a teacher. These preferences lend some support for buil-
ding services that support such activities within educational
digital libraries.
6 Conclusions, limitations and implications
Participants of the survey clearly did not need to be per-
suaded as to the value of some forms of digital resources.
But this is apparently not enough to ensure their use as the
respondents self-reported behavior shows fairly low levels
of use of materials thought to be of high instructional value,
such as, simulations [28] even though they value the gene-
ral class of digital resources highly. These survey results and
the results from our focus groups put forward numerous chal-
lenges facing educational digital libraries in informing poten-
tial users about their existence and in convincing these users
to use them to search for helpful pedagogical teaching and
learning materials, both for use with their students and for
their own use.
The results also suggest that there is a gap to be bridged
between offering digital learning materials of value and pro-
viding resources to encourage their use. The recent focus on
Web 2.0 and building social networks such as Flickr, Face-
book or YouTube, is seen as one way to begin to address this
challenge. The hope is that by providing faculty members
and instructors with tools to share their experiences in using
these materials, they will adopt the materials, use them, and
then contribute back to the site that provides access to these
resources. Our results, however, suggest that faculty mem-
bers and instructors, unlike other populations, may be reluc-
tant to engage in this type of social and professional interac-
tion and sharing of learning materials, at least with regards
to their teaching responsibilities. Perhaps most telling in that
regard is that nearly a third (28%) of our survey respon-
dents reported that they did not have a course website or use
online course management tools. Our focus group respon-
dents also offered some insight into this issue when they
responded that while they wanted to find, view and poten-
tially use other faculty members’ and instructors’ materials,
they were not as willing to reciprocate by posting or sharing
their own materials—behaviors that are essential to the Web
2.0 experience. It is not clear that whether this is a lack of
willingness to share, as it is an issue resulting from the private
nature of teaching and fear of criticism in a field (education)
in which they are often not formally trained, and whether
this might change as more faculty become immersed in the
scholarship of teaching and learning.
The heavy reliance on search engines, particularly Google,
indicates that there should not be a division between search
engines and educational digital libraries. It appears that a
synergistic relationship exists between the two in which the
search engine brings users to the content and the educational
digital library offers high quality content. Each adds value to
the other. This raises the importance of exposure of educa-
tional digital library contents to search engines, and design
of the internal portions of the library to make users arriving
from search engines aware of where they are and offering
these transient visitors a reason to stay. Failure to expose
materials in educational digital libraries to search engines
increases the costs for those libraries to bring in users, and
as users become even more focused on search, increases the
likelihood of their irrelevancy to users because they cannot
be found through a web search.
One potential criticism that might be leveled against this
research is that the majority of respondents seemed to
demonstrate a response bias towards highly valuing digital
resources. However, it is important to keep in mind that the
goal of this research was to examine how individuals use
digital resources in support of teaching and learning. This
being the goal, response bias is considerably less troubling.
More interesting is that these findings suggest a gap between
perceived value and actual use of digital resources. As the
demographics of the faculty shift in the next decade the ques-
tion will be less one of “are digital resources valuable?” to
one such as “what makes the resources and their organization
most valuable to our faculty and our students?”
This paper presents an overview of survey findings
where variables, typical to research on faculty members and
123

instructors in higher education, such as, institution type or
level of teaching experience, did not directly predict varia-
tions between respondents’ perceptions and their use of digi-
tal resources. Future examinations of the current data will
look at how demographic, attitudinal, and behavioral
variables may interact to more effectively predict educatio-
nal digital library use. As the broader goal of any research
on digital resource use should focus on how to align positive
perceptions of a digital resource with actual use of a digital
resource, the goal of future work will be to identify leading
indicators of digital resource use and explore how developers
of educational digital libraries can use these to successfully
influence use.
References
1. American Association of University Professors: Background
facts on contingent faculty. http://www.aaup.org/AAUP/issues/
contingent/contingentfacts.htm (2007)
2. Bargozzi, R., Davis, R., Warsha, P.: Development and test of a
theory of technological learning and usage. Hum. Relations 45(7),
660–686 (1992)
3. Bates, M.S., Loddington, S.P., et al.: Rights and rewards pro-
ject academic survey, final report. http://rightsandrewards.
lboro.ac.uk/files/resourcesmodule/@random43cbae8b0d0ad/
1150709518_Final_Report_of_Survey.pdf (2006)
4. Bates, M.S., Loddington, S.P. et al.: Rights and rewards in blended
institutional repositories project. ALISS Q. 1(2), 47–51 (2006)
5. Bartolo, L., Diekema, A., Khoo, M., McMartin, F.: Survey of Eva-
luation Projects in the National Science Digital Library, final report.
NSDL PI Meeting, Washington D.C., October (2006)
6. Bayard, J.P., Ehrmann, S., et al.: Learning through technology:
LT2 . http://www.wcer.wisc.edu/archive/cl1/ilt/default.asp (2000)
7. Bell, V., Rothery, A.: E-Sharing: Developing Use of E-repositories
and E-libraries for Learning and Teaching. West Midlands Share
Project: University of Worcester (2006)
8. Bishop, A., Van House, N., Buttenfield, B.: Digital Library
Use: Social Practices in Design and Evaluation. MIT Press,
Cambridge (2003)
9. Borgman, C.L., Leazer, G.H. et al.: How geography profes-
sors select materials for classroom lectures: implications for the
design of digital libraries. In: Proceedings of the 4th ACM/IEEE-
CS Joint Conference on Digital libraries, ACM Press, Tuscon,
Arizona (2004)
10. Capra, R.G. III., Perez-Quinones, M.A.: Using web search engines
to find and refind information. Computer 38(10), 36–42 (2005)
11. Carson S.: 2005 program evaluation findings report. MITOpen-
CourseWare. http://ocw.mit.edu/OcwWeb/web/about/stats/index.
htm (2007)
12. Chang, A., Matyas, M. et al.: I came, I found it, I used it, and it
made a difference. Washington, D.C., American Society for Micro-
biology: 8 (2004)
13. Chism, N.: Using a framework to engage faculty in instructional
technologies. Educause Q. 27(2), 39–45 (2004)
14. Cohen, J.: A power primer. Psychol. Bull. 112(1), 155–159 (1992)
15. Davis, F.: Perceived usefulness, perceived ease of use, and user
acceptance of information technology. MIS Q. 13(3), 319–340
(1989)
16. Duval, E.: The technology of open education. Presentation at Open
Education 2006: Community, Culture, and Content, Logan Utah
(2006)
123
F. McMartin et al.
17. Gibbs, E.J., Major, C.H. et al.: Faculty perception of the costs
and benefits of instructional technology: implications for faculty
work. J. Faculty Dev. 19(2), 77–88 (2004)
18. Giersch, S., Klotz, E.A., et al.: If you build it, will they come? Par-
ticipant involvement in digital libraries. D-Lib Magazine 10(7/8)
(2004)
19. Gilbert, S.: The Beauty of Low Threshold Applications. Syllabus
Magazine. http://campustechnology.com/articles/38981/ (2002)
20. Glaser, B., Strauss, A.: Discovery of Grounded Theory: Strategies
for Qualitative Research. Aldine, Chicago, IL (1967)
21. Green, D.: Using digital images in teaching and learning: perspec-
tives from liberal arts institutions. http://www.academiccommons.
org/imagereport (2006)
22. Hagner, P.: Faculty engagement and support in the new learning
environment. Educause Rev. 35(5), 27–37 (2000)
23. Harley, D.: The Use of Digital Resources in Humanities and Social
Science Undergraduate Education. Center for Studies in Higher
Education, Berkeley, CA (2004)
24. Harley, D.: Why understanding the use and users of OERs mat-
ters. In: Iiuoshi, T., Kumar, V. (eds.) Opening Up Education: The
Collective Advancement of Education Through Open Technology,
Open Content and Open Knowledge. MIT Press, Cambridge (2008,
Forthcoming)
25. IES National Center for Education Statistics: Digest of Edu-
cation Statistics. http://nces.ed.gov/programs/digest/d06/tables/
dt06_237.asp (2003)
26. Khoo, M., Donahue, R.: Evaluating digital libraries with Webme-
trics. In: Proceedings from the ACM IEEE Joint Conference on
Digital Libraries, Vancouver, BC (2007)
27. Koppi, T., Bogle, L. et al.: Institutional use of learning objects:
lessons learned and future directions. J Educ Multimed Hyperme-
dia 13(4), 449–463 (2004)
28. Lean, J., Moizer, J. et al.: Simulations and games: use and bar-
riers in higher education. Active Learn Higher Educ. 7(3), 227–242
(2006)
29. Lerman, S., Miyagawa, S., et al.: OpenCourseWare: Building a
culture of sharing. In: Iiuoshi, T., Kumar, V. (eds.) Opening Up
Education: The Collective Advancement of Education Through
Open Technology, Open Content and Open Knowledge. MIT Press,
Cambridge (2008, forthcoming)
30. Lossau, N.: Search engine technology and digital libraries: libra-
ries need to discover the academic internet. D-Lib Magazine 10(6)
(2004)
31. Lynch, C., Garcia-Molina, H.: Interoperability, scaling, and the
digital libraries research agenda. http://www.hpcc.gov/reports/
reports-nco-iita-dlw/main.html (1995)
32. Macdonald, R.H., Manduca, C.A. et al.: Teaching methods in
undergraduate geoscience courses: results of the 2004 On the
Cutting Edge survey of US faculty. J. Geosci. Educ. 53(3), 237
(2005)
33. Manduca, C.A., Iverson, E.R., et al.: Influencing user behavior
through digital library design: an example from the geosciences.
D-Lib 11(5) (2005)
34. Manduca, C.A., Fox, S., et al.: Digital library as network and com-
munity center. D-Lib 12(12) (2006)
35. Marchioni, G.: Exploratory search: from finding to understan-
ding. Communications 49(4), 41–46 (2006)
36. Mason, B.: Omniture and other evaluation tools for Com-
PADRE. NSDL 2006 Annual PI Meeting, Washington DC (2006)
37. Matkin, G.: Learning Object Repositories: Problems and Pro-
mise. The William and Flora Hewlett Foundation Menlo
Park, CA (2002)
38. McMartin, F., Iverson, E., et al.: Factors motivating use of digital
libraries. Proceedings of the 6th ACM/IEEE-CS joint conference
on Digital libraries, ACM Press, Tuscon (2006)
39. NSDL: About NSDL. http://nsdl.org/about/ (2007)
Online digital resources and educational digital libraries
40. Russell, D.M.: What are they thinking? Searching for the mind of
the searcher. Joint Conference on Digital Libraries, Vancouver, BC
(2007)
41. Smith, M., Barton, M., Bass, M., Branschofsky, M., McClellan,
G., Stuve, D., Tansley, R., Walker, J.H.: DSPACE: an open source
dynamic digital repository. D-Lib Magazine 9(1) (2003)
42. Spotts, T., Bowman, M.: Faculty use of instructional technologies
in higher education. Educ. Technol. 35(2), 56–64 (1995)
43. Teevan, J., Alvarado, C., et al.: The Perfect search engine is not
enough: a study of orienteering behavior in directed search. In:
Proceedings, SIGCHI Conf. Human Factors in Computing Sys-
tems, Vienna, Austria, ACM Press, Tuscon (2004)
44. Thomas, A., Rothery, A.: Online repositories for learning materials.
Ariadne 45 (2005)
45. Thong, J.Y.L., Hong, W.Y. et al.: Understanding user acceptance
of digital libraries: what are the roles of interface characteristics,
organizational context, and individual differences?. International
J. Hum.-Comput. Stud. 57(3), 215–242 (2002)
46. Uijtdehaage, S.H.J., Contini, J. et al.: Sharing digital teaching
resources: breaking down barriers by addressing the concerns of
the faculty members. Acad. Med. 78(3), 286–294 (2003)
47. Van House, N.A., Butler, M.H., et al.: User-centered iterative
design for digital libraries: the cypress experience. D-Lib Magazine
2(2) (1996)
48. Wolcott, L., Betts, K.: What’s in it for me? incentives for faculty
participation in distance education. J. Dist. Educ. 14(2), 34–49
(1999)
123