HHS Public Access

Author manuscript
J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
Author Manuscript

Published in final edited form as:

J Coll Sci Teach. 2016 March ; 45(4): 52–58.

Undergraduate journal club as an intervention to improve

student development in applying the scientific process
Conner I Sandefur1 and Claire Gordy2
1Departmentof Biology, University of North Carolina at Pembroke, 2230 Oxendine Building, PO
BOX 1510, Pembroke, NC 28372, USA
2Departmentof Biochemistry and Biophysics, University of North Carolina at Chapel Hill, 120
Mason Farm Road, 3rd Floor Genetic Medicine Building, Campus Box 7260, Chapel Hill, NC
Author Manuscript

27599, USA

Abstract
Active learning improves student performance in STEM courses. Exposure to active learning
environments generally occurs through traditional laboratory courses and independent research,
both of which require access to resources that are limited at many universities. A previously
reported active learning-based undergraduate journal club improved student achievement in
communicating science. Here, we expanded on this previous journal club to improve student
performance in the process of science. We developed and implemented a series of workshops and
seminars referred to as “CASL Club,” an undergraduate journal club targeted at improving student
development in applying the scientific process. Students were surveyed before and after CASL
Author Manuscript

club about their confidence in accessing, analyzing, and reporting scientific research. Post-CASL
club, the students reported increases in confidence in their abilities to access and present scientific
articles and write scientific abstracts. Additionally, the students reported improved confidence and
performance in their courses. Compared to the previous journal club study, the majority of
sampled journal club participants were not exposed to primary literature as part of their general
coursework. Our results illustrate active-learning based undergraduate journal clubs as a way to
expose students to primary literature and improve students’ ability to apply scientific process in an
active-learning environment at resource-limited universities.

Introduction
Recently, the National Research Council’s Discipline-Based Education Research (2012) and
Author Manuscript

Bio 2010 (2003) reports, the Vision and Change in Undergraduate Biology Education report
(2011), and others have called for transformative biology education. These reports detail
core concepts and competencies deemed necessary for students to be successful in the
increasingly complex and collaborative arena of biological sciences. Developing and
implementing transformative science education requires resources to train and support
faculty as they learn and practice new evidence-based teaching strategies. Although these
reports stress the necessity of implementing transformative biology education practices
throughout the US higher education system, the resources available to support faculty and
staff vary widely among universities. As a result, resource-limited institutions may lag
behind in implementing new biological pedagogies when compared to more well-resourced
 Sandefur and Gordy Page 2

institutions. A lag in implementation may hinder some students’ abilities to compete and
Author Manuscript

succeed alongside their peers in what the 2009 report from the National Academies Press
calls the “New Biology.”

Addressing differences in the implementation of new biological pedagogies is critically

important in preparing students from diverse backgrounds and resource-limited schools to
succeed in graduate school and careers alongside peers from majority backgrounds and/or
more well-resourced institutions. Numerous studies have established that student
performance in STEM fields is improved when active learning methodologies are
implemented (Freeman et al., 2014). Furthermore, active learning in large classrooms is
particularly beneficial to black and first-generation college student achievement (Eddy &
Hogan, 2014), suggesting that implementation of active learning-based methodologies is
exceptionally important at diverse, resource-limited institutions.
Author Manuscript

Traditionally, active learning in the sciences occurs in laboratory courses and through
independent research. Although active learning is increasingly incorporated into the
classroom, resource-limited institutions may lack the laboratory equipment, reagents,
technology, or faculty training to successfully implement active learning in classroom or
laboratory settings. We thus focused on developing active learning-based methods to address
multiple core competencies at a resource-limited institution with a highly diverse student
population. In designing our intervention, we noted that student development in several of
the core competencies identified in Vision and Change can be addressed through critical
analysis of primary scientific literature (Hoskins, Stevens, & Nehm, 2007; Rybarczyk,
2006). These competencies include the application of the process of science, communication
and collaboration, and understanding the relationship between science and society.
Undergraduate journal clubs implemented with active learning methodologies provide an
Author Manuscript

attractive mechanism to develop student skills at resource-limited institutions, as they

require few resources other than instructor time.

A previous study reported that participation in an active learning-based undergraduate

journal club improved student performance in research paper presentations (science
communication), particularly in lower achieving students (Robertson, 2012). Because the
previously described undergraduate journal club (9) was implemented at a private, well-
resourced university, we were interested in how an active learning journal club could
improve student performance in applying the scientific process at a public, diverse, and
resource-limited university.

To assess the impact of participating in an undergraduate journal club on student

performance in the core competency of applying the process of science, we modified and
Author Manuscript

expanded upon this previous journal club and investigated its impact on student confidence
in locating, critically analyzing, and presenting primary literature through pre- and post-
surveys. A series of targeted workshops and seminars were packaged into a semester-long,
non-credit, informal course called the Critical Analysis of Scientific Literature Club, or
CASL (pronounced “castle”) Club. The goal of CASL Club was to meet students where they
were and to bring them up to a level of competence in applying the scientific process
equivalent to that of their peers at more well-resourced institutions by providing

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 3

opportunities to engage with primary literature and practice the skills necessary to read and
Author Manuscript

analyze primary literature. The students were introduced to the concept of a journal club and
where they might encounter one, used tools to access and organize primary literature, and
practiced communicating critiques of scientific articles in a traditional journal club format.
Through student surveys, we found an increase in self-reported confidence in the students’
abilities to apply the scientific process and communicate science. Our results suggest that
implementing informal clubs similar to CASL Club may provide a way to improve student
development in core competencies in resource-limited environments.

CASL Club structure

CASL Club expanded upon a previously published active learning-based graded mini
workshop (Robertson, 2012). The primary learning objectives of our club were for students
to (1) develop and implement a method to read and analyze scientific literature, (2) design
Author Manuscript

and present an oral critique of a scientific paper, and (3) use electronic resources to locate
and organize scientific articles. Additionally, we hoped to improve overall student
confidence in achieving these objectives. CASL Club sessions were designed to achieve
these goals (Fig. 1).

The hour-long, non-credit bearing, non-graded CASL Club met one evening a week for 8
weeks during a standard semester. The students (n=12) were undergraduate natural and
physical science majors at a rural, public, Master’s-level university. Prior to beginning the
CASL Club, a link to an informal online survey of student background interests was
circulated via email among a group of potential student participants. Pre- and post-surveys
assessing student experience and confidence were given after consent for study participation
was obtained in writing (university IRB waived as exempt).
Author Manuscript

The structure of CASL Club was based on an expanded version of the format of the
published mini workshop and began with the collaborative development of an article
analysis worksheet based on the scientific method (Robertson, 2012). The creation of the
Critical Analysis of Scientific Literature (CASL) worksheet began with a discussion of the
standard process of science (‘scientific method’). For each component of the scientific
method (e.g. formulate a research question), the students were asked to identify which
sections of a scientific paper were most likely to contain this information (e.g. abstract, title,
introduction, conclusion). The students were also given prompts to help to help them find
key components of the scientific method that are often not directly stated in a scientific
paper. For example, in the case of a paper that does not directly state a research question, the
students were encouraged to rewrite the paper’s title as a question. Worksheet development
occurred over two class periods; student discussion was facilitated by two faculty members,
Author Manuscript

and the students were encouraged to write their comments on the white board. White board
notes were captured via photography and provided to the participants after each class period.
These worksheet development notes were aggregated by the faculty instructors into a CASL
Worksheet, which was emailed to the students. The CASL worksheet served as a guide to
read and analyze journal articles in later meetings (described below).

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 4

Near the end of the second session, the instructors introduced a demonstration paper and
Author Manuscript

provided it to the students. The students were asked to read and analyze the paper using the
CASL Worksheet. During the third session, the students and instructors analyzed the
demonstration paper as a group by discussing the students’ worksheet answers.

The fourth session was a tutorial on how to present scientific articles in a journal club
setting.. During this session, the concept of a journal club was introduced, and examples of
where students might encounter journal clubs were provided. The tutorial then demonstrated
how to present a scientific article using an assertion/evidence style (Alley, 2013) using the
demonstration paper as an example. This tutorial was presented using Microsoft Power
Point, and the slide deck was provided to the students via email after class.

During this fourth session, the instructors also provided abstracts for three instructor-
selected papers (Debruyn & Sayler, 2009; Hays & McBee, 2007; Yoo & Janz, 2003). These
Author Manuscript

papers, along with the demonstration paper, were all related to a semester-long theme:
Environmental and Health Impacts of the Tar Creek Superfund Site. This theme was selected
to appeal to student educational interests. Additionally, we selected Tar Creek because the
community near the Superfund site in rural Oklahoma had similarities to the rural
community around our university. The demonstration paper provided an overview of the Tar
Creek disaster and used straightforward experimental techniques to examine the teratogenic
effects of mining-related water contamination (Dawson, McCormick, & Bantle, 1985). Each
of the three student-presented papers focused on environmental, chemical, or medical effects
of the contamination of Tar Creek. The students self-formed groups based on their interests
and were assigned a presentation date. Once the students were grouped, they were charged
with meeting outside of class to prepare their presentations, and the instructors performed
check-ins at the end of the remaining weekly meetings to answer questions and encourage
Author Manuscript

accountability.

The fifth session included an instructor-led presentation on “Keeping up with the literature,”
which was comprised of three sections: “keeping up to date,” “keeping track,” and “keeping
it real.” The first section provided information on finding relevant journal articles and
keeping up to date with new articles. We also discussed journal alerts, search alerts, RSS
feeds, citation tracking, and tools such as Google Scholar Citations and Feedly. The second
section discussed tools to store a bibliography, focusing on the free citation software
(RefWorks) available to students at this particular university. We concluded this session with
a discussion of how to use a bibliography and demonstrated how to cite a journal article
inside a Microsoft Word document using a synced reference database.

The three remaining sessions were group presentations following a traditional journal club
Author Manuscript

format. Each of the group presentations ended with an informal evaluation and critique, as
well as discussion of any remaining unclear concepts encountered in the paper. At the final
group session, we presented the students with certificates of completion and reminded the
students to add the CASL Club experience to their resumes and/or CVs. Some general tips
for implementing a CASL Club based on our experiences are provided (Fig. 2).

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 5

Students self-identified as scientists

Author Manuscript

Twelve students completed the pre-survey, and nine completed the post-survey. Students’
scientific background was heterogeneous, and student interest fell into two general areas:
environmental and biomedical sciences. This information was used to select a theme for the
semester: Environmental and Health Impacts of the Tar Creek Superfund Site. Before
providing the students with the demonstration paper at the end of the second session, the
students were shown a trailer for a documentary concerning the Tar Creek disaster to
introduce them to the environmental and health issues related to the contamination of Tar
Creek (Myers, Beer, Beer, Beer, & Billings, 2009).

In both pre- and post-surveys, all of the students “strongly agreed” that they were interested
in pursuing research-based graduate studies. Eleven of twelve students who completed the
pre-survey had previously performed research either in a laboratory setting or in the field.
Author Manuscript

Additionally, eleven of twelve students self-identified as a scientist, and ten of twelve

students self-identified as a researcher (Fig. 3A). In accordance with this sense of scientific
identity, the majority of the students expressed a high degree of confidence in general skills
(“I can usually understand the main points of a scientific journal article” and “I am confident
in my ability to find a scientific journal article online”, mean response of “agree” for each
statement) (Fig. 3B). Despite their previous research experience and high degree of
confidence in general skills, seven of twelve indicated that they had never presented a
critique of a scientific journal article, and only two of twelve agreed or strongly agreed with
the statement “I regularly read scientific journal articles” (Fig. 3C).

Lower confidence reported for communicating science skills

In contrast, the students’ responses to more specific statements about accessing and
Author Manuscript

presenting scientific literature indicated a lower degree of confidence. For example, although
students expressed confidence in their ability to find scientific journal articles online when
presented with a general statement, the mean response to the statement “I know which
electronic databases to use to locate scientific journal articles” was disagree. Similarly, the
mean responses to the statements “I know how to write a good abstract for a scientific
journal article” and “I can create a good PowerPoint presentation of a scientific journal
article” were disagree (Fig. 4).

After completion of the CASL Club, the students self-reported increases in their ability to
apply the process of science and to communicate and collaborate. In particular, the student
responses to the three questions related to specific skills increased from means of disagree to
means between agree and strongly agree (Fig. 4). The students self-reported significant
Author Manuscript

improvement in their ability to access literature using electronic databases (p < 0.05, Mann-
Whitney test) and to present a journal club article (p < 0.005), suggesting that the targeted
instructor-led seminars on “Keeping up with the Literature” and “Ins and Outs of a Journal
Club” were effective. Moreover, the students self-reported strong agreement (4.9) that the
journal club improved their ability to orally discuss the results of scientific experiments
(Table 1). Additionally, student self-reported ability to write a good abstract for a scientific
article increased significantly (p < 0.005).

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 6

Improved confidence and course performance reported

Author Manuscript

Importantly, the survey results indicate that the journal club experience improved student
confidence and performance in other courses. After completion of the CASL Club, the
students reported strong agreement (4.9) that CASL Club improved their confidence in
reading scientific articles in other classes. Additionally, the students self-reported agreement
(4.2) with the statement that CASL Club improved their performance in other classes. The
positive effect of CASL Club on student performance in other courses suggests that journal
club participation improved the students’ scientific core competencies. It is also possible that
the informal journal club promoted the formation of a peer group. If this was the case, the
journal club could be viewed as a type of learning community. Although we did not set out
to investigate this specifically, learning communities have been shown to promote student
performance and persistence (Graham, Frederick, Byars-Winston, Hunter, & Handelsman,
2013). Therefore, we are interested in investigating the use of journal clubs as learning
Author Manuscript

communities in the future.

Impact on our student population

In the previous study describing the results of a mini-journal club workshop, the sample
student population was from a private liberal arts university (Robertson, 2012). The majority
of the students sampled had taken more than three courses that included exposure to primary
scientific literature (Fig. 5). At our institution, a public, comprehensive, historically
American Indian university, within a group of high achieving students who plan to attend
medical or graduate school, the majority had not taken any classes that included reading
and/or discussion of primary scientific literature (Fig. 5). This stark contrast is evidence of a
difference in educational experiences between students from these two institutions.
Author Manuscript

Despite this difference in educational experiences, most of the students in our sample self-
identified as scientists and had a high degree of confidence in general skills such as
searching and analyzing scientific literature. Analysis of the post-survey data showed no
change in the measures on which students scored themselves highly on the pre-survey. In
contrast, the mean response to the statement “I regularly read scientific journal articles”
increased significantly (Fig. 3C). An additional set of questions included on the post-survey
indicated that students agreed or strongly agreed that the journal club increased their skills,
interest, and confidence in science (Table 1). After completion of CASL Club, the students’
feelings of scientific independence increased as indicated by a trend toward an increase in
the mean response to the statement “I can design my own experiment to test a hypothesis”
and a significant increase in the mean response to the statement “With the skills I have now,
I can discover something new that no one else knows” (Fig. 6). Taken as a whole, our results
Author Manuscript

provide evidence for active learning journal clubs as a mechanism to improve student
development in applying the process of science in resource-limited environments.

References
Alley, M. Learning the assertion-evidence approach. 2013. Retrieved from http://
www.writing.engr.psu.edu/assertion_evidence.html

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 7

BIO2010: Transforming undergraduate education for future research biologists. The National
Academies Press; 2003.
Author Manuscript

Dawson DA, McCormick CA, Bantle JA. Detection of teratogenic substances in acidic mine water
samples using the frog embryo teratogenesis assay--xenopus (FETAX). Journal of Applied
Toxicology: JAT. 1985; 5(4):234–244. [PubMed: 4045096]
Debruyn JM, Sayler GS. Microbial community structure and biodegradation activity of particle-
associated bacteria in a coal tar contaminated creek. Environmental Science & Technology. 2009;
43(9):3047–3053. [PubMed: 19534112]
Discipline-based education research: Understanding and improving learning in undergraduate science
and engineering. The National Academies Press; 2012.
Eddy SL, Hogan KA. Getting under the hood: How and for whom does increasing course structure
work? CBE-Life Sciences Education. 2014; 13(3):453–468. DOI: 10.1187/cbe.14-03-0050
[PubMed: 25185229]
Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. Active
learning increases student performance in science, engineering, and mathematics. Proceedings of
the National Academy of Sciences. 2014; 111(23):8410–8415. DOI: 10.1073/pnas.1319030111
Author Manuscript

Graham MJ, Frederick J, Byars-Winston A, Hunter A, Handelsman J. Increasing persistence of college

students in STEM. Science. 2013; 341(6153):1455–1456. DOI: 10.1126/science.1240487 [PubMed:
24072909]
Hays KA, McBee K. Flow cytometric analysis of red-eared slider turtles (trachemys scripta) from tar
creek superfund site. Ecotoxicology (London, England). 2007; 16(4):353–361. DOI: 10.1007/
s10646-007-0135-3
Hoskins SG, Stevens LM, Nehm RH. Selective use of the primary literature transforms the classroom
into a virtual laboratory. Genetics. 2007; 176(3):1381–1389. gen17631381 [pii]. [PubMed:
17483426]
Myers, M.; Beer, T.; Beer, R.; Beer, C.; Billings, R. Tar creek trailer. 2009. Retrieved from http://
tarcreekfilm.com/trailer/
A new biology for the 21st century: Ensuring the united states leads the coming biology revolution.
Washington, DC: National Academies Press; 2009.
Robertson K. A journal club workshop that teaches undergraduates a systematic method for reading,
Author Manuscript

interpreting, and presenting primary literature. Journal of College Science Teaching. 2012; 41(6):
25.
Rybarczyk, B. Incorporating primary literature into science teaching. In: Mintzes, JJ.; Leonard, WH.,
editors. Handbook of college science teaching. NSTA Press; 2006. p. 159
Vision and change in undergraduate biology education: A call to action. American Association for the
Advancement of Science; 2011.
Yoo JL, Janz DM. Tissue-specific HSP70 levels and reproductive physiological responses in fishes
inhabiting a metal-contaminated creek. Archives of Environmental Contamination and Toxicology.
2003; 45(1):110–120. DOI: 10.1007/s00244-002-0109-7 [PubMed: 12948180]
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 8
Author Manuscript
Author Manuscript

Figure 1. Specific tools and skills were introduced during each session to help students achieve
the three objectives of CASL Club
Workshop sessions were designed around the three objectives. Objective-specific tools were
introduced and skills were practiced over the eight hour-long sessions.
Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 9
Author Manuscript

Figure 2.
Author Manuscript

General tips for implementing a CASL Club.

Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 10
Author Manuscript

Figure 3. CASL Club students identified as scientists with self-reported confidence in general
skills related to searching and analyzing scientific literature, which contrasted with the small
number of students self-reporting regular reading of journal articles
Student responses to the statements (A) “I am a scientist,” “I am a researcher,” (B) “I can
Author Manuscript

usually understand the main points of a scientific journal article” and “I am confident in my
ability to find a scientific journal article online” on a pre-survey administered prior to
beginning the CASL Club and (C) “I regularly read scientific journal articles” on a pre-
survey administered prior to beginning the CASL Club and a post-survey administered at the
end of the course were measured on a Likert scale ranging from 1 (strongly disagree) to 5
(strongly agree). Horizontal bars indicate means, and error bars indicate 95% confidence
intervals.
Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 11
Author Manuscript

Figure 4. Comparison of pre- and post-surveys shows increases in self-reported confidence in

Author Manuscript

specific skills related to searching, analyzing, and discussing scientific literature

Student responses to the statements “I know which electronic databases to use to locate
scientific journal articles” (circles), “I know how to write a good abstract for a scientific
journal article” (squares), and “I can create a good PowerPoint presentation of a scientific
journal article” (triangles) on a pre-survey administered prior to beginning the CASL Club
(open shapes) and a post-survey administered at the end of the course (closed shapes) were
measured on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
Horizontal bars indicate means, and error bars indicate 95% confidence intervals. *, p<0.05;
**, p<0.005 (Mann-Whitney test).
Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 12
Author Manuscript
Author Manuscript

Figure 5. Variation in exposure to reading and/or discussion of scientific literature between two
sampled student populations from two universities
The previous study sampled 33 students prior to beginning the journal club (Robertson,
2012). Our pre-course survey sample included 12 students.
Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Sandefur and Gordy Page 13
Author Manuscript
Author Manuscript

Figure 6. Comparison of pre- and post-surveys shows increases in self-reported feelings of

scientific independence
Student responses to the statements “With the skills I have now, I can discover something
new that no one else knows” (circles) and “I can design my own experiment to test a
hypothesis” (squares) on a pre-survey administered prior to beginning the CASL Club (open
shapes) and a post-survey administered at the end of the course (closed shapes) were
measured on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
Horizontal bars indicate means, and error bars indicate 95% confidence intervals. *, p<0.05
(Mann-Whitney test).
Author Manuscript
Author Manuscript

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
 Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Table 1

Responses to post-CASL club questions specific to the journal club experience.

Neither agree nor

Statement Strongly disagree Disagree disagree Agree Strongly agree Mean SD
This journal club improved my ability to design an experiment. 0 0 1 4 4 4.3 0.7
Sandefur and Gordy

This journal club improved my ability to interpret the results of experiments in scientific papers. 0 0 0 2 7 4.8 0.4

This journal improved my ability to discuss the results of scientific experiments orally. 0 0 0 1 8 4.9 0.3

This journal club increased my interest in science. 0 1 0 1 7 4.6 1.0

This journal club increased my interest in performing scientific research in my career. 0 1 0 0 8 4.7 1.0

This journal club increased my confidence in reading scientific articles in other classes. 0 0 0 1 8 4.9 3.0

The skills I have learned in this journal club have improved my performance in other courses. 0 0 2 3 4 4.2 0.8

After the journal club, I am interested in a science field that I wasn’t previously interested in. 4 1 2 1 1 2.3 1.5

J Coll Sci Teach. Author manuscript; available in PMC 2016 May 20.
Page 14