Professional Documents
Culture Documents
Initial Findings
Vanessa Beckham, Nathan Foster, Jasmine Ivy, Carol McFarland McKee, & Alexis Torres
As practitioners in the field of student affairs, our focus must remain on student success.
Academic support services play a critical role in supporting students’ individual needs and
fostering student success. At Northern Illinois University (NIU), the Supplemental Instruction
(SI) program has been fundamental in helping students achieve academically for decades, and it
is essential that we continue to evaluate and improve this program to meet the goals of our
students and the university. Research shows that consistent attendance in Supplemental
Instruction (SI) is directly correlated to increased student persistence and graduation rates (Malm
et al., 2018). As the university is currently undergoing a restructuring initiative to bring all
academic support services together, it is vital that we collect and use data to identify program
strengths and weaknesses to bring about positive change while maintaining program successes.
This assessment sought to answer three basic questions about SI – What are students’
perceived impacts of participating in SI? Why are some students not taking advantage of SI
academic support services? What impact does SI programming have on students’ perceptions of
NIU as a Culturally Engaging Campus Environment (CECE)? Although some insight was gained
program and staff, additional research is needed to better understand why some students do not
participate in SI services.
The team approached this assessment through the lens of critical theorists, with strong
and that of most research is transformation, both in the students and the services, to meet the
needs of students and empower them to become more capable of controlling their destiny.
Through our assessment, we have the power to transform academic support at NIU by imagining
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
and helping to create alternatives, and our work will be validated if actions are created as a
result. Our mutual paradigm leads us to prefer qualitative research that is inclusive to ensure that
all voices are heard and reflected in our data and analysis. Bridging the gap in students’ learning
needs is important to ensure that we are hitting prime areas because students come from various
students’ learning curves. With this assessment we sought to provide answers to questions that
current staff pose when trying to examine student learning. We also sought to identify and share
We recognize and embrace the influence of Critical Theory and the Culturally Engaging
Campus Model on all aspects of our assessment – including our approach to the review of the
Literature Review
math, instead of a prerequisite course – a required course given to students to take prior to a
specific course – corequisite courses have been indicated to significantly improve outcomes
because of the learning assistance that happens alongside courses as students need help. Koselak
(2017) explains corequisite courses could work with embedded tutoring centers that focus less on
labeling students and pigeonholing them into tiers and more on providing just-in-time support.
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Additionally, the staff should be able to help sort out whether a student will not do the work or
cannot do it, resulting in a will-versus-skill dilemma and targeting the appropriate support
(Koselak, 2017). Both Koselak (2017) and Barhoum (2018) suggest staffing peer mentors to
successful student traits (Barhoum, 2018). Barhoum and Koselak’s suggestion of having an
assistance program that is free to all and required by some normalizes seeing peers helping peers
and, as stated by Barhoum, “...can be an important component of helping students feel that they
are not alone, and that college is a place they belong and where they can flourish.” (Barhoum,
2018, p. 23).
The research indicates that students are positively impacted academically, emotionally,
and socially from academic support programs that perform at a high-level. Consistent attendance
graduation rates (Malm et al., 2018). A study by Osborne, Parlier, and Adams (2019) followed a
coaching. The results show that students involved in any of the programs believed that the
assistance impacted them academically, and there was also a direct correlation between frequent
attendance in these programs and higher levels of the student’s perception of how these programs
for learning assistance programs. One of these is the International Center for Supplemental
Instruction (SI), housed at the University of Missouri-Kansas City, which offers an accreditation
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
specifically for SI programs. The accreditation process identifies four areas that must be met,
which are referred to as the Core Four. They include a) training and certification of the program
coordinator by the International Center for SI; b) initial and ongoing intensive training for SI
Methods Overview
who were enrolled in one or more of multiple sessions of seven SI-supported classes during Fall
2019 and Spring 2020 semesters. A survey instrument and focus group questions were developed
Norton and Agee (2014) offer three basic categories of assessment processes for learning
assistance programs: quantitative, qualitative, and criterion referenced. In the course of the
the International Center for Supplemental Instruction. However, since the SI program at NIU has
not sought this accreditation, this evaluation is limited to and focused on the first two categories,
quantitative, and qualitative. Quantitative assessments, such as comparison of pre- and post-test
scores for student participants, or comparison of learning outcomes or grades for student
participants compared to non-participants (the control group) may be feasible and useful for
assessing program effectiveness, but present challenges of time, preparation, and access to
participation credit, or attendance points (Norton & Agee, 2014). They further indicate an
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
“ongoing need for the kinds of qualitative assessment processes that allow learning assistance
services to gauge student satisfaction and to tell students’ compelling stories of frustration or
success” and “mixed-methods approaches that combine quantitative and qualitative assessments
process are often favored” (Norton & Agee, 2014, p. 7). Effective qualitative assessment
processes include surveys, in-person interviews, and focus groups (Norton & Agee, 2014; Schuh,
et al., 2016).
Our primary goals of this assessment were twofold: a) to gain a better understanding of
students’ perceptions of their past experiences using SI services; and b) to learn how program
administrators might better reach students who are unaware of the SI program or may be aware
but choose not to use SI. Following these two primary goals of the assessment, data were
gathered through a survey and voluntary focus group of past users for two semesters (Fall 2019
and Spring 2020), and an additional voluntary focus group consisting of non-users for the same
two semesters. In keeping with the Culturally Engaging Campus Environment (CECE)
theoretical framework, data collected and analyzed from the survey and the two focus groups
culturally validating and humanized environment, a proactive philosophy, and holistic support.
Data Collection
Our goal was to assess SI based on the framework of the CECE model, but it was also
valuable to build on data that already gathered by the SI program. Survey instruments used by
the SI program to conduct past surveys were provided, however, historical results from these past
surveys were not compiled and readily available. Using the CECE model to conduct the current
assessment, however, required gathering information that had not been asked in previous surveys
of student participants. Past surveys did not include questions that pertain to a culturally
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
included such questions in both the survey and in focus group questions (see Appendices B &
C).
SI sessions are typically offered for introductory 100-level courses which may satisfy
general education requirements and must be completed to advance in any program of study. For
the Fall 2019 and Spring 2020 semesters, supplemental instruction (SI) sessions were offered for
multiple sections of seven courses at NIU: three sections of Introductory Financial Accounting
(ACCY 206); five sections of Chemistry (CHEM 110); six sections of Communications Skills –
Psychology (PSYC 102); and three sections of Basic Statistics (STAT 100).
Our data collection throughout this evaluation was to follow the two objectives of
understanding a) the perception of students who have already used SI, and b) how to encourage
students who have never worked with the program to use their services. Analysis through the
lens of the CECE model provides insight into student perceptions of SI’s effectiveness as a
culturally responsive support system focused on ensuring their students of all backgrounds
succeed.
A total of 2,062 individual students were eligible to participate in SI for the various
sections of the seven courses. Participation in SI was not required but was strongly
recommended. More than a third (749 students, 36.32%) of all 2,062 students took advantage of
the opportunity to participate in SI sessions. This included 675 students who participated in SI
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
sessions during the Fall 2019 semester and 74 students who first participated in SI sessions in
Spring 2020. Many students who had participated in SI sessions both semesters, such as MATH
108 in Fall 2010 and MATH 109 in Spring 2020. The exact number of students who participated
in SI both semesters could not be determined from the data set provided. Additionally, some
students participated in SI sessions for more than one course during each of the semesters, such
as PSYC 102 and ACCY 206 in the same semester, or LTRE 100 and MATH 108 the same
semester. Again, the number of students who participated in SI sessions for more than one class
in a single semester was not determinable from the data set provided. However, the sample of
675 students was developed by excluding duplication to provide the total number of unique
individual students who actively participated in SI over the course of the two semesters.
No information was provided about demographic characteristics of the students in the data
set.
Survey Instrument
The survey instrument was developed by the team, in consultation with the client and
course instructor for Assessment Methods in Higher Education, and incorporated elements of
past surveys conducted of SI students by the client, a previous survey of alumni of a cultural
resource center by one of the team members, and questions designed to elicit information from
respondents regarding culturally engaging campus environments (CECE model) aspects of the SI
program. The survey consisted of a total of 40 questions and 9 skip-logic questions (see
Survey Distribution
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
The SI Assessment Survey was sent via email link to all 749 students who had
participated in SI sessions during Fall 2019 and Spring 2020 semesters (see Appendix B). To
protect private student information and ensure compliance with FERPA regulations, the survey
was distributed on behalf of the assessment team by NIU’s Director of Career Services Planning
and Assessment. The initial survey invitation was sent on November 5, 2020, and a reminder
email sent on November 8, 2020. An additional reminder for the survey was sent on behalf of the
Survey results reported were collected on November 21, 2020; however, the survey
remains open and additional reminders will be sent on behalf of the SI Supervisor to gather
additional responses.
Using Qualtrics to create and distribute the survey for SI participants yielded quantitative
data in a report that was compiled and organized in seven pre-selected sections or themes:
Introduction and Consent; Familiarity and Understanding of SI; Expectations and Perceptions of
Demographic Information; and Survey Closure and Invitation to Participate in Focus Group
Additionally, open-ended questions on the survey provided more depth and context for responses
The second part of our data analysis consisted of reviewing and analyzing interview
transcripts and coding for themes. Assessment team members who conducted interviews also
transcribed the interviews, and all members reviewed and analyzed the transcripts for emerging
themes. Munoz and Espina (2017) used a similar method for their article using the CECE model
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
as the theoretical framework. After the initial reading and themes were completed, they wrote,
“We then developed a coding scheme based on the inductive codes and the literature review,
including our interpretations of the CECE model, our understanding of culturally relevant
pedagogy, and the school as sanctuary concept” (Munoz & Espina, p. 540, 2017). For our
assessment purposes, the “school as sanctuary concept” that Munoz and Espina describe were
replaced with SI. Finally, as an entire group, themes were modified through rereading and
Although we did not conduct research and did not require IRB approval, we still planned
to utilize the three basic criteria for ethical research as outlined in The Belmont Report: respect
for persons, beneficence, and justice (Schuh et al., 2016). To ensure we met these criteria we
included an informed consent document that outlined the purpose of the assessment, description
of the focus group procedures, potential risk, and how the data would be collected, processed,
and managed. To guarantee participants confidentially we solely worked with a gatekeeper in the
SI program. All forms of outreach with participants in this study were communicated through a
gatekeeper designated by the Program Director. This gatekeeper distributed the anonymous
survey to students who had previously participated in SI in one of two semesters (Fall 2019 and
Spring 2020). Additionally, an invitation to participate in a focus group was emailed to non-users
during those same two semesters by the gatekeeper. Based on users volunteering in the survey to
participate in a focus group, students were invited to join the “users focus group” by a team
member. Participant anonymity and confidentiality were maintained by allowing only the
gatekeeper to handle identifying information of the students until they volunteered to participate
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
in a focus group. At the point of opting into a focus group, participants provided their contact
Since our assessment is situational and context specific, we did not focus on reliability,
and instead shifted our attention on trustworthiness and credibility to ensure rigor (Henning &
Roberts, 2016). We used the assessment of the impact of the program to show that the findings
of this assessment are worth paying attention to. Using focus groups allowed us to increase rigor
and credibility by collecting qualitative data that included students’ firsthand accounts of their
experiences using the services or why they do not use the services. This allowed the using
participants to create and approve of the data that is collected through the survey, and the non-
using participants a chance for their voices to be heard. The assessment’s focus is on a program
at NIU so the transferability, dependability, and confirmability applied to this specific program,
and provided insight into participants’ perceptions of the degree to which SI services contribute
Theoretical Framework
As noted earlier, our team approaches this assessment through the lens of critical
assessment is that of transformation, primarily of the services, to meet the needs of students and
empower them to become more capable of controlling their destiny. Our collective paradigm
leads us to prefer qualitative research that is inclusive to ensure that all voices are heard and
Engaging Campus (CECE) Model proposed by Museus (2014), a model aimed at generating data
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
and evidence that can inform efforts to transform institutions of higher education, from policies
and pedagogy, to activities and program, like NIU’s SI program. Museus’ CECE Model outlines
the elements of campus environments that are necessary for students to thrive in college and how
these environments can positively affect student outcomes. Building on other well-known
theories of student success developed by Tinto (1975, 1987, 1993), Astin (1993, 1999) and Kuh
et al. (2005), Museus provides a more racially and culturally responsive model – one that
“advances an equity agenda that promotes the development of campus environments that allow
students from all backgrounds to thrive” (Museus, 2014; Museus & Smith, 2014). The CECE
model includes two clusters of indicators. The first cluster – cultural relevance – includes five
indicators that “characterize campus environments that meaningfully engage and reflect the
cultural backgrounds, communities, and identities of diverse students” (Museus & Smith, 2014).
Cultural responsiveness, the second cluster, consists of four indicators that “reflect environments
in which an understanding of diverse students’ cultural norms and values undergirds campus
learning and support systems that respond to these students’ needs” (Museus & Smith, 2014).
The CECE Model is congruent with the collective epistemology of our assessment team.
A total of 19 students, or 2.5% of the targeted sample, consented to begin the study. Of
the 19 students, 12 moved beyond consent and completed most of the survey, and 11 completed
the survey to the end, for a full response rate of less than 1.5%. One student chose not to answer
Asked to describe their ethnicity, 9 participants responded with the following: Black or
African American (2), Hispanic or Latino (4), Asian (2), White (1). Two students chose not to
answer. It should be noted that this was an open-ended question “When asked, how do you
define or describe your ethnic background?” to remove limitations. It is possible that a multiple-
choice question with common ethnicities and races options and one “Prefer to self-describe”
option may have yielded more complete results. Nine students identified as female and two as
male. No responses were noted for other gender options, including non-binary/third gender;
genderqueer, prefer to self-describe, and prefer not to say. Asked about sexual orientation, nine
students indicated heterosexual/straight, one preferred not to say, and one preferred to self-
describe with “straight.” Ten students identified as being 18-20 years old, and one student
indicated an age range of 25-29. One of the eleven respondents self-identified as a transfer
opportunity to participate in SI for certain courses, and promising practices for marketing the
program to students. Survey respondents indicated that they most often learned about SI from an
SI Leader (7 responses). SI Leaders are students who are screened and hired by SI staff to
provide the supplemental instruction sessions for the selected courses. As part of their SI Leader
role, they attend the course with the students and recruit prospective students to SI by making in-
class announcements during one of the first few class sessions. Respondents also learned about
bulletin board flyers were other less-cited ways students learned about SI. Program staff both in
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
SI and in CHANCE indicate that participation in SI sessions is not required but is recommended.
However, when asked in the survey to choose one answer as to their understanding about
attending SI session, half of respondents indicated SI was required; 25% that it was optional, and
The three classes for which respondents most utilized SI were Fundamentals of
Algebra MATH 110P. Introductory Financial Accounting (ACCY 206), Communication Skills –
Reading (LTRE 100P), College Reading Study Strategies (LTRE 190), Chemistry (CHEM 110),
Basic Statistics (STAT 100), and Rhetoric and Composition I (ENG 103) were other courses for
which respondents indicated they had participated in SI sessions. It should be noted that SI
sessions were not offered for Rhetoric and Composition I (ENG 103) during the two semesters of
this study.
According to program staff, SI sessions are typically offered twice each week, during the
same time slot as the course is offered, but on the off days. For example, SI sessions are offered
on Tuesday and Thursday at 9 a.m. for classes that are taught on Monday-Wednesday-Friday at 9
a.m. When asked on average how many SI sessions they attended each week, six responded they
attended twice a week; three attended once a week, and three attended three times a week.
A corollary objective of the SI program is to encourage students to engage more with the
humanized educational environment per the CECE model (Museus, 2014). Half of all
respondents indicated they had met 2-3 times with their instructor outside of class for assistance,
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
and 17% met four or more times. A third of respondents indicated they never met with the
instructor outside of class. Engaging with the curriculum is also an important indicator of success
for students in the SI program. Not counting class time and SI sessions, five respondents (42%)
indicated on average that they studied or did homework for each of their SI supported classes 4-6
hours each week, and five indicated they studied or did homework for 2-3 hours each week for
the classes. Two indicated one hour or less per week was spent on studying or homework for
these classes.
Johnson, is if students move on to the next course in the sequence the next semester, such as
taking MATH 108 and then MATH 109 the following semester. Eleven of twelve respondents
indicated that they did enroll in the next level course for the following semester.
ways: helped with my understanding of the subject (31%); helped improve my grade in the
course (23%); helped build confidence in my abilities (23%); helped me connect with other
students in my class (14%); helped me connect with informal mentors (6%); and helped me
connect with my instructor (3%). Asked about how their final grade in the course compared to
the grade they had expected to get at the beginning of the semester, 67% indicated it was “quite a
bit higher” and 25% that it was “a little bit higher” than expected.
moderately helpful (33%) and remaining respondents found it somewhat or slightly helpful.
Two-thirds of all respondents indicated that they would definitely (5) or probably (4) plan to use
SI services for other courses in the future. Three students indicated that they would like to have
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
or wish they would have had SI services for these classes: CHEM 211, MKTG 295, MGMT 217,
Given the opportunity to share some of the things that they found helpful about SI, students
wrote:
· “It helped me understand any material the professor did not clear up.”
A significant majority (92%) of respondents felt that their SI Leaders were very (50%) or
extremely (42%) knowledgeable and well-prepared, and 86% of respondents who met with their
SI Leader during office hours indicated that their sessions worked well with their schedules.
Engaging Campus Environment (CECE) at NIU. All respondents indicated that they agreed
(36%) or strongly agreed (64%) that SI Leaders and staff members made them feel welcomed.
Responses were similar regarding “SI Leaders and staff members made me feel respected”: 73%
strongly agreed and 27% agreed. These responses indicate a positive impact of SI on the
philosophies and holistic support under the CECE model, respondents strongly agreed (64%) or
agreed (27%) with the statement “SI staff members sent me important information about where I
can access academic support.” One respondent strongly disagreed with this statement. Responses
education environment that offers holistic support. For “SI Leaders or staff checked in on me
regularly” 73% strongly agreed or agreed with the statement, and 18% disagreed or strongly
Two additional statements were offered to assess participants’ perceptions of the extent to
which SI fosters a culturally responsive humanized educational environment: “SI Leaders were
interested in getting to know who I am” and “SI staff members were interested in getting to know
who I am”. Fifty-five percent of respondents agreed with the statement regarding SI Leaders,
with 36% somewhat agreeing or neutral. One participant somewhat disagreed. With regard to SI
staff, 60% of participants agreed or strongly agreed; 30% somewhat agreed or were neutral; and
challenge, I knew I could go to SI Leaders for support” and the same statement for SI staff.
Responses for both questions were very similar, indicating that 82% strongly agreed or agreed
with the statement regarding SI Leaders, and 18% somewhat agreed. For SI staff, 91% agreed or
were also measured to assess the program’s cultural relevance, especially cultural familiarity, for
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
SI participants, with the statement “It is easy to connect with SI Leaders who share similar
backgrounds and experiences as me” and the same statement, but for SI staff. Eighty-two percent
of respondents strongly agreed or agreed with this statement for both SI Leaders and SI staff,
Focus Groups
Focus groups were held via Zoom both for students who did not participate in SI, and for
students who did participate in SI. All 2,062 unique students who were enrolled in the multiple
sections of the seven courses identified earlier were invited and encouraged to participate in SI
during the Fall 2019 and Spring 2020 semesters. Just over 36% of these students took advantage
of the opportunity for additional academic support offered by SI for these courses.
Of the 2,062 students who were eligible to participate in SI, 749 did participate in SI
sessions and received the email invitation to complete the survey. In addition to containing a link
to the survey, the invitation that was emailed to these 749 students also notified students that at
the end of the survey, they would have an opportunity to sign up for a focus group. Twelve
students completed the survey, and at the end of the survey, were asked if they would like to
participate in a focus group to provide more information. One of the twelve survey participants
responded positively to the focus group invitation and signed up on an external link using Sign-
Of the two times available to the students to participate, the student chose the later time,
11/17/20. The students received an email (Appendix C) the day that they registered for the focus
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
group that thanked them for participating in the focus group, provided them with instructions and
a link to join the Zoom meeting, and allowed them to respond with any questions they may have
had. They then received email reminders on the day before the focus group and a few hours
before the focus group took place, if perhaps they had forgotten since they had originally
There were several themes throughout the focus group from the student that participated
in SI. First, the communication to the student was an important factor for their participation in
the program. The student mentioned that they received notifications of where and when their
scheduled SI appointment would take place, and they mentioned email and text as their preferred
method of contact. They also acknowledged that they learned about Supplemental Instruction in
orientation and through the CHANCE program, but they believed that reaching out to students
Another theme that presented itself throughout the focus group was the benefits and
challenges of using Supplemental Instruction. The student indicated that it helped to improve
their grade, and it also helped them to get caught up with anything that they were having
difficulty understanding. It also helped them with homework completion, and they benefited
from both their SI leaders and the other students in the class when having difficulty working out
a problem. They mentioned that it then allowed them to have a better understanding of the
material when the professor would discuss it during their lecture. Also, for this particular
student, they felt that SI provided times that fit their schedule well. The only challenges that
were discussed were minor technological issues, feeling as though they were falling behind the
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
others when missing several sessions consecutively, and many students compared to the number
of SI leaders.
The third theme that ran throughout the focus group was the helpfulness and respect
shown by the SI leaders and other students. They believed that the SI leaders were willing to
help, very kind, and very helpful with assisting the students in understanding the subject. The
student commented on how students within the program would help each other, as the SI leaders
were sometimes busy helping others. However, they mentioned several times that the leaders
were “nice,” “helpful,” and able to help the students learn the material.
Sixty-four percent of all students eligible to participate in SI during the Fall 2019 and
Spring 2020 session did not participate in SI sessions. An email invitation to participate in a
focus group was sent to all 1,313 students who did not attended SI sessions. Of these 1,313
invitations, three students responded positively to the focus group invitation, signing up through
a link using Sign-Up Genius with their email addresses. However, of the three students that
registered, only one student eventually participated in the focus group, and the other two
participants did not respond or reach out to say that they would not be participating.
Of the two times available for the students to participate, all three students chose the later
time, 8:00-9:00pm CT on 11/12/20. There was also an option to participate from 6:00-7:00pm
CT on 11/13/20. Each of the four students that signed up for either of the two focus groups
chose the later of the two times, and, thematically, this has implications as to when we should
provide services to our students. It is essential that we recognize that times that are most
convenient for our students may often be different than times that are convenient for faculty or
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
staff. However, if we wish to provide services that can be widely accessible to our students, it is
important to recognize when they are available and most willingly to participate in SI or any
other student service. Similar to the focus group of those that had participated in SI, the students
also received an email (Appendix C) the day that they registered for the focus group. They then
received email reminder on the day before the focus group and a few hours before the focus
group took place, if perhaps they had forgotten since they had originally registered (Appendix D
& E).
Conclusion
In approaching this assessment through the lens of the CECE Model, we sought to let the
voices of the students themselves provide insight to the students’ sense of belonging and success
at NIU with the assistance of SI services and to help identify opportunities for more culturally
responsive support systems and culturally relevant high-impact practice in SI at NIU. Our goal
with having student input is to make sure we are allowing the space for students to feel heard and
be seen. A lot of times we have discovered that students recognize when they are not given an
opportunity to share their viewpoint on experiences they have throughout our university and that
may leave a negative connotation in their minds. That negative connotation could potentially
come from spaces that are not existing to allow students to express themselves and their
experiences. With this in mind, we hope to create an inclusive space for diverse students to have
the freedom to give their thoughts and opinions so that programs like SI can still be in existence
References
Arendale, D. R., & Hane, A. R. (2014). Holistic growth of college peer study group participants:
Association of Colleges for Tutoring and Learning Assistance (2019). ACTLA principles,
content/uploads/2019/04/2019_Online_Tutoring_Standards.pdf
Banta, T. W., Jones, E. A., & Black, K. E. (2009). Designing effective assessment: Principles
https://doi.org/10.1111/j.1467-9647.2011.00700.x
https://files.eric.ed.gov/fulltext/EJ1210442.pdf
Bates, D. K. (2016). Perceptions from Athletic Training Students Involved in an Intentional Peer
Bosmans, D., Young, E., & McLoughlin, R. (2019). Does PAL Work? An Exploration of Affect
Council for the Advancement of Standards in Higher Education (CAS) (2018). Learning
PDF=E86D2FCA-DBEC-AD47-33AB941E185E1E67
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Dawson, P., van der Meer, J., Skalicky, J., & Cowley, K. (2014). On the effectiveness of
assisted study sessions literature between 2001 and 2010. Review of Educational
Hatch, D., & Bohlig, E. (2016). An Empirical Typology of the Latent Programmatic Structure
Henning, G. W., & Roberts, D. (2016). Student affairs assessment: Theory to practice. ProQuest
Ebook Central
Koselak, J. (2017). The revitalized tutoring center. Phi Delta Kappan. Retrieved September 24,
LaClare, E., & Franz, T. (2013). Writing centers: Who are they for? What are they for? Studies
Malm, J., Bryngfors, L., & Fredriksson, J. (2018). Impact of Supplemental Instruction on
Dropout and Graduation Rates: An Example from 5-Year Engineering Programs. Journal
Munley, V. G., Garvey, E., & McConnell, M. J. (2010). The effectiveness of peer tutoring on
student achievement at the university level. American Economic Review, 100(2), 277-82.
http://doi.org/10.1257/aer.100.2.277
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Muñoz, S. M., & Espino, M. M. (2017). The freedom to learn: Experiences of students without
legal status attending Freedom University. Review of Higher Education, 40(4), 533–555.
https://doi.org/10.1353/rhe.2017.0021
Museus, S. D. (2014). The Culturally Engaging Campus Environments (CECE) model: A new
theory of success among racially diverse college student populations. In M.B. Paulsen
(Ed.), Higher education: Handbook of theory and research (Vol. 29, pp. 189-227).
Springer.
Museus, S. D., & Smith, E. J. (2014). The Culturally Engaging Campus Environments model
and survey: A report on new tools for assessing campus environments and diverse
https://www.naspa.org/images/uploads/main/CECE_Report_EXEC_SUM_DOWNLOA
D.pdf
Norton, J., & Agee, K. S. (2014). Assessment of learning assistance programs: Supporting
professionals in the field [White paper]. College Reading and Learning Association.
https://www.crla.net/images/whitepaper/AssessmentofLearningAssistancePrograms2014.
Osborne, J. D., Parlier, R., & Adams, T. (2019). Assessing Impact of Academic Interventions
9–26.
https://www.qualtrics.com/experience-management/research/t-test-analysis/
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Qualtrics. (n.d.). What is factor analysis and how does it simplify research findings? Retrieved
from https://www.qualtrics.com/experience-management/research/factor-analysis/
Schuh, J. H. and Associates (2009). Assessment methods for student affairs. Jossey-Bass.
Schuh, J. H., Biddix, J. P., Dean, L. A., & Kinzie, J. (2016). Assessment in student affairs (2nd
ed.). Jossey-Bass.
Schuh, J. H., Biddix, J. P., Dean, L.A., & Kinzie, J. (2016). Assessment in student affairs (2nd
ed.). Jossey-Bass.
Strange, C. C. & Banning, J. H. (2015). Designing for learning: Creating campus environments
Trochim, W. (2014). Experimental design. Retrieved from Research Methods Knowledge Base
website at http://www.socialresearchmethods.net/kb/desexper.php
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix A
The following survey was sent to all SI participants during Fall 2019 and Spring 2020.
1. How did you learn about Supplemental Instruction (SI) at NIU?(Check all that apply)
b. NIU Orientation
e. SI Leader
f. NIU Website
g. Friend/Roommate
h. Academic Advisor
j. CHANCE Advisor
l. NIU Announcements
2. What was your understanding about attending SI group sessions? (Check one)
3. For which of these semesterss) did you participate in Supplemental Instruction classes?
a. Fall 2019
b. Spring 2020
4. For which classes did you participate in SI sessions? (check all that apply)
k. Other(s)
5. How many sessions of SI were offered each week for each of the classes you checked
above?
a. 3
b. 2
c. 1
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
d. 0
6. For each of the classes you checked above, on average, how many SI group sessions did
a. 3
b. 2
c. 1
d. 0
7. For each of the classes you checked above, on average, how many times did you see the
a. Never
b. Once
c. 2 – 3 times
d. 4 – 5 times
e. 6 or more times
8. In Fall 2019, approximately how many total times did you attend SI sessions for each of
a. Never
b. 1 – 4 sessions
c. 5 – 9 sessions
d. 10 – 15 sessions
e. 16 or more sessions
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
9. In Spring 2020, approximately how many total times did you attend SI sessions for each
a. Never
b. 1 – 4 sessions
c. 5 – 9 sessions
d. 10 – 15 sessions
e. 16 or more sessions
10. Not counting class time and SI sessions, on average, how many additional hours each
week did you study or do homework for each of the classes you checked above?
b. 2 – 3 hours a week
c. 4 – 6 hours a week
d. 7 – 10 hours a week
11. For classes in which you participated in SI for Fall 2019 and/or Spring 2020, did you
enroll in the next level of the course for the following semester? As an example, if you
were in SI classes for MATH 108 in Fall 2019, did you enroll in MATH 109 in Spring
2020?
a. Yes
b. No
2. How helpful did you find Supplemental Instruction group sessions? (Branched open-
ended question if answer a-d: What are some of the things that you found helpful about
a. Extremely helpful
b. Moderately helpful
c. Somewhat helpful
d. Slightly helpful
3. Thinking about the grade that you had expected to get in the course at the beginning of
the semester, how did your final grade in the course compare to the grade you had
expected?
b. A little higher
d. A little lower
4. How knowledgeable and prepared did you feel your SI leaders were for the classes for
5. If you met with your SI Leader during office hours, how well did your session(s) with the
SI Leader work with your schedule? (Branched open-ended question if answer is e: What
was it about your SI Leader session(s) that did not work well with your schedule?)
a. Very well
b. Moderately well
c. Somewhat well
d. Slightly well
e. Not at all
6. Where did you attend Supplemental Instruction group sessions? (Check all that apply). If
answer is both ‘a’ and ‘b’, branch to question 6a: Do you prefer one mode over the other?
If yes, “Which mode of SI instruction did you prefer?” and “Why do you prefer that
mode of SI instruction?”)
a. Face-to-face at ____________________________
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
b. Online
a. Yes, definitely
b. Probably
c. Maybe
d. Probably not
8. Are there other courses for which you would like to have SI services for? You can
include classes that you have already taken for which you wish you would have had SI
services.
a. None
b. These classes
1. SI leaders and staff members made me feel welcomed. (If answer is either ‘d’ and ‘e’,
branch to question “Was there a specific situation or incident that led you to not feel
welcomed?”)
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
2. SI leaders and staff members made me feel respected. (If answer is either ‘d’ and ‘e’,
branch to question “Was there a specific situation or incident that led you to not feel
respected?”)
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
3. SI staff members sent me important information about where I can access academic
support.
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
e. Strongly disagree
9. It is easy to connect with SI leaders who share similar backgrounds and experiences as
me.
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
10. It is easy to connect with SI staff members who share similar backgrounds and
experiences as me.
a. Strongly agree
b. Somewhat agree
d. Somewhat disagree
e. Strongly disagree
NIU)?
a. Yes
b. No
2. How many continuous semesters have you attended NIU, including Fall 2020?
a. 3.5 or higher
a. Male
b. Female
c. Non-binary/third gender
d. Genderqueer
e. Prefer to self-describe
a. Gay or Lesbian
b. Bisexual
c. Straight/heterosexual
d. Prefer to self-describe
1. Are you willing to also participate in an online focus group to further share your
for the focus group, and will also receive a message: “Thank you for your responses to
this online survey – your input is very valuable to this assessment process!” If answer is
a. Yes
b. No
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix B
The following survey will be sent to all SI participants for Fall 2019 and Spring 2020.
Survey Invitation
Thank you for choosing to complete our survey and provide feedback on your experience with
Supplemental Instruction at Northern Illinois University! By clicking the button below, you
acknowledge that your participation in this study is voluntary, that you are 18 years of age or
older, and that you are aware that you may choose to terminate your participation in the study at
any time and for any reason. This survey is part of a class project in the Applied Assessment
Methods in Higher Education class and through the guidance of Academic Support Services at
Northern Illinois University. Please contact Gail Jacky (gjacky@niu.edu), Dr. Kimberly (KJ)
Johnson (kjohnson@niu.edu), or our professor Dr. Jacqueline Mac (jmac@niu.edu) if you have
any questions.
Appendix C
For the focus group conducted with students who did and did not attend Supplemental
Instruction sessions, the following email will be sent. Since this focus group is meeting via the
Zoom platform, this email should contain a link to the Zoom meeting. It will also contain the
details of the focus group (i.e. time) and thank the student for attending. This email should be
sent as soon as possible after the student registers for the focus group.
Title of the Email: “Supplemental Instruction Focus Group Registration & Zoom Link”
Email:
Dear ________,
Thank you for registering for the focus group to help us improve Supplemental Instruction at
Northern Illinois University. You have registered for the _:__ time slot on (day) , __/__. Please
use the Zoom link provided below to join the meeting, and please plan to join the meeting 5
minutes prior to our starting time, to ensure that Zoom opens properly. We are excited to have
you join our focus group, and we look forward to hearing your input on Supplemental
Instruction. Please feel free to email us with any questions that you may have.
Best,
Nate Foster, nfoster@niu.edu
Alex Torres, atorres8@niu.edu
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix D
1. Briefly introduce yourselves: your name and what year you are in school.
2. What are your perceptions of using academic support services?
a. What do you think the benefits are of using academic support services?
b. What are the challenges of using academic support services?
3. Have you used any academic support services at NIU? Which ones have you used?
4. As a student, have you heard of SI?
a. If yes, how did you hear about SI?
b. If no, how could they have better reached/advertised SI to the students?
5. What do you think the most/least appealing part of the program was?
6. What kept you, or prevented you from using SI?
a. Do you think you would have benefited from using SI?
b. Do you have a preferred time that would be more accessible for you to use SI?
c. Do you have any friends who have used SI? If yes, do you think they benefited
from using SI? (Optional)
7. Anything else you would like to add about academic support services impacting your
level of satisfaction with NIU?
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix E
Users of SI
1. Briefly introduce yourselves: your name and what year you are in school.
2. Tell us a little bit about how you heard about SI.
a. How could they better reach/advertise to the students? Did you receive reminders
before tutoring sessions? If you did not, would this have been useful?
3. What, or who, would have encouraged you to participate in SI?
a. How often did you use SI?
b. What subject did you use SI for?
c. What were some of your perceptions of the program before you participated?
4. In what ways have your experiences with SI influenced your learning?
a. In what ways were you able to apply what you learned in SI in your classroom?
5. What are the benefits of being engaged in SI?
a. What are some of your perceptions of the program now?
6. What were the challenges of using SI?
7. Would you have preferred a more accessible time for SI?
8. Do you recommend other students use SI? Why or why not?
9. Is there anything else you’d like to share about your participation in SI?
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix F
The following email will be sent to students the day before their scheduled focus group. As a
substantial amount of time may have passed between when the students first registered and
received their Zoom meeting links, this will work to ensure that students remember the time and
details of the focus group. This email will remind the students of the focus group and will allow
them some time to plan around the meeting, if, perhaps, they had made other plans during the
focus group time.
Email:
Hello ________,
Thank you for registering for the focus group on Supplemental Instruction at Northern Illinois
University. This is a reminder that you have registered for the _:__ focus group tomorrow,
__/__. Please email us if you have any questions about the focus group. To be as accurate as
possible while analyzing our focus group discussion, we intend to record the meeting. However,
if you prefer that we do not record the focus group, please email us and let us know that you
would not like us to record. Also, no names will be documented in the analysis of the focus
group, as all volunteers will be referred to as Student A, Student B, etc. If you know of anyone
that would also be willing to participate in our focus group, then please invite them to join us and
let us know. We look forward to meeting you and hearing your thoughts on Supplemental
Instruction.
Here again is the Zoom link information: __________________________________________
Thank you,
Nate Foster, nfoster@niu.edu
Alex Torres, atorres8@niu.edu
INITIAL FINDINGS: ASSESSING ACADEMIC SUPPORT
Appendix G
The following email will be sent to students the day of their scheduled focus group. This email
should be sent a few hours before the focus group begins, and it will ensure that the students
remember the time and details of the focus group. It will also provide them an opportunity to ask
any questions that they may have.
Email:
Hi ________,
Thank you for joining us tonight for the focus group on Supplemental Instruction at Northern
Illinois University. This is a reminder that the focus group will take place at _:__ this evening.
Please email us if you have any questions about the focus group. To be as accurate as possible
while analyzing our focus group discussion, we intend to record the focus group. However, if
you prefer that we do not record the focus group, please email us and let us know that you would
not like us to record. Also, no names will be recorded or documented during the focus group. In
our analysis of the focus group, all volunteers will be referred to as Student A, Student B, etc.
We are looking forward to tonight’s meeting.
Thanks,
Nate Foster, nfoster@niu.edu
Alex Torres, atorres8@niu.edu