Artifact

Jobs PLUS Team A: Piloting Pre- and Post-Assessments 

Olivia Elliot, Hans Varga, Mitchell Huftalin, Timothy Klagues 

Department of Higher Education and Student Affairs, Northern Illinois University 

HESA 573: Applied Assessment Methods in Higher Education 

Dr. Mac 

December 7th, 2020 

  

 
 Abstract 

Jobs PLUS presented the opportunity to collaboratively create pre- and post-assessments

for each of the four competencies: communication, leadership, diversity and inclusion, and

building your network. Due to the limited time to work on this project, the team built four of

eight assessments, focusing on the two core competencies that had the most events. Those were

leadership and building your network. The pre- and post-assessments were piloted in early

November to collect information on student learning for a leadership event. As a group, several

recommendations and future implications for events were drawn. The methods used for this

project included multiple surveys created through Qualtrics. The pre-assessment survey included

60 responses out of 71 attendees and the post-assessment contained 42 student responses. The

purpose of these assessments was also to gain more in-depth and specific information on student

learning from each of the four competencies. 

Introduction 

Upon meeting with a campus partner, a need for a reevaluation of assessment tools

appeared. As a program that prides itself on the opportunities for connections and growth within

the personal and professional lives of undergraduate students, the program sought to learn more

about what students gain from attending their events. One of our guiding questions relates to the

four competencies of Jobs PLUS (communication, leadership, diversity and inclusion, and

building your network) and what students learn or gain from each event that focuses on each

respective competency based on the learning objectives of the program.  

Jobs PLUS 
 The Jobs PLUS program at Northern Illinois University seeks to bridge the gap between

professional development needs for students and desirable experience and knowledge desired by

employers. The program identifies the difficult balance a student encounters as they complete

academic coursework while employed. Student workers have less time to attend programming

designed for professional development as students need to reserve time for classes, homework,

and personal lives outside of work. Jobs PLUS allows students an opportunity to learn and apply

their new knowledge to their current position and prepare for their future careers. As stated

previously, Jobs PLUS centers on four core competencies to build the student leaders:

communication, leadership, diversity and inclusion, and building your network. 

We sought to look at the core competencies within the lens of learning outcomes, thus

allowing us to see the effects of the competencies when employed in the program. It was

important to investigate and assess what students learned at each event as it provided our

team the opportunity to look at the learning objective of each event. Further, we were able

to assess whether the learning objectives were met based on the assessment results. By assessing

student learning outcomes, we were able to look at the data to determine what improvements

can be made for the next events. Additionally, we looked to benchmark and

research other institutions to see what types of programming they have similar to Jobs PLUS.

Regarding benchmarking, we researched and assessed other institutions to gather information

about how they are conducting assessment with their programing.  As a result, our

team implemented some ideas, offered suggestions for practice, and used findings to improve

current efforts. We intend to incorporate information from other institutions to determine what

can be incorporated into the Jobs PLUS program at NIU.  

 Review of Literature 

The team found multiple areas we needed to become more familiar with. The first area

we needed to look at was research that gave information about the general description of

programs similar Jobs PLUS and the impact it had on student experiences and learning. Doing

this assisted the team in analyzing results by giving examples of how other programs are run, and

the results they received. The second area knowledge needed to be gained was about different

methodologies applicable to conducting this assessment. Each team member had some course

background with assessment but wanted to ensure a firm understanding prior to proceeding. The

final area the team wanted to look at was the impact programs such as Jobs PLUS had on other

aspects of student development. These areas included job preparedness and application of

learnings inside and outside of the classroom. By looking into the effects such programs had on

retention, the team was better prepared to tailor the assessment to the needs of the client. 

Hybrid Programs in Higher Education  

In an article written by Bennett, Knight, and Rowley (2020), they expressed employers

are looking for students to be ready to join the workforce prior to graduation. However, students

identified themselves as not prepared for such a task. Bennett et al. continued to state students

must participate in career preparatory activities, including technology, that highlights their

experiences to showcase job preparedness. 

Hybrid programs combine learning online and in the classroom. It is important to tailor

such programs to students needs as some students can only participate in programs online, while

others prefer to or are able to participate in-person. Some barriers that may prohibit students
from participating include not having the technology to participate online and not having a way

to commute to in-person programs.  

Staff who work hybrid programs should assess programs after each is completed.

Assessing at the end of each program and sharing findings with students is important because it

gives staff members the opportunity to discuss results with students, ask questions, and make

necessary changes prior to the next program. By creating self-assessments for students

participating in the program, researchers were able to identify how to cater career workshops to

reflect student's needs. Following the career workshops, students were tasked with creating a

self-care plan on how to transition into the workforce after graduation. This can create an action

plan and return responsibilities to the student to apply their learnings from the event to their life. 

Bennett, Knight and Rowley (2020) also established that, in the authors minds, a key

component to the success of these programs is allowing students to see collected results. Seeing

the collected results allows students to have buyback on the progress programs are bringing to

students. As the team moved further into the assessment project, this proved important to

remember, especially as we were attempting to look at the impact these studies can have. If a

program that is based off hybrid learning is moderately successful, it should be looked at

regularly and changes should be made to make it more successful or more targeted to aid

students in their transitions in the future. The authors also “contend that the hybridity combined

with the use of existing resources positions this approach well for use with large student cohorts”

(Bennett, Knight & Rowley, 2020, p.13) As the team looked at the roles these critical programs

hold in higher education, understanding how they are best implemented and adapted to work in

higher education was important.  

 Bullen, Kordecki, and Capener (2018) outlined the disparities that may occur between

students’ careers as students to employees. This connection starts in the classroom but must

continue outside. Bullen et al. (2018) expressed “professors of all backgrounds help students by

making opportunities available for practitioner/student interaction to learn about the profession

in the classroom setting – and by encouraging opportunities for this interaction and learning

outside the classroom” (p. 5). While Bullen et al. continued to focus on how the curriculum can

benefit students when preparing them for their careers, the curriculum should have included

additional events built in to supplement learning in the classroom. Something Jobs PLUS

identified was students not having time to attend leadership meetings or conferences. As a result,

they attempt to offer resources to students while on the job. Bullen et al. further suggested

“students may not only have limited hours to put forth in outside endeavors, but also the timing

of such endeavors can conflict with other activities” (p. 9). 

Impact of Working and Employment on Student Outcomes  

In a dissertation written by Savoca (2016), she discussed student engagement, high

impact practices, student employment, and the impact of being a working student. To emphasize

the true purpose of the research, Savoca emphasized engagement was not limited to the

classroom, but also included out of the classroom experiences like employment or other

leadership experiences. A study conducted by Kuh (1995), concluded “leadership and work

experiences helped them (participants) clarify career goals, apply knowledge to real-world

situations, and develop practical, career-relevant skills” (as cited by Savoca, 2016, p. 25). Savoca

continued to present several other studies emphasizing the positive impact out of classroom

experiences had on the development of students. The information presented from this
dissertation signifies a linkage between the positive outcomes from other studies and the benefits

of Jobs PLUS.  

Programs like Jobs PLUS can also be referring to as experiential learning opportunities.

Savoca took some time to explain her research of high impact practices as experiential education.

Several years of research had demonstrated “experiential education positively contributes to

student learning, academic achievement, and persistence” (Astin, 1994; Hesser, 2014; Patel et

al., 2010 as cited by Savoca, 2016, p. 30). Further, it has been proven such activities provide

chances for students to apply what they learned in the classroom to gain real-world, applicable

experiences. Relating to Jobs PLUS, such programs and employment provide opportunities for

students to apply knowledge to future jobs, interviews, and more.  

Academics 

Baran (2010) focused much of her research on the academic impact of on-campus

employment. Several aspects covered are persistence, academic achievement, balance, and

institutional integration. While many assume increased amounts of time participating in

extracurricular activities can take away from one's academic focus, Baran researched to prove the

opposite. Research prior to the 2000s presented a negative impact on academics for those

working. However, data collected by the National Survey of Student Engagement (NSSE)

proved to be the turning point. An overall emphasis was placed on the amount of time spent

working. Reportedly, students who worked 20 hours or less per work on-campus had higher

grades (NSSE 2000 & 2004 as cited in Baran (2010). Overall, as with any extracurriculars,

balance is required to manage responsibilities and still be high achieving. 

Importance of Pre- and Post-Assessments  

            An article written by Hockett and Doubet (2013) mentioned pre-assessments are

assessments given before an event or lesson and explained how pre-assessments play an

important role in education along with determining learning goals or objectives. It was important

to note pre-assessment questions should be aimed at measuring student understanding.  In one

articles, McDevitt, Patel, Rose, and Ellison (2016) concluded that post-assessments are

assessments given at the end of a lesson or event. Pre-assessments should evaluate a student's

understanding about a respective topic(s) for an upcoming event or lesson, and post-assessments

should evaluate student learning after an event or lessor. Pre- and post-assessments proved

appropriate for this project because the team was looking to gain and understanding of students'

knowledge about the two Jobs PLUS competencies, leadership and building your network, prior

to the respective competency's event and an understanding of how/if that knowledge transformed

from attending the event.  

Literature Gaps 

After conducting research for the literature review, the team found a lack of information

related to career development high impact practices in the United States. Many of the studies

found were from other countries like Australia. While this information is helpful, it is unclear

how relatable it is to educational systems in the United States, as well as the development of

students. Due to the scope of the project, the team was also unable to draw conclusions on the

impact working and participating in Jobs PLUS has on a student's academics. The research could

prove helpful for future studies on the impact Jobs PLUS has on student learning inside and
outside of the classroom, the impact participation has on grades, and if participation impacts the

amount of time spent on schoolwork. 

Methodology 

When starting any great endeavor, it is important to first develop a plan in which to

follow. This stands true for most things, but especially stood true for the team in the case of

creating a data analysis survey, as well as completing an assessment of a certain program.  The

following sections dive into the method for data collection, with a special note on the creation of

the team's assessment. Following, will be an in-depth review on how the analyzed the retrieved

data. Specifically, this section will cover analyzing and making meaning of responses to both the

pre- and post-assessment surveys. To ensure the team upheld the correct standards, there is a

discussion of the different standards and ethics that played a role in conducting this semester

long assessment project. 

Data Collection  

Data collection was among one of the most crucial pieces of the project and of the events.

Of course, the creation and execution of Jobs PLUS events is critical, but feedback on the

experiences of those who attended proved incredibly insightful to the team in order to provide

recommendations for the future. It was the teams hope to use the collected data to provide

recommendations for improving the created assessment and an outline for creating the remaining

four pre- and post-assessment for the competencies: communication, diversity and inclusion.

Overall, the data collected will assist in strengthening events and assessment for semesters to

come. 
 The team was able to collect some information from our initial meeting with the

client. Some of the information the team was able to collect was on the scope of the project, the

requested outcomes, and the team also received copies of previous surveys and their results. The

team utilized this information to create four unique pre- and post-assessments for leadership and

building your network, the two Jobs PLUS competencies with the most events of the Fall 2020

semester. 

Source of Data 

The sources of data for our project came from Jobs PLUS. Specifically, the participants

were the students who attended Jobs PLUS events and took the pre- and post-assessment. The

team was incredibly lucky with the collection and recruitment aspect of collecting data as

students are required to register for Jobs PLUS events prior to their attendance. As a result, the

team did not have to recruit participants to take our assessment, but rather relied on the

recruitment of students to the events. Another aspect the team had to consider was the reminder

to complete the pre- and post-assessment. Fortunately, Qualtrics, the team's tool for creating and

distributing the assessment had options to email participants had not yet completed the survey. In

order to send follow-up communication, somebody from the Jobs PLUS program would need to

create and input a contact list of those who should be contacted for each event.   

Survey Creation   

To ensure our team had thoroughly thought of and created the most effective data

collection survey, we determined it was beneficial to have one individual create the survey and

have the remaining team members check the survey and offer suggestions. The team chose this
option to ensure a well-structured and cohesive survey, as well as to ensure the writing style

stayed the same throughout. The team used several questions from an existing survey and created

additional questions for each of the respective competencies. In regards to what the team asked

about leadership and building your network at the initial meeting, some questions included:

asking about previous or current leadership experiences and experiences with building their

network for their future job/career, rating how applicable leadership skills/experiences and

building your network are to their future careers/jobs before attending and after attending the

event, how their knowledge expanded about the topics covered during the event, and how they

will apply what they learned to their future career. Other questions the team asked included: what

they wish to gain from attending the event and if they gained what they wanted from attending.

The team determined this would be the best plan of action to ensure accuracy and consistency,

responses, and length among other things. Finally, before piloting the survey, we had the client

review and provide feedback on necessary or suggested changes. 

Data Analysis  

This semester, the team collaborated with the client to determine a focus would be placed

on two of the four competencies in order to create four of the eight assessments. This allowed the

team the chance to focus solely on the competencies that were most prominent through the

remainder of the semester. This also allowed the team the opportunity to launch surveys and gain

feedback on what can and should be adjusted. 

One of the ways the team analyzed the data was by looking at student learning outcomes

of the two competencies. Essentially, the team looked at the data from the pre- and post-

assessment from each of the two competencies to see what students knew about each
competency prior to attending the Jobs PLUS event and what they learned or gained from

attending the event. This helped the team determine whether the learning outcomes of each event

was met or if there needed to be a re-evaluation of future events.   

One of the ways the team made meaning of data from the pre- and post-assessment was

by determining future recommendations for events with each respective competency. Based on

future recommendations, the team also made meaning from the data collected by modifying

questions on the pre- and post-assessments. Another way the team made meaning of data was by

comparing the results to the learning objectives of each event.     

Standards and Ethics  

Standards and ethics are important in any work, especially work of institutions. In order

to maintain confidentiality of the data, or in this case, the students, the team determined not to

ask for personal information including students' names or Z-IDs. Instead, asked students what

year they were in school. The team wanted to gain information about year in school to determine

how and which students are being reached, and which group of students attended most. As a

result, this information allowed the team, and will allow the coordinators of Jobs PLUS to better

create events for the future and meet attendees where they are.   

Each team member had some knowledge and hands-on experience working with

Qualtrics. The team's knowledge and previous experiences provided us the foundation to

create an in-depth data collection survey that best suited the needs of our client and collected the

information they were seeking to gain. The team's knowledge and experiences made us

trustworthy and emphasized others can rely on the work completed. For students, trustworthiness
was gained by understanding no personal information was collected while providing the most

critical of feedback on programs. By collecting no personal information, the team hoped students

knew they could provide honest feedback and that there would be no repercussions of doing so. 

Recommendations 

After reviewing the data, as a team, we believed that there were recommendations that

could be implemented for future Jobs PLUS events. We believe that the recommendations that

we suggested will help Jobs PLUS foster more successful events in the future. We also believe

that the recommendations will help Jobs PLUS receive the data they are seeking to get from each

event. While these recommendations are related to one event, they would be beneficial to

implement in any event carried out by Jobs PLUS.  

For Practice 

After reviewing the pre- and post-assessment results, the team developed a few

recommendations we thought would be beneficial for future events. First, we recommend

including the respective pre-assessment in the registration process. This would ensure all

students seeking to participate in the respective event will complete the survey and share their

current knowledge about the topic, their experiences, and hopes for the meeting. Having students

complete the pre-assessment as part of the registration process can also assist the speaker, guest,

or Jobs PLUS staff in meeting students where they are.  

Second, we recommend cross-checking the number of attendees to post-assessment

responses. The team formulated this recommendation as we noticed 84.5% of student attendees
responded to the pre-assessment versus 59.2% of student attendees who responded to the post-

assessment. 

Third, we recommend included providing follow-up communication after the event,

specifically related to reminding students to complete the post-assessment survey. Creating

follow-up communication could increase the number of responses received on the event,

resulting in better feedback to update events in the future.  

Fourth, we recommend clarifying the goals of each event prior to registration and the

event. Clarifying goals for the event can assist students in staying focused and on task during the

event, giving their full attention to what they registered for. It also provides students the

opportunity to see what they will learn about during the event.  

Finally, we recommend creating a more streamlined system of assessment going forward

with events. Assessments are reviewed thoroughly and compared to each other determine the

primary purpose and goals for the assessments that are administered.  

Conducting additional research can prove helpful in the future as Jobs PLUS continues to

grow and develop. First, we recommend looking at what institutions are doing for programming

similar to Jobs PLUS. By completing this research, Jobs PLUS would be able to identify

components that are similar and different regarding their events and can also incorporate new

ideas for future events. Second, we recommend conducting research about incorporating student

learning inside and outside of the classroom events. This will allow students to demonstrate prior

knowledge from what they learned in the classroom and what they learned from previous jobs or

other experiences they have. 

 Implementing conversation based qualitative assessments at the end of each event or

semester is another recommendation the team believes would benefit Jobs PLUS. This can

include allowing students to see and discuss the results or their responses with staff who worked

the event. By implementing this type of assessment at the end of an event or semester, Jobs

PLUS would have an opportunity to see what changes can be made and given the time frame,

implement those changes prior to the next event.  

Given the dynamic times currently and the abrupt switch from in-person to online events,

and option for the future could be a combination of the two. Incorporating hybrid programs into

future events could prove beneficial to increase participation. While the team's findings

concluded most students would prefer for events to be in-person, offering a hybrid event would

give students who are unable to attend in-person the opportunity to participate. This also

provides students the option to choose which method would work best for them.  

Increasing promotional content for Jobs PLUS events on social media platforms would

help lead to an increase in student participation in events. Based on assessment results, only a

few students mentioned they heard about the event on social media. If events are shared over a

period of time on different platforms, it is more likely the event will get more registrants, and

there is an increased possibly it will be shared for others to see. Assessment results also showed

students mostly heard about the event from others. Sharing on social media could lead to campus

departments, students, alumni, and other companies sharing and promoting the event further.  

Major Findings 
 The goal of this assessment was to analyze the effectiveness of the Jobs PLUS programs

in meeting the goals established in the mission statement of the program. In analyzing the results

of the assessment, there were quite a few things that could be considered of major note. This

section will start by looking at the results that were compiled with the use of the quantitative

survey questions. After completing that review, this section will then look at the results that can

be seen from the qualitative questions. Then there will be an overall review of the complete

findings before concluding the rest if the paper. Please do note that the results of the post

program survey can be found in Appendix A. 

Quantitative Findings 

When reviewing the quantitative data that was collected, a few things can be surmised.

First, a good portion of students attending Jobs PLUS events are upper classmen, with the largest

selection of attendees coming from the junior class. The next statistic that holds importance is the

composition of the attending students regarding the reason for their attendance. Most of the

students attending came from one of two sources, either the Passport Program or the Honors

Program. The last thing that is of significance in regarding the quantitative section was the

response of the students about the length of the event, with most of the students saying that the

event was the correct length. 

These three specific questions provide a wealth of information that can be used for future

programming done by the Jobs Plus Team. If you analyze what is seen, you can develop not only

advertising that is directed at the less represented minorities of the student population, but you

can also tailor events that will either bring in more students that represent the norm, or students

that would not normally attend Jobs Plus events. Every piece of feedback is important, and if
utilized in certain ways, can help shape planning and advertising to maximize the effectiveness

of the events, but to also give students more access to them. 

Qualitative Findings  

Now looking at the qualitative side of things there are also a few questions that help

provide information that can be used to support and grow Jobs PLUS. The first question that

yielded results was related to the participants feelings on if the event was knowledgeable

(Questions 10, Appendix A). By looking at the data provided, it is possible to improve the event

to serve more students. As it is, the program that was examined during this survey was well

received, with some students offering feedback as to what might make the program even better.

If this data is used, you may see a larger return rate from previous participants who are trying to

gain real world experience. The second question that can provide a look at the effectiveness of

the program and allows planners to critique future events to better suit what students are looking

for (Questions 14 and 15, Appendix A). This part of the survey was conducted by asking

students what they liked about the event, and what they disliked about the event. While this event

was well received, it is important to always look for ways to improve your programs. By looking

at the results of questions 14 and 15, it gives you feedback directly related to the event, and helps

to shape future events so that they can be as well received if not better received when they are

hosted in the future. 

By using the data that was collected, it is possible to look at the direction that Jobs Plus is

heading and make minor adjustments to not only benefit the program, but the students that attend

events as well. To analyze every question from the survey would take multiple more pages than

what is currently written, but if you examine the data that was collected and used in the creation
of appendix item B, the PowerPoint presentation, you can see further information that helps to

explain the findings that were mentioned above. 

Conclusion

Assessment allows creators and observers to learn more about their programs, their

efficiency, and their effects on participants. Results can help improve programs and inform

creators on how they can best support others through specific data and responses. The Jobs

PLUS program identified a possible need for a revised assessment method, including pre- and

post-assessments, to help better understand the interworking of student learning and outcomes

through their programming. By implementing a pre-assessment, Jobs PLUS was able to identify

where the audience was at in terms of knowledge of leadership experience. Following the event,

students were offered the opportunity to express how the event may have changed their

experience with leadership opportunities and experience as it relates to their current position or

future career.  

As results showcased positive results for the leadership program, the implications of the

assessment process presented with several opportunities for change from a singular post-event

assessment. Offering an assessment opportunity to participants before and after the event allows

the presenter to cater their programming to the knowledge of the attendees and can create a more

streamlined process for learning. Student employees already have the challenge of adding an

event to their schedule. Giving a preview of the event through a pre-assessment allows for the

student to begin their engagement and look forward to the application that the event can bring to

their job or future career. Evaluation of these assessments should be conducted to ensure the

participants have clear outcomes prior to the event, and review of the post-assessments should be
used to create more purposeful and transformative events. The future of Jobs PLUS is ever-

changing with the constant change of student populations, as should the assessments used to

evaluate student and program outcomes. 

 
 References

Baran, J. M., (2010). On-campus employment: A form of engagement? Journal of Student  

Affairs, XIX, 57-66. Retrieved September 27, 2020,  

from https://mountainscholar.org/bitstream/handle/10217/37632/JOUF_JSA2010.pdf? 

sequence=1#page=59 

Bennett, D., Knight, E., & Rowley, J. (2020). The role of hybrid learning spaces in enhancing

higher education students’ employability. British Journal of Educational Technology,

51(4), 1188-1202. doi:10.1111/bjet.12931 

Bullen, M.L., Kordecki, G.S., & Capener, E.D. (2018). Student engagement activities to enhance

professional advancement in accounting and business careers. Journal of Instructional

Pedagogies, 20, 1-12.  

Freeman, M. (2013). Meaning making and understanding in focus groups: Affirming social and 

            hermeneutic dialogue. Qualitative Research: A Reader in Philosophy, Core Concepts, 

                and Practice, 354, 131-148. 

Hockett, J.A., & Doubet, K.J. (2013). Turning on the lights: What pre-assessments can do. 

Educational Leadership, 50-54.  

McDevitt, A.L., Patel M.V., Rose, B., & Ellison, A.M. (2016). Insights into student gains from 

undergraduate research using pre- and post-assessments. BioScience, 66(12),  

1070-1078.  

Rosenthal, M. (2016). Qualitative research methods: Why, when, and how to conduct interviews 

            and focus groups on pharmacy research. Currents in Pharmacy Teaching and Learning, 

                8(4), 509-516. 

Savoca, M., (2016). Campus employment as a high impact practice: Relationship to academic  

success and persistence of first-generation college students (Doctoral dissertation,  

Colorado State University, Fort Collins, Colorado). Retrieved  

from https://mountainscholar.org/handle/10217/173353