Running Head: 3-TIERED ASSESSMENT PLAN 1

3-Tiered Assessment Plan

Jessica Jett

Trevecca Nazarene University

IDT 5130

Interactive and Digital Tools for Instructional Design – Dr. Nicole Arrighi

March 21, 2021

3-TIERED ASSESSMENT PLAN 2

3-Tiered Assessment Plan

Formative Assessment

The Formative Assessment is a tool used by instructors or planning teams to evaluate the

effectiveness of a program in the development or “tryout” stages. A selection of one or two units

are used to complete the Formative Assessment in the interest of determining whether instruction

is effective early on, giving the instructors or planning teams time to modify or eliminate

weaknesses before full-scale implementation.

Formative Assessments benefit Instructional Designers by showing what is working for

their learners and what is not. It helps weed out unimportant information or assignments and

build up assignments that may be important, but have weakness. It can be used determine learner

engagement and thought processes regarding the training.

My project was on Excel Education for employees and new hires to the Health Science

Division at Columbia State Community College. When considering my Target Audience, I had

to take into consideration that many of the current faculty and staff members are not as tech savy

as others. Many of the not so Tech Savy faculty and staff members prefer to use Microsoft Word,

because they are more familiar with the program, even though the tasks they are completing in

Word are much easier in Excel. This is because it is a “new to them” program, and the odd look

when you open it (the cells), many of the no so Tech Savy members needed demystification.

When considering the other members of the faculty and staff, it becomes evident that many have

just never used Excel, or are not confident in their use of Excel, while there are still others who

are quite competent. The strategies I used to determine the needs of the Target Audience are my
3-TIERED ASSESSMENT PLAN 3

past experiences with them in regards to Excel, a short survey of current members of the faculty

and staff, and a discussion with the Dean regarding wide spread use of Excel.

I designed the gamification model only for new hires, but can easily update to include

current employees as well. The major assessment used in the gamification model is the personal

projects from the Dean. Those involve, updating, revising, or formatting currently in use

spreadsheets, giving the student real world experience. This also allows them to adjust to the way

spreadsheets are in use throughout the Health Sciences Division. The final project is an

assignment from the Dean and requires creation of a spreadsheet from the ground up. Originally,

I had planned to have the Dean grade the project, but after much thought and review, I decided

that should be a job for the Excel Education Task Force. I was quite interested in the eight-step

model for Formative Assessment, and I have included one in Appendix A of this assignment.

In Week 4 we created a sample of the education, in my case Excel Education. Though I

did not complete this assignment, due to some technical difficulties on my end (my personal

computer went up in smoke and I cannot download Articulate to my work computer due to

administrative issues), I did think a lot about this assignment and how I would complete it. My

desire was to work through the Excel ribbon, with each unit reviewing items on in a specific

ribbon section, for example, fonts, on the home ribbon or tables, on the insert ribbon. It would

mean many units, though each would be short and concise. For formative assessment, each unit

would end with an assignment related to that section of the ribbon. In fonts, that would include

font type, color, size, cell shading. Students test on their ability to change the shape, size and

color of fonts. During this Formative Assessment, I would also love to know how they are

feeling about the training in general. Since many of the target audience members will have some

familiarity with Excel and many will not, it needs to be engaging while informative.
3-TIERED ASSESSMENT PLAN 4

If learners are having issues with any particular topic, remediation can be gain through

Zoom conference with any member of the Health Science Division office staff, all of whom can

answer in detail how to complete a task in Excel and answer questions about what, where, when,

how, and why. There is also a Slack group dedicated to answering any questions learners have.

Remediation will be completed through social learning and tutoring.

The data collected from the Formative Assessment will be used to explore the Excel

Education program for weaknesses, omit redundant or useless information, and modify the

lessons for a more meaningful learning experience.

SEE APPENDIX A FOR THE FORMATIVE ASSESSMENT.

Summative Assessment

The Summative Assessment is an unbiased and objective approach to evaluate if a

program is achieving the expected outcomes. The Summative Assessment explores effectiveness,

efficiencies, cost, attitudes, and benefits.

The Summative Assessment was more difficult to pin down and may require further

research, on my part. I did not think that it was explained very well in the book and did my best

to come up with the following plan. If you know of a place where I might find further

information, I would greatly appreciate a link. I understand the reason for the Summative

Assessment and what it tells us, and the connection to Formative Assessment, but not the

planning. Some of the items just did not make sense and I was not able to find adequate

description of them to determine what I needed to cover (or maybe I did, I am not sure).

Seven Procedures for Determining Program Effectiveness

3-TIERED ASSESSMENT PLAN 5

1. Specify program objectives:

a. Excel Education is designed for use by current employees of Columbia State’s

Health Sciences Division, along with new hires. It is desired that all Health

Science employees be efficient in the use of Microsoft Excel, which is used

widely throughout the division for scheduling, data storage, evaluation, and other

administrative tasks. The purpose of this evaluation is to determine whether the

implemented resources are providing the desired training and understanding of

Microsoft Excel by Health Science Division employees, new and current.

2. Determining the evaluation for each objective:

a. Pretest-posttest with one group

i. One group will experience a pretest-posttest learning style within the

chosen units. They will be tested on their knowledge of the materials

before and after completion of the unit. This will help provide an idea of

the effectiveness of the learning.

b. One-group descriptive

i. One group will be asked to describe the training after completion. This

will help understand the attitudes, opinions, and interest of the learners

about the training.

c. Experimental-control group

i. Honestly, I am uncertain about this one. I searched out information but all

I can come up with is general control group information. I think this

means that one group will not be tested or asked for descriptions but I

cannot figure out what good that will do.

3-TIERED ASSESSMENT PLAN 6

d. Analysis of costs, resources, implementation

i. Costs are inconsequential as the Excel Education program will be using

already available resources. There are no additional costs for the program.

Columbia State already has LMS for the program, and it will be built by a

current member of the staff, when other projects are lacking. There is

some concern about reviewing the materials submitted by learners, but

will be worked out between members of the Health Sciences Division

office staff. All have down time at different times. Rewards, as mentioned

in the gamification model, will come directly from the division budget.

Most of the rewards are already on hand and were funded by private

donation.

ii. Resources that will be used are the time of the Health Sciences Division

office staff for grading, and remediation, as well as the time spent

designing. All of these items will be done in the downtime of the

aforementioned members of staff. Grading of projects completed by the

Excel Education Task Force, as mentioned in the Formative Assessment

plan, can also be completed in downtime, for which they are already being

paid. Again, gamification rewards will be used, but are already on hand by

private donation.

iii. Implementation may take some time, as the course is being designed in the

downtime of one employee. However, there will be no cost for

implementation and no need for resources to implement.

3. Developing data collection instruments and procedures for each objective:

3-TIERED ASSESSMENT PLAN 7

a. In the Formative Assessment plan, it is stated that evidence will be based on a

sample size of the first twenty students to complete the sample training. Each unit

will end with a questionnaire/survey, to gain an understanding to the students

attitude and thought processes throughout the current program.

b. Interviews of each student will be completed by the Excel Education Task Force

by written answer to protect student information. These interviews will give an

understanding of the students experience and outline any changes that may need

to be made to the program.

c. Observations will be made based on grades, projects, and time data. If students

are taking a long time to complete assignments they may be too difficult, or not

well explained.

d. Achievement tests will determine the effectiveness of the program. Achievement

tests will be designed by the Dean and graded by the Excel Education Task Force

as described in the Formative Assessment Plan.

4. Carrying out the evaluation:

a. Scheduling the data collection will be done once the twentieth student has

completed the course.

b. An email message to the Information Technology HelpDesk describing the data

need will be sent. Clerks will then be able to pull the data from the LMS and

provide a spreadsheet, which will need to be reviewed by the Excel Education

Task Force.

c. Other data, will be collected once each student completes the program. It will be

added to a spread sheet and stored until all twenty students complete the program.
3-TIERED ASSESSMENT PLAN 8

This will include information from questionnaires/surveys, interviews,

observations, and achievement tests.

5. Analyzing the results from each instrument:

a. The Excel Education Task Force will be responsible for analyzing the results from

each instrument and determining viability. Learner information will be redacted

from all results. There will not be any preferential treatment of results based on

the learner.

6. Interpreting the results:

a. Results will be interpreted by the Excel Education Task Force, and approved by

the Dean of the Health Sciences Division. Results will be redacted and not include

any identifying information. All students will remain anonymous.

b. Special care should be taken when interpreting the results. Outliers should be

determined and examined using standard deviation. Depending on the number of

outliers, the Excel Education Task Force may wish to remove them from the

results.

7. Disseminating the results and conclusions:

a. Evaluations reports will be distributed to the members of the Excel Education

Task Force once they have been approved by the Dean. Reports should include

graphs and diagrams along with a transcript of comments by students and Excel

Education Task Force Members.

b. The Excel Education Task Force will meet as a group at the end of the Summative

Assessment to discuss the results and conclusions of the assessment.

3-TIERED ASSESSMENT PLAN 9

c. Individual discussions may take place among the members of the Excel Education

Task Force or with the Dean, as needed.

Confirmative Assessment

Of the three assessments, the Confirmative Assessment is the one with which I am most

familiar. Each year, I for several of our programs, I create, send out, and collect results for

Graduate Surveys and Employer Surveys. This type of assessment is done after completion of a

learning experience, but at an extended length of time from the Summative Assessment, say six

months to one year. Confirmative Assessments are very important because they help determine

what is working and what is not, what the student remembers versus what they forgot.

Personally, I think that the employer surveys are more telling than the surveys sent to students.

The employer often feels no obligation to the school, educational experience, or program like the

student might. Students may also be angry about something and that may skew their responses.

Employers generally suffer no skewing. I also believe that some students are harder on

themselves than necessary and may feel like because they have not mastered a skill, they are not

efficient.

One of the things that I really like about Confirmative Assessment, whether from the

student or the employer, is that they give the ability to identify components that were helpful to

long-term success and those that were not. This helps determine if some units may end up

needing complete removal, while others might need modification to provide a more memorable

experience.
3-TIERED ASSESSMENT PLAN 10

For the Excel Education Program assessment will be continuous. Learner effectiveness

will be evaluated throughout their time in the program, with completion assessments and

ongoing assessments through assignments and a six-month follow up survey.

Assessment will occur in realistic contexts since the learner will be using Excel in the

real world, working with Health Science Division spreadsheets that are used throughout the

division. Their job may require daily use of Excel. Many of their projects will be sent directly to

the Dean for review before distribution to the rest of the division. This will allow the Dean not

only a chance to review the document for correctness, but also quality of employee work.

Since most employees will be using Excel daily performance over knowledge is

important. It is not enough to just know how to complete a task in Excel, Health Science

Division employees must be able to perform at a high level. Post education Dean evaluations and

the six-month survey will confirm that learners are able to perform at the necessary level.

The six-month survey will require students to review their own performance with Excel

and highlight areas that they feel need improvement. This will help determine if a specific area

of the program needs revision. For instance, if 40% of learners report trouble creating formulas,

we will know the formulas unit needs to be reviewed and modified.

The six-month survey may address new performance requirements since the course may

change over time. By surveying past students on those topics, we can see how the additional

resources may have affected the integrity of the program.

The sample size may change from the size of the sample used in the Formative and

Summative Assessments due to the original twenty students leaving the Health Sciences

Division. Indirect measures may be used to collect follow-up data.

3-TIERED ASSESSMENT PLAN 11

References

Morrison, G. R., Ross, S. M., Morrison, J. R., & Kalman, H. K. (2019). Designing effective

instruction. Hoboken, NJ: Wiley.

3-TIERED ASSESSMENT PLAN 12

Appendix A – Formative Assessment – 8-Step Model

Purpose

Excel Education is designed for use by current employees of Columbia State’s Health

Sciences Division, along with new hires. It is desired that all Health Science employees be

efficient in the use of Microsoft Excel, which is used widely throughout the division for

scheduling, data storage, evaluation, and other administrative tasks. The purpose of this

evaluation is to determine whether the implemented resources are providing the desired training

and understanding of Microsoft Excel by Health Science Division employees, new and current.

Audience

Evaluation results will be shared with the Excel Education Task Force. The task force is a

group consisting of faculty and administrative support. They have been tasked with overseeing

the Excel Education and reviewing the effectiveness of the implemented training.

Issues

As with all educational resources, questions arise regarding effectiveness of the materials.

With Excel Education, in particular, there is interest to know the gains and attitudes of learners,

whether they are current employees or new hires to the Health Sciences Division.

 After receiving instruction, are current and new employees better able to use basic

formatting techniques in Excel (for example: font selection, size, color, shading,

alignment, direction, wrap, merge and center.)

 Does the ribbon section/unit approach work? (Each unit is based on a section of the

ribbon. For example, unit “Home  Font” or “Home  Alignment”.)

3-TIERED ASSESSMENT PLAN 13

 On average, how long does it take to complete each instructional unit?

 Are learners eager to move to the next unit?

 Do learners find the hands-on lessons helpful?

Resources

To evaluate the effectiveness of the program, learners will need to be evaluated in several

ways. They will need to be tested, surveyed, and data will need to be collected. A list of needed

resources follows:

 Learners/students

 Instructors

 Access to data regarding timing of instructional units

 Access to data regarding interval between units

 Survey of students who have completed the program

Evidence

The sample size will be the first twenty students to complete the program. Testing of

students will be completed by assigning a real project that will be used throughout the Health

Sciences Division. This assignment would be basic formatting of a currently in use Excel sheet.

Objectivity of the information will be determined by outliers and standard deviation.

Data Gathering Techniques

Data will be gathered from each student through a performance test, graded by the Excel

Education Task Force, observations on the time taken to complete the units, and questionnaires

after completion of the units. The Performance test will not be a simulation. It will be a
3-TIERED ASSESSMENT PLAN 14

personalized project, assigned by the Dean to each individual, geared toward their specialty.

Once the Dean has approved the project, it will be in use throughout the Health Sciences

Division. Data regarding timing can be retrieved from the LMS and used to assess the average

amount of time taken by each student and the interval between units. A questionnaire regarding

the student’s experiences with the program will be built into the program as the last assignment.

Analysis

Two randomly chosen members of the Excel Education Task Force will grade each

completed personal project to guarantee reliability of the data. Validation of responses will be

discussed and determined by the entire Task force. If determination is not resolved, the learner

will be asked for clarification of response. Review of the program will be completed and scored

by each member of the Task Force. They will be asked to review for length, theme, readability,

relevance, and engagement of each unit.

Results will be in the form of graphs and diagrams along with a transcript of learner

responses to open ended questions and comments.

Reporting

The report will follow the below format:

3-TIERED ASSESSMENT PLAN 15

I. Executive Summary
II. Purposes of evaluations
A. Evaluation objectives
B. Description of target course
III. Methodology
A. Participants
B. Instruments
IV. Results
A. Analyses
B. Findings
V. Conclusions and recommendations