Developing a Scoring Criteria (Rubrics)

DISCLAIMER: This data in this section is fictitious and does not, in any way,
represent any of the programs at Gallaudet University. This information is intended
only as examples.
TYPES OF SCORING CRITERIA (RUBRICS)
A rubric is a scoring guide used to assess performance against a set of criteria. At a
minimum, it is a list of the components you are looking for when you evaluate an
assignment. At its most advanced, it is a tool that divides an assignment into its
component parts, and provides explicit expectations of acceptable and unacceptable
levels of performance for each component.
Types of Rubrics
1 - Checklists, the least complex form of scoring system, are simple lists indicating
the presence, NOT the quality, of the elements. Therefore, checklists are NOT 3 - Holistic Rating Scales use a short narrative of characteristics to award a single
frequently used in higher education for program-level assessment. But faculty may scored based on an overall impression of a student's performance on a task. A
find them useful for scoring and giving feedback on minor student assignments or drawback to using holistic rating scales is that they do not provide specific areas of
practice/drafts of assignments. strengths and weaknesses and therefore are less useful to help you focus your
Example 1: Critical Thinking Checklist improvement efforts.
The student… Use a holistic rating scale when the projects to be assessed will vary greatly (e.g.,
__ Accurately interprets evidence, statements, graphics, questions, etc. independent study projects submitted in a capstone course) or when the number of
__ Identifies the salient arguments (reasons and claims) assignments to be assessed is significant (e.g., reviewing all the essays from
__ Offers analyzes and evaluates major alternative points of view applicants to determine who will need developmental courses).
__ Draws warranted, judicious, non-fallacious conclusions
__ Justifies key results and procedures, explains assumptions and reasons
__ Fair-mindedly follows where evidence and reasons lead
Example 2: Presentation Checklist
The student…
__ engaged audience
__ used an academic or consultative ASL register
__ used adequate ASL syntactic and semantic features
__ cited references adequately in ASL
__ stayed within allotted time
__ managed PowerPoint presentation technology smoothly
2 - Basic Rating Scales are checklists of criteria that evaluate the quality of elements
and include a scoring system. The main drawback with rating scales is that the
meaning of the numeric ratings can be vague. Without descriptors for the ratings, the
The Holistic Critical Thinking Scoring Rubric: A Tool for Developing and Evaluating
raters must make a judgment based on their perception of the meanings of the
Critical Thinking. Retrieved April 12, 2010 from Insight Assessment.
terms. For the same presentation, one rater might think a student rated “good” and
4 - Analytic Rating Scales are rubrics that include explicit performance
another rater might feel the same student was "marginal."
expectations for each possible rating, for each criterion. Analytic rating scales are
especially appropriate for complex learning tasks with multiple criteria.
Evaluate carefully whether this the most appropriate tool for your assessment needs.
They can provide more detailed feedback on student performance; more consistent
scoring among raters but the disadvantage is that they can be time-consuming to
develop and apply.

Results can be aggregated to provide detailed information on strengths and

weaknesses of a program.

Example: Critical Thinking Portion of the Gallaudet University Rubric for Assessing
Written English

STEPS FOR CREATING AN ANALYTIC RATING SCALE (RUBRIC) FROM

SCRATCH

There are different ways to approach building an analytic rating scale: logical or
organic. For both the logical and the organic model, steps 1-3 are the same.

Steps 1 – 3: Logical AND Organic Method

Determine the Best Tool

1. Identify what is being assessed, (e.g., ability to apply theory) as this is

focused on program-level learning assessment. Determine first whether
an analytic rating scale is the most appropriate way of scoring the
performance and/or product. An analytic rating scale is probably a good
choice
o if there are multiple aspects of the product or process to be
considered
o if a basic rating scale or holistic rating scale cannot provide the
breadth of assessment you need.
Building the Shell
The Rows

2. Identify what is being assessed. (e.g., ability to apply theory).

o Specify the skills, knowledge, and/or behaviors that you will be
looking for.
 o Limit the characteristics to those that are most important to the
assessment.
Other possible descriptors include:
 Exemplary, Proficient, Marginal, Unacceptable
 Advanced, High, Intermediate, Novice
 Beginning, Developing, Accomplished, Exemplary
 Outstanding, Good, Satisfactory, Unsatisfactory

Step 4:

Writing the Performance Descriptors in the Cells

examples of inconsistent performance characteristics and suggested corrections.

4. Use either the logical or the organic method to write the descriptions for
each criterion at each level of mastery.
The Columns
3. Develop a rating scale with the levels of mastery that is meaningful.
Tip: Adding numbers to the ratings can make scoring easier. However, if you plan to
also use the rating scale for course-level assessment grading as well, a meaning
must be attached to that score. For example, what is the minimum score that would
be considered acceptable for a “C.”
Example:
Components of Analytic Rating Scales
 Criteria that link to the relevant learning objectives
 Rating scale that distinguishes between levels of mastery
 Descriptions that clarify the meaning of each criterion, at each level of
mastery
 Use rating scales for program-level assessment to see trends in strengths and
weaknesses of groups of students.
Examples
 To evaluate a holistic project (e.g., theses, exhibitions, research project)
in capstone course that pulls together all that students have learned in the
program.
 Supervisors might use a rating scale developed by the program to
evaluate the field experience of students and provide the feedback to
both the student and the program.
 Aggregate the scores of rating scale used to evaluate a course-level
assignment. For example, the Biology department decides to develop a
rating scale to evaluate students' reports from 300- and 400-level
sections. The professors use the scores to help determine the students’
grades and provide students with feedback for improvement. The scores
are also given to the department’s Assessment Coordinator to summarize
to determine how well they are meeting their student learning outcome,
"Make appropriate inferences and deductions from biological information."

Steps 5-6: Logical AND Organic Methods

5. Part 6. Scoring Rubric Group Orientation and Calibration” for directions

for this process.
6. Review and revise.

STEPS FOR ADAPTING AN EXISTING ANALYTIC RATING SCALE (RUBRIC)

1. Evaluate the rating scale. Ask yourself:

o Does the rating scale relate to all or most the outcome(s) I
need to assess?
o Does it address anything extraneous?
2. Adjust the rating scale to suit your specific needs.
o Add missing criteria
o Delete extraneous criteria
o Adapt the rating scale
o Edit the performance descriptors
3. Test the rating scale.
4. Review and revise again, if necessary.

USES OF RATING SCALES (RUBRICS)