Grading Handbook

Version 1, June 2021

Educator Training & Development

THINKFUL, INC – A CHEGG SERVICE
Table of Contents
Introduction ............................................................................................................. 2
The Grading Process ................................................................................................ 2
Steps to a Typical Grading Session ................................................................................. 4
Chrome Extension ........................................................................................................... 8
Markdown ........................................................................................................................ 8
Effective Feedback .................................................................................................. 11
Working with Students ........................................................................................... 16
Common Concerns When Working with Students ........................................................ 19
Grading Disputes ........................................................................................................... 20
Performance Metrics for Graders .......................................................................... 21
Grader Handbook Acknowledgement .................................................................... 21
Introduction
Grading is a deeply impactful format that serves as one of the most direct instructional
resources for our students. As a Grader, you’ll have a unique perspective on student progress and goals
as you’ll help to shape their actual deliverables into professional-level portfolio items that they’ll take with
them on their career search at the end of the program.

In this handbook, you'll learn how to:

• Access the Grading Queue and filter, claim, and access the correct assignments
• Use the Grading Queue to access other key areas of the Thinkful Platform
• Offer objective, rubric- and curriculum-based feedback to students
• Escalate concerns to the correct groups
• Identify key requirements for each assignment in your field

The Grading Process

Expectations for Graders:
• Reference the provided rubrics to offer instructional, observational feedback on
student assignments
• Offer opinion-based suggestions as “Optional Changes” that would help a student to
succeed in the workplace
• Provide feedback on assignments within 24 hours of submission
• Refer students to other formats or staff members as directed
As a general rule, follow the steps below to complete a successful grading session.
The amount of time you should expect to spend on each assignment will vary
according to the assignment, the quality of the work, and the amount of support the
student requires. That said, billable time is only the time you spend committed to
assessing the assignment itself. You are welcome to contact the student directly for
clarification or to track down missing items, but this is not considered billable grading
time. If you have questions or concerns about how a typical billed grading session
should go, contact educator.support@thinkful.com
Steps to a Typical Grading Session

Step 1: Go to the grading queue by clicking on this link: Grading Queue

The following window will open:

Step 2: Under the Select Program filter, select the course you need to grade the
submission for.

For example, to grade a Data Analytics, submission, apply the filters as shown below:

Step 3: Select the assignment you want to grade by clicking on

the Claim button. The platform will un-claim your checkpoint after 2 hours.
Step 4: Once you claim the assignment the student submission along-with the rubric
opens, as shown below (if it does not, simply add /rubric to the end of the URL
address):

Scroll down the page to get the student submission, which can be found under Your
work. (Highlighted in the screenshot below).
 • Open the link to student submission in a new tab, check if the student's work
matches the criteria mentioned in the rubric.
• Evaluate the level of completion by answering the “Yes/No” questions
• Answer the subjective remarks questions, using the following guidelines:
o DO NOT
Use “I like / don’t like…” in your remarks
Offer suggestions that are not reflected in the rubric
Reverse a previous graders “Passing” remark (if an item passed
previously, it must pass this time as well)
o DO
Identify the missing rubric item
Direct the student to the location where the missing item should
be located
Offer guidance on how to implement (if applicable)
Point to the curriculum item that teaches the missing / developing
skill

Example:

[* Socials *] Per the checklist, please include links to your LinkedIn and any other social media
you feel comfortable sharing.

[* Alt text *] Per the checklist, all images should have meaningful alt text. The alternative text
for an image must reflect the purpose of the image and not contain the image file name. (The
problem often occurs with auto-generated alternative texts). The alternative text is used by a
screen reader to read aloud the purpose of the image for a visually impaired user. A file name is
rarely of use, as everything is read out, including underscore and slash. Make sure the alternative
text reflects the purpose of the image. I use the Chrome plugin ‘Alt Text Tester’ to check my
work.

In the example above, the grader successfully identifies the rubric/checklist item that
needs improvement, defines the need, and offers instructional guidance and context
to help the student to understand the missing component in an educational way.

Remember, these are students, not professionals. Thinkful’s curricula scaffold the
student experience to ensure mastery. You cannot and should not expect
professional-level deliverables at every stage. You can, however, work closely with
the rubric to ensure that students are ready for your professional sphere by the end
of their program!
Step 5: Once you are done with the rubric, select the final decision as
either Approve or Request Revision. (Refer the screenshot below)

Step 6: On completion of the entire rubric scroll to the bottom of the page and click
on Submit Feedback button. You cannot edit feedback once you’ve clicked submit.

Notes:

You can claim an assignment for a maximum duration of 2 hours.

Claim only one checkpoint at a time.
Checkpoints should be graded within 24 hours of submission.
Grade the oldest checkpoints in the queue first.
Chrome Extension
For more efficient student communications in your grading feedback, we recommend that
you install our Chrome extension by following the steps at the GitHub repo found below.

https://github.com/Thinkful-Ed/grading-chrome-extension

This extension auto-populates a canned grader introduction using personalized messaging

for the student. This allows you to focus more time on grading and less on checking your
spelling a student greeting.

Markdown
We use markdown in the text field where you'll submit your grading feedback. If you’ve never
used it, it’s easy. You can be a minimalist or take it to the next level. It’s up to you. Use this
extensive Markdown Cheatsheet or these basics:

Markdown Input
Rendered Output
Effective Feedback
Think of a time that you received feedback.
It may have been in a school setting, at work, or even at home from a partner or parent. How
did it feel to receive that feedback? Did the feedback meet your expectations? Did the
provider tell you what you should do or did they describe what you did? Did they tell you
that what you did was good or bad? Did you walk away feeling prepared to act on that
feedback?

MANAGING EXPECTATIONS
When we ask someone to give us feedback, we often have expectations for what that
feedback might look like. Generally, we expect the provider to tell us what we did well, and
what we should do differently.
This rings especially true for students. If students are in need of feedback, it is because they
do not know how to perform according to standards and are in need of guidance.
As an Educator, your feedback is absolutely invaluable to students. Your guidance and
support is what helps lead them to their ultimate success in their program!
As you progress through this module, keep it in mind that feedback should be:
• specific - targeted to a clearly defined context and set of behaviors
• timely - delivered immediately following the noted behavior
• meaningful - framed around adjustable behaviors that apply to the recipient's role
• candid - presented openly and honestly, directly to the recipient

When we help someone with a problem, that help generally comes in one (or more!) of three
modes:
Evaluation
Observation
Advice
EVALUATION
An evaluation of someone's efforts or work generally comes with an opinion: it's good, it's
bad; looks great, what a mess.
Evaluations tend to be somewhat subjective and require an understanding that the provider's
expectations are more important than the receiver's. Evaluations should only take place
when those expectations have been previously made clear.
Evaluations should typically be combined with other types of feedback in order to provide
the receiver with a clear pathway to success.

OBSERVATION
An observation of someone's efforts or work is an objective reflection from an outside
perspective that the receiver could not have of their own work.
Observations are the most objective approach to feedback, allowing the receiver to make
their own decisions about quality and outcomes, thereby taking the emotion out of
feedback.
Observations are the most effective type of feedback for those trying to correct a behavior
in another person. If someone has not specifically requested the feedback, an observation-
based approach is typically the most effective type of feedback to offer.
HINT: This approach to feedback is what we will encourage you to use the most!

ADVICE
Advice is a form of feedback that focuses more on the future than on the behavior that
currently exists.

Advice is almost entirely opinion-based, whether it feels that way or not. Anytime a statement
begins with "I would..." or "You should..." (or any other modal verbs, for that matter!), the
content that follows is generally subjective. Even if it is not meant as subjective, it will likely
be interpreted that way!

Advice is the best type of feedback for someone who knows what their problem is and is
struggling to determine a solution. In most cases, it's best to reserve advice only for those
who are asking for it.

Different types of feedback will have different types of impact. So, it's important to consider
when and where it is best to offer advice, observations, and evaluations.
Timing is always critical with feedback. Any feedback, whether positive or negative, an
observation or an evaluation, should be delivered as soon as possible. Feedback is the result
of a specific set of behaviors -- whatever was done to elicit the feedback in the first place --
meaning it should be given within the context of those behaviors. When the context is
removed and/or too much time has passed since the behaviors took place, the feedback loses
its meaning and ends up presenting more like criticism or a grudge.
To that point, regardless of which type of feedback you're offering, your delivery matters.
DELIVERY MATTERS
Remember back to your most recent feedback again. How did you walk away feeling? Like
you were ready to make a change based on that feedback? Like you weren't good at
anything? Like the feedback had no impact? These feelings typically have less to do with
what the feedback was, and a lot to do with how it was presented.

Much to the demise of popular belief, the "feedback sandwich" is not meant to soften the
blow of bad news. Instead, it's meant to present a foundation on which to build improvement.
If the recipient of the feedback knows what's working, they'll know which behaviors to
continue and which to stop. So, instead of thinking of the "feedback sandwich" as "positive
bread - negative meat - positive bread", think of the bread as the support structure in which
to hold the necessary changes:

Start with what's working, not just what is positive. Truth be told, you may not even agree
with the methods someone is using, so you may feel negatively about it. But, if it's working,
then they should keep doing it.

Then, move to what's broken or unsuccessful. Which specific elements are causing them to
not meet your expectations? We'll talk about observation-based language in the next section,
but think about what you actually see happening that is causing their lack of success. Most
students are not yet familiar with on-the-job practices or workplace culture in your industry.
Connecting the desired outcome to on-the-job/real life needs weaves mentorship into
Grading.
Finally, rather than buffer or discredit your feedback with a fluffy positive finale, use their
successes to guide their next steps. What, along the lines what's working, can they do next
to be more successful?

Earlier, we covered the differences between evaluative feedback, observational feedback,

and advice. In your role, observational feedback will almost always be your best approach.
So, let's take a closer look at how observational feedback operates when used successfully.

Take a look at this image:

What do you see? Without passing any judgement on how you feel about what you're looking
at, and without considering any experience that might tell you what the cause/effect of the
actions pictured are, describe only what you see.

An experience-based judgment about the image above might include one of the following
statements:
• the dogs are playing
• the dog is scared
• one dog is attacking the other
• the dog is reactive / mean / aggressive
While an observation of the image above would, instead, include statements such as:
• there is a black dog and an orange dog
• the orange dog's mouth is open
• the orange dog is showing motion toward the black dog
• the black dog's eyes are wide
Observational feedback needs to have a certain level of separation from personal
experience to be successful. At Thinkful, we expect Educators to provide observations rather
than judgements when providing feedback.

Instead of: You did not put enough effort into this assignment ...
Try: You included abc in your assignment, but did not include xyz ...

Instead of: You clearly don't care about this course ...
Try: You have missed several sessions and assignment submissions this month ...

The key is to use language that describes only the behavior that you are seeing, not your
opinion or interpretation of what that behavior might mean.
When you find yourself needing to have a difficult conversation with a student, use the
following template to help you guide the conversation.

Remember: you're the teacher in the room, regardless of whether you're a Mentor, and
Instructor, a Grader, or any other Educator at Thinkful; students need to hear tough news
from you, and they need you to take the lead. When we don’t hold students to the
assignment’s standards we set them up for more work later in the course or, worse yet,
struggles in their job search.
Working with Students
As noted in previous sections, grading is a unique format that puts you in a position of both
coach and judge. You are determining a student’s readiness to proceed, while also offering
guidance on how to get to that level of readiness. Sometimes, this can create a feedback loop
that can leave the student feeling stuck and frustrated. In this section, we offer guidance to
A) avoid this loop as much as possible and B) address it successfully when it does happen.

Tone

The first step in avoiding frustrating feedback loops is to be aware of your tone when offering
feedback in the first place. At Chegg, we are student first – meaning we utilize a growth
mindset at every stage. If you open a checkpoint and do not feel that this student has any
hope of succeeding in your professional sphere, you should probably close the assignment
and unclaim the checkpoint. Instead, you should go into each checkpoint believing that this
student will shine in your field someday.
Words to live by:

grow / growing / growth

progress / progressing
improve / improving
develop / developing
practice

Words to avoid:

enough / not enough (as in there’s not enough…”)

can’t / shouldn't (as in “you shouldn’t use…”)
always / never (as in “never put…”)
even (as in “you didn’t even…”)
Teamwork
You’re not alone out there, and it’s critical that you involve your other team members early
and often to help support you, the student, and your colleagues throughout a student’s
program. If you have concerns about the student’s work, another grader’s feedback, advice
another Educator gave a student, or anything else that you might see come through the
grading queue, tell someone!
Your resources include:
• Your Educator Success Management team: educator.support@thinkful.com
• The Student’s Academic Success team: success@thinkful.com
• Your fellow Graders: Slack
• Your Training & Development team: educator.training@thinkful.com

Timing
Take breaks, my friend! Burnout is a genuine concern for all Graders. As a Grader, you will
often see the same mistakes being made by students over and over. This is a part of their
growth and development as a professional and should not be taken as a personal attack on
you, your knowledge, your communication, or any other offering that you’ve given to the
student’s progress. Instead of pushing through hours and hours of grading back-to-back,
allow yourself breaks to walk away from the screen and look at something else so that you
can come to student work with fresh eyes every time.

As a general rule, we recommend that you spend no more than 3 hours grading at a time to
avoid burnout and to keep yourself fresh. If you choose to dedicate more than 3 hours at a
time to grading, take 10-15 minute breaks away from Thinkful for every 90 minutes of work
to allow your brain to reset between checkpoints.
Common Concerns When Working with Students

The Student's Work is Ungradable

Occasionally, a student will submit material to be graded that contains locked files, is missing
large portions of the assignment, or is otherwise not ready to be graded. In these instances,
you should still provide the student with clear feedback that includes a clear statement of
what they must do to fix the problem:

"You are missing sections 3 and 4, so we will wait to grade until those are included. Please
resubmit."
or
"The file is locked. Please unlock and resubmit for grading."

A Student is Really Struggling

If the student has tried ThinkChat and Office Hours sessions, and just isn’t getting something,
feel free to "Add a Note" to their file using the Lark Student Search
at lark.thinkful.com/student-search or by going to their Dashboard, then Activity page.
Further, this is a good time to reach out to the student's ASM to give them a head's up.

A Student Contacts You Directly

Sometimes a student reaches out to a grader via email or Slack and wants to talk about
feedback. We discourage this because conversations are better handled with grading
submissions and feedback, but there are times when you can clarify a simple thing quickly.
Just know that having an extensive conversation outside the grading format usually proves
counter-productive and encourages more of the same in the future, which is even more
counter-productive. You might respond to outreach with something like,

“I’m excited to see you’re reaching out for more feedback. I’d recommend talking with your
mentor, using ThinkChat, or visiting with an educator in Office Hours. Someone in those
formats will be able to answer more quickly and comprehensively than I can right now.
Thanks!”

Abusive or Inappropriate Behavior

If you see a student being rude or inappropriate towards graders or anyone, report it to the
Academic Success Team by emailing success@thinkful.com. We ask students to treat this like
a job and that means being professional and respectful.
Suspected Cheating
It’s important to keep in mind that there’s almost always more context to a situation so you
don’t want to jump to conclusions. Students new to fields like development and design may
not realize that copying code or designs can be viewed as cheating. That said, you need to
raise a flag and here’s how

• Send an email to Academic.affairs@chegg.com with [Cheating] in the subject line and

they'll conduct a full investigation.
• Do not alert, involve or email the student yourself.
• Provide as much detail as possible including links to checkpoints and assignments.

Remember that synchronous engineering students sometimes work in pairs or groups. They
should indicate this in their ReadMe file and they should receive identical grader feedback.

Grading Disputes
At Thinkful, we collectively assess several thousand human-graded checkpoints every week
(this is in addition to our Live Assessments and auto-graded checkpoints!). Inevitably, we
expect to have a handful of errors and a handful of unsatisfied students. In response to this,
we have a Grading Dispute protocol that allows students to contest their grading feedback
and get a second set of eyes to take a look.

The goal of this process is never to “get a grader in trouble,” but rather to help us identify
inconsistencies in presentation and/or tooling that graders use.
If your feedback ever comes into dispute, don’t sweat it. Your internal teams will work directly
with the student, regardless of the outcome of the dispute. If we do come to the conclusion
that there’s an area in which you can improve your grading feedback, you’ll hear from us
directly in a way designed to help you continue your own development.
Performance Metrics for Graders
We're all experts in our fields and passionate about helping others learn, but working in
education is new to most of us. To help you learn how you're doing and where you can grow,
we have educator evaluations. The evaluation metrics for grading are:

FEEDBACK CONTEXT
Grader refers to real- Grader refers to the
Grade justification(s) world contexts or curriculum or other
is/are based solely on examples as support Thinkful content to
the contents of the for rubric-based drive student toward
checkpoint rubric grading feedback a clear outcome
FEEDBACK DELIVERY
Feedback Delivery Grader uses
[Grader feedback is observation-based
relevant to student language as
work and curriculum, demonstrated by
as demonstrated by Grader offers similar statements such as,
references made to feedback across "Your work contains
actual items in the Grader feedback is submissions of similar xyz and the result of
student's submission clear and concise quality that is abc"

Grader Handbook Acknowledgement

Click the link below to acknowledge your receipt and agreement to the terms and
expectations outlined for you in this handbook.

I AGREE TO THE TERMS AND EXPECTATIONS OUTLINED IN THIS HANDBOOK