Reviewer Responsibilities & Instructions Loft 2.

This file will try to provide clear instructions on how to be an effective reviewer and what your
responsibilities will be.

How to Start Reviewing

● To start reviewing, your PM should assign your SDO account to the review pool.

● Then once you open the DataCompute system, a completed task will appear on your screen
randomly and automatically.

● You should park it immediately and start quality-checking. It is highly recommended to start
writing your feedback during task check rather than preparing it afterwards.

● As we have no “rework” option, the reviewer should fix the errors and make it ready for the
delivery. So providing feedback and training the transcribers is now much more significant.

How to Provide Feedback

● Each task (except for the valid_skip cases, you can see below), regardless of how good or bad it
is, must have feedback entered in the Feedback tab. This is one of the most significant
responsibilities of a reviewer.

● After completing the quality-check of an audio file, the reviewer should open the WTA Live and
go to the “Feedback” tab.

● The reviewer should fill out only the following sections:

○ Question ID (Review Pool): This is the ID of the task that appears in the review pool.
○ Reviewer’s Feedback: This is where the reviewer tells the transcriber about her/his mistakes
in the audio file and how to avoid them in the future.
○ Quality Rate: Quality rate is the score that the reviewer sees fit for the task based on the
quality rate scheme in the “RawData” tab. Every score should be given very cautiously as
this will be very decisive about the transcriber’s position within the team.
How to Use “Skipped/Rejected Tasks” Section

This section is for a reviewer to enter the question ID of a task that has been submitted “empty” for a
reason by the transcriber. There can be two reasons for submitting a task without any work: Valid_Skip
and Rejected_Task.

❖ Valid_Skip Cases: It refers to the tasks which should be skipped and should not be worked on. This
kind of tasks do not get paid either to the transcriber or the reviewer.

● What to do in WTA: The system asks workers to “skip” some tasks based on some reasons like
the following: No Audio, No Sound, Other Locale, Silent Audio, Noisy Audio, Other. In these
cases, the reviewer should enter the question ID and choose “Valid_Skip” for the Skip Category
and then choose the relevant reason in the next menu, which is Reason for Rejection/Skip. This
task should NOT be entered to the Feedback tab because it’d affect the overall average of a
transcriber.

● What to do in Loft 2.0: In this case, the reviewer should also submit the task by choosing the
relevant skip reason in the Loft 2.0 system.

❖ Rejected_Task Cases: It refers to the tasks that should have been transcribed properly but the
transcriber sent it empty by mistake OR with no proper work.

● What to do in WTA: If the task does not fall under the categories specified in the guidelines
(meaning that the task is supposed to be transcribed normally), then the reviewer should choose
“Rejected_Task” for the Skip Category and then choose the “Submitted by Mistake” option
for the Reason for Rejection/Skip section. This kind of tasks should be entered to the Feedback
tab as well and rated as one (1) because it is a high-level mistake by the transcriber.

● What to do in Loft 2.0: If the task is submitted by mistake, this task should be transcribed by
the reviewer immediately and submitted afterwards. “Task Completed Properly” can be marked
as “Yes”.

How to Use “20+ Speaker Tasks” Section

Whenever there is a task with more than 20 speakers, the responsible reviewer should enter the task ID
(column A) and the new length (Column B) here. Payment will be arranged accordingly.

The new length format to be placed in Column B should be as the following: 00.00 (hour-format should
be converted into the decimal format.

Example: If the transcription stopped at 16:30:00 (this is hour format) in the task itself, then the
reviewer should type 16.50 (this is decimal format). If it stops at 16:45:00, then it should be 16.75 as
the new length.
We have received clear instructions from the client on two very specific points regarding how to
use parking and pinning features on Loft 2.0 platform.

1 .Going forward, you are highly encouraged to use ONLY the PARKING feature to hold a
task, not the Pinning one until any further announcement.

2. If any of your resources is using more than one Google Chrome tab to transcribe,
please give them specific instructions NOT to do that. The audio files should be
transcribed using only one Google Chrome tab open.

The reasoning behind this is that using the tool in this way leads to a ton of different possible
errors with regards to questions being over-replicated, over-assigned, and inappropriately
duplicated, which has other downstream impacts (specifically on reporting).