UHRS Web Entry Instructions

Updated December 31, 2013

1. Required – Sign up for a Windows Live ID if you do not already have one.

4 l
 Appen will create your user account on UHRS using the Windows Live ID with same email that
you used for your contract.

60 tia
 If you currently do not have a Windows Live ID account associated with that email address, sign
up for one by going to https://signup.live.com/ . (Please note that you do not need to sign up for
any other Microsoft service like Hotmail to get a Windows Live ID). Remember to create your
Windows Live ID with the email you provided in your contract under the Windows Live ID
section.

2. Using Internet Explorer only, go to the UHRS Web Entry Link

https://prod.uhrs.playmsn.com/Judge/Views/Login.aspx

37 n
3.

e Bookmark or make this address a Favorite (Ctrl+D)

17 id
4. Click Sign in with LiveID and enter your username and password. Your WLID will be formatted like an
nf

email address (for example, johnsmith@internet.com)

 If you do not recall your WLID password, you can reset it yourself at
https://account.live.com/ResetPassword.aspx
 If you any difficulty logging in, please send an email with complete details, including the
username and password you are using along with a screenshot to crowdhelp@appen.com
Co

5. Click on Marketplace.
The page is separated into two groupings. On the left is Favorites, once you work on a task, it will appear
here in the future when you log in. The right is All HitApps, in which you can pick and choose what to
work on.

Appen www.appen.com

All materials copyright Appen Butler Hill and confidential. Do not forward.
 6. Click on View Guidelines, Preview, and then Start Judging.

Be sure to review the guidelines for any crowdsourcing tasks. You will be evaluated on the quality of your

4 l
ratings, and judges who do not comply with project guidelines may no longer be considered eligible to
work on crowdsourcing tasks. For complete details, refer to your contract and Statement of Work.

60 tia
37 n
7.

8.
e
Some tasks may require a qualification test. You must successfully pass the Qualification Test before the
Start Judging link will be active.

Start judging! The ratings pane and the landing page may appear side by side, or stacked on top of each
17 id
other as in the examples below.

Select the rating for the page, click Submit, and the next page should be loaded automatically.
nf
Co

9. Make sure that you are not clicking the Skip icon. This will advance to the next query, and your rating
will not be counted.

Appen www.appen.com

All materials copyright Appen Butler Hill and confidential. Do not forward.
 10. When you are done rating, be sure to Logout by clicking the small, gray arrow on the ratings pane, and
then click Logout on the dashboard.

4 l
60 tia
37 ne
11. Click on My Report to view a summary of judgments completed and earning totals.
17 id
nf
Co

The ‘My Report’ feature is very useful for keeping track of HitApps, valid judgments completed, and earning
totals. Reports can be viewed by day (previous day, today, and next day) or weekly (previous week, this week,
or next week). The totals do not accumulate and it is recommended to keep a weekly log of completed
judgments and earnings for your own records.

Appen www.appen.com

All materials copyright Appen Butler Hill and confidential. Do not forward.
 General Help and Assistance

4 l
Send questions to crowdhelp@appen.com . Minimal technical support is available. As independent contractors,
you are expected to maintain and utilize your own equipment and resources to complete the tasks.

60 tia
No assistance will be provided related to the judging guidelines or annotation questions.

If there are technical issues with a task, such as queries or data showing up incorrectly, please send a detailed
report so we may investigate. Make sure to include the following information:

 Windows Live ID
 Market

37 n




e Name of HITapp or task
Screenshot of error message or issue

All of our escalations are addressed in order of severity:

 Issues affecting the entire tool/all rosters.

Issues affecting one task or HITapp
17 id
 Issues affecting one judge

Crowdhelp Service Hours are 8 am – 5 pm Pacific Time (Seattle). You may log in to the tool on weekends, but be
aware that no support is available.
nf
Co

Appen www.appen.com

All materials copyright Appen Butler Hill and confidential. Do not forward.