Selectively Anonymous Rankings: Design, Analysis
and Impact on Computer Science Students
Simon Spacey
Computer Science Department
Waikato University
Hamilton, New Zealand
Email: sspacey@waikato.ac.nz
Abstract—This paper presents a method to deliver course
rankings that maintain student confidentiality while still allowing
students to selectively prove their position in a class to others if
they wish. The method’s selective anonymity is implemented
through a secure hashing algorithm that is designed to protect
student privacy even where a student’s name, their student ID,
their project teams and the student’s ranks for other work items
are known. The paper includes results showing student
perceptions of the approach and the impact on their performance
for a second year Computer Science course at the University of
Waikato in New Zealand.
Keywords—Student Privacy; Grade Rankings; Course Hashes
I. INTRODUCTION
O
NCE upon a time, there was a set of students who
lived in a land without mobile devices, free internet
and social networking sites. Each semester the
students took part in a tournament organised by an ivory tower
and their names and results were released as rankings on a
large notice board in a public arena for all to see. The rankings
were respected as being a fair and transparent indicator of
achievement in the field and were far preferred to the approach
of the neighbouring kingdom where every competitor was
simply given a letter grade of A for “A”ttendance which the
people criticised as not rewarding individual effort and hiding
information from future patrons.
But then one day there came a magical wind of change that
connected all the students through mobile devices, free internet
and social networking sites. The change brought great
opportunity for the students but also great risks given the
traditional named ranking system. In particular some were
concerned that evil mage would misuse the technology to, for
example, anonymously cyberbully students both high and low
in the rankings and so parliamentary bills were proposed [1] to
change the laws of the land [2], [3], [4], [5] to govern the use of
the new technology and the ivory towers, whose councils were
explicitly charged with the well-being of their students [6],
introduced guidelines for dealing with the new technology [7]
and changed their approach to the tournaments to publish ranks
by student ID rather than by student name.
Unfortunately, the Student ID ranking approach cast a
cloud over the land with:
978-1-4799-7672-0/14/$31.00 ©2014 IEEE
2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 198
1) students no longer able to use the rankings to
demonstrate their position to others,
2) the ivory towers unable to award student prizes without
breaking the anonymity of the system and with
3) the ranking of small groups jeopardised by the peril of
name identification through grade correlation
and so a quest for a new ranking approach was championed by
the ivory towers.
This paper details the result of our quest to find a way to
provide student rankings that removes the disadvantages of the
Student ID ranking system itemised above while still providing
students with the protection of rank anonymity. We begin in
Section II by providing an overview of alternate methods that
deliver various levels of student ranking privacy and examining
their benefits and issues. In Section III we then provide
technical details of our approach and show how it addresses all
the issues inherent in the Student ID ranking system and in
Section IV we provide results from an initial case study of the
use of our approach by students on a core Computer Science
course at the University of Waikato.
II. BACKGROUND
In this section we review several approaches to delivering
student ranks and identify a set of characteristics that we can
use to contrast the different approaches as summarised in
Table I.
The traditional Student Name ranking system simply
provides a list of student names ordered by their position in the
class often with a column of actual mark values. The system
has the advantage of being perhaps the most straightforward
ranking approach to understand and use and allows students to
quickly identify their own position and prove their position in a
ranking to others. However Student Name rankings do not
provide any form of anonymity and have been largely replaced
by Student ID rankings at universities today.
The Student ID ranking system used as standard in New
Zealand Computer Science and Engineering departments today
can be seen as a simple change to the Student Name ranking
system wherein student names are replaced by their student
IDs. Student ID rankings have the inconvenience of an ID
lookup not seen in Student Name rankings but provide the
benefit of immediate anonymity which ensures third parties can
08-10 December 2014, Wellington, New Zealand
 TABLE I. CHARACTERISTICS OF DIFFERENT STUDENT RANKING SYSTEMS. FROM LEFT TO RIGHT, THE COLUMNS DESCRIBE WHETHER THE SYSTEM: ALLOWS
STUDENTS TO IDENTIFY THEIR OWN RANK FROM INFORMATION THEY HAVE (KNOW POSITION), PROVE THEIR RANK TO THIRD PARTIES (PROVE POSITION), PROVIDE
ANONYMITY TO THIRD PARTIES WITHOUT OTHER INFORMATION (IMMEDIATE ANYM.), ALLOWS POSITIONS TO BE SELECTIVELY CONFIRMED BY THE RANKING
AUTHORITY WITHOUT DISCLOSING STUDENT RANKS ON OTHER WORK (INDIRECT ANYM.), PREVENTS GROUP INFORMATION BEING USED TO CORRELATE IDENTIFIERS
(CORRELATION ANYM.) AND WHETHER THE SYSTEM ALLOWS STUDENTS TO DENY A RANKING IF A POTENTIAL MATCHING IS DEMONSTRATED BY A THIRD PARTY
(PLAUSIBLE DENI.).

Approach Know Position Prove Position Immediate Anym. Indirect Anym. Correlation Anym. Plausible Deni.
Student Name  




Student ID 




No Rankings

   
Unique ID 
   
Course Hashes      

not identify a student from the ranking without further
information. Unfortunately, however, immediate anonymity is
permanently removed from future rankings if a student’s
position in one ranking is disclosed to others for example
through a class prize for top students. Further, Student ID
rankings are vulnerable to correlation analysis attacks wherein
project team members can identify the IDs of other members in
group results and, like Student Name rankings, Student ID
rankings suffer from a lack of plausible deniability once a
student’s ID is known.
Of course, the most secure ranking system is one that does
not exist or is kept entirely private. Unfortunately, while being
secure, the No Ranking approach does not allow students to
know their own position in a class and has the potential
drawback of preventing students from being able to prove their
position to third parties which may be required to join some
companies [8] given the risk that letter grade assignments may
not be representative of ability and are almost certainly not
comparable between course variants, institutions and even
years for the same course at the same university.
A secure alternative to not providing public rankings is to
provide rankings with a unique ID for each course work item
for each student. This approach has the advantage of allowing
the student to know their own position in the class, provides
immediate anonymity, correlation anonymity and plausible
deniability and also provides indirect anonymity which allows
prizes to be given for a work item without disclosing the
unique ID mapping for other work items to third parties in the
case where the unique IDs are randomly assigned. However the
approach has the disadvantage of requiring students to manage
multiple IDs and of not providing any way for a student to
prove their position in a ranking without the university
providing explicit confirmation of a mapping.
The ranking approach we present in this work is called
Course Hashes. As Table I shows, Course Hashes provide a
means to disclose rankings to students in a way that is just as
private as using unique IDs but has the advantage of allowing
students to demonstrate to a third party that they have a
particular ranking position for a work item at any point in the
future without the need for a long lived central authority to
store the mappings for years to come. Unlike the Unique ID
system, Course Hashes have the advantage of not necessarily
requiring students to manage multiple keys and have other
benefits such as providing plausible deniability even after a
student discloses a mapping to “prove” they have a particular
position in a ranking as you will see in the following sections.
III. METHOD
In this section we discuss our approach to generating
selectively anonymous rankings. We start with our abstract
Course Hash algorithm in Section III-A before describing and
analysing our specific algorithmic implementation choice in
Sections III-B and III-C. The system architecture used to
deliver our algorithmic choice is described later in Section IV-
A where we detail our implementation case study.
A. Course Hashes
We define a Course Hash to be a function that maps course
and student information to unique IDs for use in rankings. In
contrast to the Unique ID system, access to the Course Hash
function is provided publicly so students can enter their
information to prove their position in a class to others thus
removing the main issue seen in the Unique ID approach as
illustrated in Table I.
The student information used in the Course Hash needs to
uniquely identify a student but also to provide direct
anonymity. To achieve this we can use student IDs and a
private key making our general Course Hash function:
courseHash(courseID, workItemID,
studentID, privateKey) (1)
The keys need to be provided on a per student basis to prevent
enumeration attacks given the low search space of student IDs
in most university classes. A different student key can be
supplied for each work item for each student to act as work
item proof certificates in the hash mapping or the same student
key can be used by a student across multiple work items where
a trusted Course Hash Generation Form such as the one
presented in Fig. 1 is provided as in our practical
implementation which we discuss further in Section IV-A.
An example of a Course Hash function would be to use the
variable block size Spacey Cypher [9] acting as:
spaceyCypherbk(courseID.workItemID.studentID,
privateKey) (2)

978-1-4799-7672-0/14/$31.00 ©2014 IEEE 08-10 December 2014, Wellington, New Zealand

2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 199
to encrypt a block consisting of a concatenation of the course,
work item and the student IDs (padded to the block size b) with
the private student key. The output would be a block of b-bits
in length that could be represented as a textual Course Hash ID
using an encoding scheme such as base64 [10].
The Course Hash ID generated by equation 2 would not be
guaranteed to be unique for each student ID because of the
different keys used however the probability of clashes can be
controlled through the choice of block size in the Spacey
Cypher which is not directly possible with fixed block sized
cyphers [11]. Additionally, the fact that any student ID could
map to any Course Hash ID given a suitable key means that the
approach provides plausible deniability even if a key is
provided that maps a student to a Course Hash ID which we
will discuss further after we detail our specific algorithm
choice next.
B. Specific Algorithm
Unfortunately, variable block size cyphers such as [9] are
not publicly available and with their unbounded key sizes
export may be controlled in some jurisdictions [12].
Consequently for our implementation we selected the Secure
Hash Algorithm (SHA1) [13] for our Course Hash function,
which is a 160-bit cryptographic hash function [14] that is
readily available with standard implementations for most
operating systems.
There are two problems using SHA1 instead of the Spacey
Cypher as a Course Hash function. They are that:
1) SHA1 does not take a key and
2) SHA1’s output is 160-bits which would lead to forty
hexadecimal or twenty seven base64 character rank IDs.
We can deal with the first issue by composing the key with the
course, work item and student IDs in the hash block to make a
key dependent output. There are a variety of standard methods
to compose the key data into a hash functions message block
including the keyed-Hash Message Authentication Code
(HMAC) [15] format, however as our IDs and key can be
restricted to a fixed length subset of the available characters we
use a simple concatenation of the form:
courseID.workItemID:studentID.privateKey
in this work with the IDs, “.”, “:” and the key represented in
standard ASCII which results in a fixed input message length
of 30 characters (240 bits) for our implementation.
To address the issue of the hash output length we can take
advantage of the characteristics of cryptographic hash functions
[14] which make any subset of SHA1’s output bits just as
suitable as any other subset when constructing a smaller sized
hash. Given this, we chose to use the first 72 bits of the SHA1
result for our implementation making our specific Course Hash
function:

 where N is the search space and n is
b64(SHA1(courseID.workItemID:
studentID.privateKey)[0..71]) (3)
where [0..71] indicates a bit range and the function b64() is the
standard base64 encoding [10] which generates twelve base64
character rank IDs for 72 bits of input.
978-1-4799-7672-0/14/$31.00 ©2014 IEEE
2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 200
C. Analysis of the Specific Algorithm
Our Course Hash function of equation 3 has 72 bits of
output variability thus inputs with any more than 72 bits of
variability must produce output clashes. With this in mind, our
private keys need not have an entropy H [16] of more than:
72 − H(studentID) (4)
bits to obtain a complete coverage of the 72 bit Course Hash ID
space for a set of students given that the course and work item
IDs of equation 3 are constants for a ranking. Thus for a class
of say 128 students (7 bits of student ID entropy), a private key
of only 65 bits would be sufficient to cover the entire Course
Hash ID space (assuming we have a perfect hash function
[17]).
In our implementation which we detail in Section IV we
use a private key space of 72 bits rather than the minimum
required key space for a set of students. This approach allows a
student ID to map to any Course Hash ID given an
appropriately selected key and so provides for full plausible
deniability (again assuming we have a perfect hash function).
The downside of allowing plausible deniability is that we have
the possibility for clashes in the output where say student A
with key X matches to the same Course Hash ID as student B
with key Y. However due to the Birthday Problem [17], [18]
the probability of a clash is only 1 in around 60 bits or 1 in:
580,999,813,345,182,728
for a class of 128 students given our 72 bit Course Hash IDs
which increases to 1 in around 58 bits (1 in
144,680,345,676,153,346) for a class of 256 students meaning
the chance that we need to generate a new key set because of
Course Hash ID clashes can be considered negligibly small for
practical class sizes.
The fact that a student ID could be mapped to a different
Course Hash ID given an appropriate key introduces a
challenge for students namely to find a private key that they
can use with their student ID to “prove” to others that they
have a better ranking than they actually do. The chance of
finding a better Course Hash ID increases as we go lower in the
rankings as there are more higher Course Hash IDs to match to.
For example, in a class of 128 students, the chance of the
bottom student finding a key that maps them to a better Course
Hash ID is 127 in 72 bits which means an average of around
1.4 times 264 full SHA1 computations would be required to find
a higher Course Hash ID1. Performing this number of
computations within a reasonable time would be a challenging
High-Performance Computing [19], [20], [21] project for most
Computer Science or Engineering students.
1
The average computations would be no more than 0.6% less than the figure
quoted (i.e. still well above 264) for an implementation that avoids repeating
the random keys it tries. To sketch the proof of the 0.6% figure, the
probability of not finding a matching Course Hash after T non-repeating key
tries is given by 
the number of used Course Hash IDs above the student which are 272 and 127
in our example. Thus the non-repeating probability of continuing is bounded
between  and  at step T which allows us to
lower bound the non-repeating tries required to obtain a probability of 0.5
using high scale logarithms with bc -l (to avoid the issue of calculating large
factorials).
08-10 December 2014, Wellington, New Zealand
 IV. RESULTS
To evaluate Course Hashes we used the approach to
provide rankings for a compulsory second year Computer
Systems course at the University of Waikato with course ID
COMP200. The course had 115 enrolled students of which 104
were male and 11 female and 99 were either Citizens or
Permanent Residents of New Zealand and 16 were
international students. In this section we begin by detailing the
system architecture we provided students to generate their
Course Hash IDs before providing anecdotal and survey results
to confirm the student opinion of our efforts in Section IV-B
and examining the impact the approach had on student
performance in Section IV-C.
A. System Architecture
For our case study we provided students with a key to
generate unique Course Hash IDs for each of the ranked work
items of the course which were a test, labs, an exam and the
overall results. We decided to use the same student keys across
the four work items to help make it clear to students the
difference between Course Hashes and Unique IDs and to
reduce key management issues in this first case study and we
present the students opinion of this implementation decision
through user survey results in Section IV-B.
To ensure the private student keys were randomly selected
we generated our 72-bit keys using the OSX secure random
number entropy pool with the command line:
base64 -i /dev/random -b12 | \
perl -e 'open(ID, "<./ids.txt");
while(<ID>) {
chomp; $key=<>;
print "$_, $key";
}' > keys.csv
where the input file ./ids.txt held the set of student IDs for the
class.
The private keys were distributed to the students using
Moodle [22] by creating a new extra credit grade item called
“Private Student Key” to store the keys as text results that were
only visible to the intended students. To get around limitations
in the Text Grade Type of Moodle 2.5 [23] we had to create a
Custom Scale for our Private Student Key grade item and load
all the keys into that scale (along with a first invalid key of “---
---------” to prevent the first text key being disclosed with the
Moodle Scale Range Display feature) to specify the set of
possible text grades/keys that could be assigned to students in
this class. Following the configuration of the new grade item,
we simply uploaded the keys.csv file to initialise the new
Private Student Key grade item using the standard Moodle
CSV grade import process.
To allow us to generate rankings by Course Hash ID we
needed to generate the Course Hash ID for each work item for
each student. We did that automatically using a command line
like the one below on OSX:
978-1-4799-7672-0/14/$31.00 ©2014 IEEE
2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 201
cat keys.csv | \
perl -e 'while(($id, $key)=(<> =~ \
/ˆ([ˆ, ]+), ([ˆ, \n]+)$/)) {\
@hash=`echo \
"COMP200.L:$id.$key\\c" | \
openssl sha1 -binary | \
base64 -b12`; \
print "$id, $key, $hash[0]"; \
}' > course_hash_ids.csv
with the input file ./keys.csv the student ID to private key
mapping file generated by the previous code and the
“COMP200.L” text on the fifth line replaced by the courseID
and workItemID we needed the Course Hash IDs for. We then
simply used the Course Hash IDs from the third column of the
output in place of student IDs in our public ranking reports.
In our case study, there were no Course Hash ID clashes for
any of the four rankings as one would expect given the low
probability of collisions discussed in Section III-C. However, if
a clash were to be seen, it would be a simple matter to generate
new private keys and Course Hash IDs using the scripts above
before distributing the keys to students for a course.
To allow students to identify their position in the rankings
they needed to know their Course Hash IDs for each ranked
work item. Rather than publishing these to students privately
on Moodle we provided the public Course Hash ID Generation
Form shown in Fig. 1. This approach allowed the students to
use a single form to find their own Course Hash ID and to
prove they possessed a key that maps their student ID to a
specific Course Hash ID when demonstrating their position in
\ the rankings to third parties.
\
\
\
Fig. 1. The Course Hash ID Generation Form for the COMP200 course at
Waikato University [24]. Pressing the “Generate” button after entering a
student ID and key returns the Course Hash ID for the selected work item.
08-10 December 2014, Wellington, New Zealand
 courseID.workItemID:studentID.privateKey
Course
Web Page
Student
Fig. 2. Illustration of our Course Hash implementation architecture using the SaSe Secure Hashing Service [25] to perform hash calculations. The response from
the SaSe service is a base64 encoded SHA1 of the passed data string from which the first 12 characters are shown as the 72-bit Course Hash ID by the course
web page JavaScript code.
The form was made accessible on a trusted website for the
course hosted on the university web servers [24] and
implemented our Course Hash algorithm by simply calling the
SaSe Secure Hashing Service [25] over a standard REST-
JSON [26] [27] client connection as illustrated in Fig. 2. Note
that as we only use student IDs (which are traditionally
published and thus considered non-private in Student ID
rankings), our generated keys and work item codes in our
Course Hash algorithm, we are not sending any personal
information such as e-mail addresses or names to the hashing
service. Having said that, it should be noted that the case study
architecture is not immune to Pharming attacks and so trusted
networks or clients secured with techniques such as [28] are
assumed when verifying a student Course Hash ID mapping
and in any case, the form of Fig. 1 makes students and third
parties aware that official verification is still required in order
for ranking positions to be relied on.
B. Student Reception
To this point we have considered laws and bills currently
before parliament [1], [2], [3], [4], [5], [6], hybrid cyphers [9],
cryptographic hashing algorithms [13], the Birthday problem
[17], [18], High-Performance Computing [19], [20], [21] and
provided formal probabilistic bounds on clashes with repeating
and non-repeating key selections over large search spaces.
While this level of complication was necessary to motivate the
need for Course Hashes and analyse the security of our
algorithmic choice in a robust manner, it should be recognised
that users only need to be able to enter their student ID and key
in the form of Fig. 1 to be able to actually use Course Hashes.
However, this “black box” usage simplicity does not mean that
our case study implementation can be considered user friendly
in requiring, as it does, private key lookups in Moodle, typing
of keys and student IDs into a public hash generation web page
and then a final grade lookup in a separate ranking PDF
document. Despite this lack of integration, the students adopted
the implementation without significant issue and were mostly
enthusiastic about the new ranking approach as we demonstrate
with anecdotal and survey evidence in the remainder of this
section.
978-1-4799-7672-0/14/$31.00 ©2014 IEEE
2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 202
Hash
Calculation
b64(SHA1(data)) Service
We introduced the idea of Course Hashes to students on the
COMP200 course through a Moodle notice explaining that
rankings could not be given for the course by student ID if we
were also going to give prizes to the top students and so we
were introducing an alternative ranking system. Technical
details of the Course Hash approach were not given and we did
not explain the alternate approaches considered like Unique
IDs in the e-mail. We followed the initial notice up after
finalising our Course Hash implementation with a notice
explaining how students could get their private keys from the
pseudo grade item in Moodle and provided a link to the Course
Hash Generation Form of Fig. 1 which has some motivation
and usage information above the entry fields.
To assess the ease with which students grasped the concept
of Course Hashes we decided to only release the grades of our
first returned assessment item (a mid-semester test) through a
Course Hash ranking and held off releasing the scores through
the Moodle standard grade system. To recap the composition of
the case study, there were 115 students enrolled on this core
second year course of varying levels of prior computer
knowledge and ability but despite this, there was only one
request for support after we released the first Course Hash
rankings on Moodle. In that request two local students
questioned how to get their Course Hash ID from the form and
one of those students questioned why they had to use Course
Hashes in the first place. Before the support staff had a chance
to respond, both local and international students replied to the
post explaining that the key required for the hash generation
form was the Private Student Key from Moodle rather than say
the student’s Linux password and detailed the problem with the
traditional Student ID ranking system explaining how they
understood and appreciated the anonymity afforded to them by
the new Course Hash approach. Given the student responses
staff did not need to respond to the support post and Course
Hashes were accepted by all as the primary form of grade
distribution for the course from that point on.
Following the course we provided the students with an
opportunity to provide feedback on their Course Hash
experience. We created a survey adhering to Total Survey
08-10 December 2014, Wellington, New Zealand
 TABLE II. SENTIMENT CLASSIFICATION RESULTS FROM THE COURSE HASH USER SURVEY. THE SURVEY ASKED STUDENTS TO RANK THEIR AGREEMENT WITH
THE STATEMENTS BELOW USING THE STANDARD SURVEYMONKEY SENTIMENT LABELS OF: STRONGLY DISAGREE (SD), DISAGREE (D), NEITHER DISAGREE NOR
AGREE (NAND), AGREE (A) AND STRONGLY AGREE (SA). THE MODE COLUMN GIVES THE MOST COMMON RESPONSE TO EACH QUESTION WITH A “-” USED TO
INDICATE THE NEITHER DISAGREE NOR AGREE (NAND) RESPONSE. THE FIGURES IN BRACKETS ARE THOSE FOR STUDENTS WHO ENTERED A VALID FEEDBACK ID
AND THE FIGURES OUTSIDE BRACKETS ARE FOR ALL RESPONSES.

Survey Question Mode SD D NAND A SA

Valued the Attempt to Protect Privacy Agree 2 (1) 1 (1) 4 (3) 11 (7) 8 (4)

Interested in Additional Technical Information - 2 (1) 7 (4) 8 (5) 6 (4) 3 (2)

Comfortable Using the Course Hash Generator Agree 5 (2) 6 (2) 2 (1) 9 (8) 4 (3)

Would Prefer Per Work Item Keys - 7 (3) 7 (3) 8 (7) 4 (3) 0 (0)

SA
SD SD SA SD SA
D A
SA SD
NAND D A
D A
NAND
D
A NAND
NAND
Valued More Information Usable Work Item Keys
Fig. 3. Pie charts corresponding to the total survey result figures of Table II for the questions (from left to right) of whether the students: valued the attempt to
protect their ranking privacy, were interested in additional technical details, were comfortable using the Course Hash Generation Form and whether they would
prefer individual work item keys to a single key. The segment labels correspond to the SurveyMonkey classifications of: Strongly Disagree (SD), Disagree (D),
Neither Disagree Nor Agree (NAND), Agree (A) and Strongly Agree (SA).

Quality (TSQ) design principles [29] to try to assess the level
to which the students:
1) valued our attempt to protect their ranking privacy,
2) would be interested in more details on the approach,
3) were comfortable using the ID Generation Form and
4) would prefer per work item private keys instead of the
single Private Student Key provided.
as well as to collect general feedback on the good and bad
points of the student’s initial Course Hash experience and any
specific suggestions for improving the implementation and the
assessment questionnaire itself (to identify any TSQ usability
issues). To satisfy the timelines, accessibility, col- lection
completeness and accuracy dimensions of the TSQ
requirements and to reduce measurement, item non-response
and processing errors [29] we used SurveyMonkey [30] with
ranged radio button and free text entry boxes to collect and
collate survey responses. To satisfy the credibility and
comparability dimensions of TSQ we provided an optional
additional text field on the survey form for students to enter a
“Feedback” Course Hash ID which they could generate using
the same Private Student Key and form they used to generate
the Course Hash IDs for the standard work items of the
COMP200 course [24]. The additional Feedback ID allowed us
to identify confirmed unique survey responses from actual
COMP200 course students without having to provide student
personal e-mail addresses or connect student IDs with student
views on the externally administrated survey website.
The survey invitation e-mail was constructed according to
the Web Survey Invitation Design principles of [31] and e-
mailed to the student’s with two reminders sent over a 1 week
period. No prize or inducements were given to complete the
survey and in total 26 students responded (22.6%) which is
consistent with the student response rates seen in [31].
Table II and Figure 3 summarise the survey results. It can
be seen that the responses of the group of students who elected
to enter their Feedback ID (whose figures are quoted separately
in brackets) is not inconsistent with the responses of the other
students in the table who just answered the survey quickly.
The survey used labelled sentiment fields to collect
responses, which makes creating an average view difficult
without a set of weights that arguably would need to be
calibrated on an individual respondent basis. However, taking a
cue from SurveyMonkey where the five sentiments are given a
linear weighting scale, if we weight the sentiments with the
integers from -2 to 2 with negative weights for disagreement
and positive weights for agreement, we can summarise the
views of Table II as that the students as a group were quite
positive about our attempt to protect their ranking positions
from discovery by other students and people outside the class
(+0.85 average), had mixed views about the need for additional
information and on the ease of use of the implementation
(+0.04 average for both) and did not think additional work item
keys would be of any benefit given the current implementation
(-0.65 average). This interpretation of the average student view

978-1-4799-7672-0/14/$31.00 ©2014 IEEE 08-10 December 2014, Wellington, New Zealand

2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 203
TABLE III. CHANGE IN GRADES AFTER INTRODUCING COURSE HASHES HALF WAY THROUGH THE LABS AND AFTER THE MID-SEMESTER TEST OF THE COURSE.
THE STUDENT ID RANKING COLUMNS REFER TO THE PREVIOUS YEAR WHERE ONLY STUDENT ID RANKINGS WERE USED AND ARE PROVIDED FOR COMPARISON.
THE  FIGURES SHOW THE CHANGE IN THE AVERAGE GRADE AND THE

 FIGURES SHOW THE CHANGE IN GRADE STANDARD DEVIATION OF THE WORK
ITEMS COMPARED IN ROWS. THE P-VALUE COLUMN GIVES THE PAIRED TWO TAILED STUDENT T-TEST FOR THE UNDERLYING RESULT DATA WITH THE NULL
HYPOTHESIS BEING THAT THE GRADES FOR THE RELATED WORK ITEMS ARE FROM THE SAME DISTRIBUTION. THE PAIRED SAMPLES CONSISTED OF 108 STUDENTS
FOR THE STUDENT ID RANKING COLUMNS AND 93 STUDENTS FOR THE COURSE HASH RANKING COLUMNS AFTER PRUNING OF INCOMPLETE SAMPLES
CORRESPONDING TO STUDENTS THAT DID NOT SUBMIT ONE OR MORE WORK ITEMS. SIMILAR FIGURES WERE SEEN FOR MALES, FEMALES, LOCAL AND
INTERNATIONAL STUDENTS.

Student ID Rankings Course Hash Rankings

   
    
p-value    
     p-value
Final Exam / Mid-Semester Test 0.91 (1.24) 3% 1.43 (1.17) 0%

Second Half / First of Labs 1.01 (1.04) 73% 0.94 (1.18) 9%

was supported by the students’ textual feedback comments V. CONCLUSION

where most students confirmed they appreciated the attempt to
protect their ranking privacy but some explained that they
wished the implementation provided tighter integration
between their private keys, the Course Hash Generator Form
and the ranking lists and it remains for future work to consider
user experience enhancements as discussed in Section V.
C. Impact on Student Performance
In an attempt to understand the impact of Course Hashes on
student performance we had to first obtain a baseline for the
students this year with this lecturer, support staff and course
material. To enable this we elected to introduce the idea of
Course Hashes after the mid-semester test for the course
(before the test results were returned) and half way through the
labs. This allowed us to compare the student grade distributions
for the mid-semester test with those of the final exam and for
us to identify any changes in lab performance after the
introduction of Course Hashes.
Table III shows the change in grades between the
assessment items of the first and second half of the Computer
Systems course for a year when just Student ID rankings were
used and the year where Course Hashes were introduced for the
second half. The table shows that the exam grades for students
increased significantly over the mid-semester test grades
following the introduction of Course Hashes, a reversal of the
position in the Student ID only ranking year where the exam to
test grade distribution was also wider. In contrast to the exam
grades, the grades for the labs were slightly decreased after the
introduction of Course Hashes although not statistically
significantly so with a p-value of 9%. If we were prepared to
accept the drop in lab performance as significant, the decrease
could be attributed to students deciding to work more
independently on the second half of the labs after Course
Hashes were introduced given the additional rank protection
Course Hashes provides for students to learn and make
mistakes without being singled out and it is noteworthy that
this interpretation is supported by the increase in standard
deviation for the second half lab results over the same period in
the previous year of the course. However, while the foregoing
changes may seem significant, it should be noted that the
course exam, test and, clearly, the students themselves
necessarily changed between the two years of the course
considered and further research will be required before any
causality can be reliably attributed to the use of Course Hashes.
This paper introduced the idea of Course Hashes as a way
to deliver selective anonymity in student course work rankings.
The need for selective anonymity was justified through the
consideration of laws and bills currently before parliament
concerned with protecting student and personal privacy in the
face of new technology [1], [2], [3], [4], [5], [6] and the
understanding that the current Student ID ranking system offers
little protection for classes where performance prizes are given
or where students work in groups over any period.
Course Hashes are a cryptographic solution to privacy that,
in effect, provides students with a key they can use to certify
their position in a ranking to others. To analyse the security of
our Course Hash algorithmic choice it was necessary to look at
hybrid cyphers [9], cryptographic hashing algorithms [13], the
Birthday problem [17], [18] and formal probabilistic bounds on
clashes with repeating and non-repeating key selections over
large search spaces. However, from a users perspective, our
implementation consists of a simple web form [24] that maps a
student ID to a Course Hash ID with the click of a button and
this “black box” simplicity allowed Course Hashes to be
introduced with positive reception on a course of 115
Computer Science students with minimal explanation or
support in our case study.
In Section IV we provided evidence of the impact of
Course Hashes from our case study and showed that after we
introduced Course Hashes our student exam results increased
and variance ratios decreased in stark contrast to previous
years. However, while our results showed improvement in a
statistically significant manor, we recognise that further
research will be required to confirm the impact of Course
Hashes given the year to year variability in exams, tests and
other items of courses at universities.
The Course Hash approach we implemented has the
advantage of being a publicly disclosable algorithm that does
not need a long-lived database of scores, access to a login
server or significant development work to implement. However
the student survey feedback discussed in Section IV-B
indicates that additional work may be required to improve user
experience in future implementations potentially at the cost of
one or more of the current implementation advantages. For
example it is easy to see how user experience could be
improved by providing a parameterised URL to initialise the

978-1-4799-7672-0/14/$31.00 ©2014 IEEE 08-10 December 2014, Wellington, New Zealand

2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 204
Course Hash Generator Form instead of requiring a Moodle
key lookup step, by linking the Course Hash Generator Form
with a rank database to remove the PDF lookup step or even by
effectively reversing the Course Hash process by providing a
trusted website with log-in server integration that generates
position certifying URLs on request for third parties. While
these user experience improvements may require more
resources than the case study implementation, they are no more
complex than a modern on-line store or social networking site
to implement and it is hoped that this paper will help highlight
the issue of privacy in Student ID rankings, demonstrate that
students value efforts to improve their ranking privacy and
focus efforts to make selective anonymity a standard feature in
the student rankings of the future.
ACKNOWLEDGEMENTS
The author would like to thank the students of COMP200
who took part in the Course Hash trial, SaSe Business
Solutions for the use of their SSP services and the anonymous
reviewers and conference organisers for their feedback and
support.
REFERENCES
[1] Harmful Digital Communications Bill, New Zealand Government, 2014.
[2] Harassment Act, New Zealand Government, 1997.
[3] Privacy Act, New Zealand Government, 1993.
[4] Human Rights Act, New Zealand Government, 1993.
[5] Crimes Act, New Zealand Government, 1961.
[6] Education Act, New Zealand Government, 1989.
[7] “Cyberbullying information and advice for teachers and principals,”
New Zealand Ministry of Education Sponsored White Paper, NetSafe.
[8] General Questions. SaSe Business Solutions. [Online]. Available:
http://www.sase.biz/faq.html
[9] S. Spacey, “The Spacey Cypher,” UK Patent GB2 379 587B, 10, 2001.
[10] The Base16, Base32, and Base64 Data Encodings, Internet Engineering
Task Force (IETF) Std. RFC 4648, 2006.
[11] Advanced Encryption Standard (AES), National Institute of Standards
and Technology (NIST) Std. FIPS PUB 197, 2001.
[12] Encryption Export Controls, vol. 75, no. 122, Bureau of Industry and
Security, 2010.
[13] Secure Hash Standard (SHS), National Institute of Standards and
Technology (NIST) Std. FIPS PUB 180-4, 2012.
978-1-4799-7672-0/14/$31.00 ©2014 IEEE 08-10 December 2014, Wellington, New Zealand
2014 International Conference of Teaching, Assessment and Learning (TALE)
Page 205
[14] P. Rogaway and T. Shrimpton, “Cryptographic hash-function basics:
Definitions, implications, and separations for preimage resistance,
second-preimage resistance and collision resistance,” in Proc. Fast
Software Encryption (FSE), vol. 3017, 2004, pp. 371–388.
[15] The Keyed-Hash Message Authentication Code (HMAC), National Insti-
tute of Standards and Technology (NIST) Std. FIPS PUB 198-1, 2008.
[16] C. Shannon, “A mathematical theory of communication,” SIGMOBILE
Mob. Comput. Commun. Rev., vol. 5, no. 1, pp. 3–55, 2001.
[17] X. Wang, Y. Yin, and H. Yu, “Finding collisions in the full SHA-1,” in
Proc. Advances in Cryptology (Crypto). Springer, 2005, pp. 17–36.
[18] B. Schneier, Applied Cryptography: Protocols, Algorithms, and Source
Code in C. Wiley, 1996.
[19] S. Spacey, W. Wiesemann, D. Kuhn, and W. Luk, “Robust software
partitioning with multiple instantiation,” INFORMS Journal on
Computing, vol. 24, no. 3, pp. 500–515, 2012.
[20] S. Spacey, W. Luk, P. Kelly, and D. Kuhn, “Improving communication
latency with the Write-Only Architecture,” Journal of Parallel and
Distributed Computing, vol. 72, no. 12, pp. 1617–1627, 2012.
[21] S. Spacey, W. Luk, D. Kuhn, and P. Kelly, “Parallel partitioning for
distributed systems using sequential assignment,” Journal of Parallel
and Distributed Computing, vol. 73, no. 2, pp. 207–219, 2013.
[22] Moodle Home Page. Moodle Pty Ltd. [Online]. Available:
http://www.moodle.com
[23] “supplied grade is invalid” error when importing text grades from csv
file. Bug Tracker. Moodle Pty Ltd. [Online]. Available:
https://tracker.moodle.org/browse/MDL-35574
[24] “Course Hash ID Generation Form for COMP200,” The University of
Waikato, 2014. [Online]. Available:
http://cs.waikato.ac.nz/
sspacey/teaching/COMP200/#course hash
[25] “SaSe Server Pages (SSP),” Product Information Sheet, SaSe Business
Solutions, 2012.
[26] R. Fielding, “Architectural styles and the design of network-based
software architectures,” Ph.D. dissertation, University of California,
Irvine, 2000.
[27] The JavaScript Object Notation (JSON) Data Interchange Format,
Internet Engineering Task Force (IETF) Std. RFC 7159, 2014.
[28] S. Spacey, “Site Continuity Management (SCM),” D.CSC. thesis,
Cambridge University, Cambridge, UK, 2007.
[29] P. Biemer, “Total Survey Error: Design, implementation, and
evaluation,” Public Opinion Quarterly, vol. 74, no. 5, pp. 817–848,
2010.
[30] SurveyMonkey Home Page. SurveyMonkey Inc. [Online]. Available:
http://www.surveymonkey.com
[31] M. Kaplowitz, F. Lupi, M. Couper, and L. Thorp, “The effect of
invitation design on web survey response rates,” Social Science
Computer Review, vol. 30, no. 3, pp. 339–349, 2012.