Department of Informatics

University of Leicester
CO7201 Individual Project

Preliminary Report
Online judge system for programming competitions

[Your name]
[Your email address]
[Your student ID]

Project Supervisor: [XXXX]

Second Marker: [XXXX]

Word Count: [XXXX]

[Submission Date]
 DECLARATION
All sentences or passages quoted in this report, or computer code of any form whatsoever
used and/or submitted at any stages, which are taken from other people's work have been
specifically acknowledged by clear citation of the source, specifying author, work, date, and
page(s). Any part of my own written work, or software coding, which is substantially based
upon other people's work, is duly accompanied by clear citation of the source, specifying
author, work, date and page(s). I understand that failure to do this amounts to plagiarism
and will be considered grounds for failure in this module and the degree examination as a
whole.

Name: [Your name]

Date:[Date of submission]

1 Contents

2
Abstract 3
1. Aims and Objectives 3
1.1. Aim 3
1.2. Objective 3
2. Requirements 4
2.1. Input/output stream 4
2.2. Signal interaction mechanism 4
2.3. Message box technique 5
2.4. Sandbox connecting manager 5
2.5. Sandbox Business Manager 5
2.6. Parallel judgment 6
3. Technical Specification 6
4. Requirements Evaluation Plan 6
5. Background Research and Reading list 7
6. Time-plan and Risk Plan 8
7. References 9
Abstract

In programming competitions, an online judge is an online method for evaluating programs.

For such competitions, they are often used for practice. Your code can be compiled and
executed by the system and your code checked with pre-constructed results. You can run the
submitted code with limitations, including time limit, memory limit, security limit, and so on.
The code output will be recorded by the machine, and the normal output will be compared.
Then the machine will return the result. Once defects in a standard production have been
detected, the application would be unsuccessful. Some mistakes in the code must be corrected
and resubmitted for re-judgement.

1. Introduction
1.1. Background

An online judge system for programming competitions (OJS) is a distributed system for
executing and (to some point) systematising a programming competition. The system is
developed to be universal and to handle numerous different types of contests, tasks, and
scorings. Besides, OJS has been explicitly developed to be utilised in students and college
competitions in Informatics. The candidate's code can be compiled and executed by the
system and user’s scripts scrutinised with pre-constructed results. Users can execute the
submitted code with limitations, comprising time and memory limits, security limits, and
much more (Chang et al., 2014). The scrips output will be documented by the computer, and
the normal output will be assessed Then the machine will return the result. After flaws in a
standard production have been noticed, the system would be unsuccessful. Some errors in the
code should be modified and resubmitted for re-examination.

The script output would be obtained by the system and associated with the standard output.
The system will then return the result. When mistakes were determined in a typical output,
pre-evaluation utilising the same method must be created. The entire script tasks are resolved
using a virtual machine. The OJS system is created with a container that incorporates an
original memory, having a pre-set network authorisation, Time limit for the system container.
All the system functions are considered by the virtual machine. System users can concentrate
on the other roles (Wirawan et al., 2017).

In this application, the entire online coding judging procedure is the most essential aspect.
Therefore, to make the judging system work, the process carefully listens to the principles of
an operating system and its functions. Besides, the principle of each programming language
such as calling and declaring a variable and executing commands. The software will be a
(PHP: Hypertext Pre-processor (PHP) application that would be capable of parsing codes
comprised of JAVA, Perl, C++, and python content, obtain web services descriptions, outputs
and show them as outputs activity scores for assessment. Finally, the software application can
handle submission queues, and download submitted codes in readable file formats such as
Excel. The solutions to the problem are to create can handle numerous user roles such as
admin, head instructors students, and instructors and sandboxing for Java, cheat detention
such as script similarity identification, and a custom rule for grading late submission. This
paper describes the requirements, features, application concepts, and computer technologies
used in the project as well as the design and implementation detail of the online judging
application.
1.2. Project Aim
The main task of this project is to implement a judging system, which contains a cross-
platform online judge system based on spring MVC framework and Active MQ. Online judge
system being the optimum online web application for online programming competitions.
With the increasing online programming competitions globally, there should be a technical
solution that selects the competition winner fairly and transparently. The Online judge system
would be the most used online software functionality for determining a winner of the
programming competition hosted with a database and security features such as sandbox to
mitigate hacking. The project aims to obtain a fair judging process by designing features for
online instant judging abbreviated as "OJS". The software aims at achieving an easier
judgement matrix using a sandbox judge interface and the client-side website interface. The
sandbox design would have features of instantly compiling and executing the tested scripts,
producing assessment scores, and subsequent analysis reports with the programming winner
(s). This software development project also aims at using thread pools to realize parallel tests,
and implement database optimisation techniques; for instance, parallel split tables.
1.3. Project Objective

To obtain the study aim, the key project objectives are as follows;

i. In order to develop an online automatic judging web application for programming

competition in PHP
 In the system, the assessment on test scripts are performed and an automatic report
is issued with judgement results to determine a winner

ii. To design a web and server-side client using the MVC procedure

The MVC procedure provides control, business logic, and service layers to support
the system interface based on functionality being performed. In order to streamline
retrieval, the server-side would offer a parallel table split to store the submitted
codes.
iii. To integrate the OJS web application with sandbox
Since similar systems are exposed to hacking, PHP offers a sandbox architecture design
to mitigate malicious coding and hacking during analysis and protect the system from
hacking (Coker et al., 2015).

2. Requirement Analysis
2.1. Functional Requirements
Functional requirements are essential to meet system objectives. The functional requirements
of the OJS are designed according to the review of similar existing applications and research
conducted in the introduction chapter. The OJS system provides functions of question
management, user management, system management for the administrator, and provides
functions of joining the competition and checking the score for the user. The overall system
architecture for functional requirements is illustrated in Figure 1.

Figure 1: Functional Requirements

2.1.1. Question management
Also referred to as problem management, the OJS shall allow system admin to set questions
for the competition, allow candidates to answer questions, search and read problems.
2.1.2. User management
The OJS will ensure user registration for competition candidates, instructors, and system
admin. Users shall have to log in to the system to access their roles, update personal
information, retrieve their details.
2.1.3. System management for the administrator
The system shall contain settings, sandbox manager, role manager, problem manager user
manager, and competition manager, announcement manager as well, all of which will be
controlled by system admin. The admin shall set roles and data access options.
2.1.4. Score submission and joining the competition
The system contains a mechanism to examine the submitted solution against the fundamental
test cases and generate scores. Equally, each competition must have some hidden test cases,
editorial accessibility after competition submission, and comments.

2.2. Non-Functional Requirements

2.2.1. Uniform Accessibility
All the system users should have the capability of using the web application regardless of
their physical disability status. All Web pages should be validated with W3c for Html and
CSS to ensure acceptable global navigation. Besides, the verification from w3c ensures that
the system scripts conform to w3c guidelines Further, consistency in the web pages must be
maintained by the software application to ensures that users can effortlessly execute the
function without much workload as explained by Schniderman’s rules for web development
2.6.3.
2.2.2. System administrator to obtain competition scores using MS Excel
The system admin interface needs to be designed with functions to allow the download of
competition codes and submission answers. The function will enable instructors to examine
competition results and establish how the candidate won. The system to perform competition
analysis using graphs or charts in MS Excel.
2.2.3. Maintainability and Support Maintenance Requirement

Database data should easily be updated by the system administrator while ensuring site
availability.
2.2.4. Security Access Requirement
The web application shall ensure data protection using password and username before access
to the system to perform different roles. Therefore, the fit criteria will require user roles
(Actors) to have access to allowed sections as labelled in the section.
2.2.5. Privacy Requirements
The web application should inform the system used to collect competitors' consents to utilise
their use cases on the system.
2.2.6. Cultural Requirement
The web application shall not be offensive to religious and ethnic groups. All contents should
conform to diversity and cultural appropriateness.
2.2.7. Legal Compliance Requirement
The implementation for the competition submission by each candidate shall conform to the
Data Protection Act. Users should ensure they obtain attorney's opinion that the web
application does not violate any regulations and policies.
2.3. High-level Function

The OJS web application provides specific high-level functions such as linking programming
competition and examining test scores and display the score of each candidate in the
competition. To obtain the technical functions listed, the judging web application comprise
five fundamental modules, consisting of quiz module, competition module, system user
module, a messaging module as well as software module. All the module tests of programs in
coding contests, recruiters can conduct hiring challenges and for intra-inter student coding
competitions.
2.4. Low-level Requirements

2.4.1. Entities Identification

The online judging system contains the following entities: users; programming contest;
question; submissions; registrations; question test cases; cumulative leader boards
2.4.2. Entity Relationships
 A user entity has_many competitions via participations — user. competitions—
utilised for queries for instance “Get all competitions userA registered for”
 A competition entity has_many users through participations — competition.users —
“Get all the users registered for competitionA”
 A competition entity has_many questions — competition.questions — “Get all the
question for challengeA”
  A question entity belongs_to contest — question.competition — “What competition
questionA appeared in?”. belongs_to relationship suggests that the question entity has
a foreign key of competition_Id
 A question has_may submissions — question.submissions — “obtains all the
submissions of questionA” or “Gets all the submissions of questionA by userA”
 A submission belongs_to user — suggests that submission has a foreign key of userId
 A submission belongs_to question — suggests that submission has a foreign key of
questionId
 A participation belongs_to user- suggest that participation has a foreign key of userId
 A participation belongs_to competition – suggests that participation has a foreign key
of competition_Id
2.4.3. Data Modelling
Table 1 presents database entities (along with their attributes) and data types in bytes for each
column (Utilising SQL storage data).
Entity Attributes
Users userId(36), name(150), dateOfBirth(3), email(30), college(30),
username(20),
password cumulativeScore(8), addedAt(8), updatedAt(8)

Competition competitionId(36), name(150), competitionStartTime(8),

competitionEndTime(8)
Questions questionId(36), competitionId — foreign Key(36), name(150),
description(300), questionText(2000), testCases(2000),
author(100), addedAt(8), updatedAt(8), editorial(3500),
comments(2000), points(8), difficultyLevel(10)
Submissions submissionId(36), userId — foreign Key(36), questionId —
foreign Key(36), programmingLanguage(15),
submittedCode(3000), score(8), createdAt(8)

Registration participationId(36), userId — foreign Key (36), competitionId

— foreign Key(36), score(8), acceptedSubmission

2.4.4. Data Size

For each row, the Users entity contains storage of ~ 400 bytes, Competition~ 250 bytes,
Questions ~ 12KB, Submissions ~ 4KB, Registration ~ 300bytes.
3. System Design
3.1. Software Architecture
3.1.1. Overall System Design
The OJS architecture ensures that the user converses with the OJS using the web browser
client. The web application has numerous application servers that query data from the system
database to distribute the load. The whole system servers are stateless thus offering the users
an easy load balancer to distribute the load across the OJS application servers. The system
database, which stores metadata of the system users, competitions, questions, and
competition. Besides, a simple Central Processing Unit (CPU) concentrated collections of
machines to execute the submitted code against the test cases. The data flow diagram is
shown in Figure 2.

Figure 2: System Data Flow

The OJS system is based on B/S architecture, comprising a security sandbox and environment
web section. The architecture offers a sandbox judging side and client-side and obtains the
flexible deployment and enlargement as shown in Figure 3.

Figure 3: System Architecture

3.1.2. Client-side Design
As observed from Figure the client-side design is developed using spring model Model-
View-Controller (MVC) framework system architectural pattern, which splits the OJS into
three main logical components Model, View, and Controller. MVC separates the business
logic and presentation layers, service layers as well from each other. The systems client-side
has a visual interface for users and stores their data into the OJS database. The client-side
web pages are connected to PHP sandbox to necessitate registration, login, and business logic
processes.
3.1.3. Security Sandbox Design
The application uses PHP sandbox security to protect submitted codes and the entire
competition process from malicious codes. To enforce the integrity of the codes, candidates
submit codes only using their limited access resources via the client-side interface within the
sandbox offered for each candidate. The PHP secure sandbox code is demarcated from the
client-side but connected via an association procedure. The secure sandbox judge can request
the client-side and compile as well as execute the submitted codes y users. After which the
sandbox outputs the executed results based on the present protocol input files. The output
then compiles and submits back to the client-side interface.
3.1.4. Business Process Design
The main key business process is the judging of codes once the client-side push the
competition scripts from users, then compile the codes and sends them to the requested
secure PHP sandbox. The sandbox then compiles and executes the obtained codes and then
submits the results based on the designed standard input instruction. The results for each
candidate are then submitted back to the client-side. Finally, the client-side matched the
returned results with the anticipated output and estimated the detail scores for each user
(Mizuno and Nishizaki, 2014). The process for system judging is illustrated in Figure 4.

Figure 4: Online Judging Process

The scoring procedure uses a multiplier for scoring submitted codes for the pre-installed text.
The scores are validated in the score column under the submissions table, and a candidate’s
overall score in the competition can be reorganised in the candidate's table and downloaded
using various files such as Excel.
3.2. Application Technologies
3.2.1. Programming Languages
The web technologies that the OJS system uses are PHP; Hyper Text Markup Language
(HTML); Structured Query Language (SQL); MySQL as the server, and JavaScript. The
application is developed using a general-purpose coding language of PHP and executes a web
server that listens to PHP scripts being the input and outputs framework for the HTML client-
side web pages, which are also styled using Cascading Style Sheets (CSS). The website
joined a MySQL relational database and utilises SQL to outline, operate and query the OJS
database. Advanced scripting functions are obtained using JavaScript language. JavaScript
connects the client-side to a client-server architecture. Figure 5 illustrates the run-time static
view of the client-server web architecture for the OJS.

Figure 5: OJS Static view of the client-server web architecture

3.2.2. Separating Input and output stream

Programming in Java input and output streams are used to handle software inputs and
outputs. The streams allow the program to consistently communicate with numerous sources
of data; for example, network connections and memory blocks. In the case of project
software, the streams can be used to execute test cases (O'Reilly, 2021). The inputStream in
the system would read the codes submitted, the outputStream would be used to write the code
to a destination. This will further be strengthened by split function to eliminate concurrency
issues.

The input and output stream of a file stream of System.seIn () method can halt some threads
from getting the input stream or possibly retrieve the input stream from other threads
(O'Reilly, 2021). Obstructing some threads might interfere and jeopardise the validity of the
judgment process as well as the output stream. In a multi-thread setup, when multiple test
cases output the results to the same output stream redundancy might occur hence impacting
the validity and reliability of the judging output. The standard of separating input and output
stream is illustrated in Figure 6.
 Figure 6: Input and output stream

3.2.3. Signal interaction mechanism

The system would need both the client-side interface and sandbox to listen to each other
based on the defined signals in the network. Therefore, the system will have a detailed
communication protocol for signal and winning rules. Also, a fundamental test process that
forms the sandbox specification and abstraction. Ideally, the signal communication technique
offers data needed for interaction across the two interfaces. Further, it forms the base on
which the sandbox is informed via signals such as if the client-side needs to ensure the
services being offered by the sandbox side. Also, the type of function to achieve, as well as
the technique to send the information. The system will have main signals for judging
requests, get the sandbox status, and exit the sandbox to mention a few. For instance, if the
client-side sends requests from the end judge, the system will stock judge signals to the
required information Also, the sandbox-side gets the request, assesses it, and interprets the
remaining information like the required data for the judgement procedures. Thereafter,
utilises the data as the source code in order to compile, execute and finally summarise the
operation output, and return the request result to the client-side interface.
3.2.4. Message box technique

Given that interaction between client-side and sandbox via network connection might be
slow, there is a need to use asynchronous mechanism to the client-side when waiting for the
sandbox side to respond to the result. Also, if the thread sends judgment requests to the
sandbox and inspects the request the communication selected might halt the order of the
request array and return output with errors or possibly using a lot of time. Therefore, the
message box technique can eliminate the interference and enhance data efficacy during the
interaction. Each sandbox data contains a unique message identifier ID that helps to return
back the sandbox message using the unique ID. As such the system would obtain accurate
information verified with the unique message ID.
3.2.5. Sandbox connecting manager

The Sandbox connecting manager will maintain the status of each sandbox to obtain optimum
load balancing between the sandbox and the client-side interface. The connecting manager
will also maintain the queues of the sandbox and judging requests. Since the requests are
being received in bulk the manager will use the FITO (First in First out) procedure to sort out
the queue and finish the task. Also, to ensure efficiency, the queue will be protected, and keep
each sandbox inside the queue to make sure that the subsequent sandbox completes the
assignment based on the sorted order. Java sandbox is supported by default in Java Security
Manager (Coker et al., 2015).
3.2.6. Sandbox Business Manager

The requirement will be exerted to handle sandboxes that get and update the information of
the cache of the sandbox existing condition systematically. The sandbox business manager
streamlines fairness in the buffer layers on the client-server interface. The sandbox business
manager will ensure that each sandbox condition is cached in the memory to improve general
system performance (Liang et al., 2017).
3.2.7. Parallel judgment

Java code contains a single static main method after compilation, and during judgement, the
judging application calls the main method to get the output of the software via the reflection
technique of the Java program. The program should consist of several test cases on which
each test case is also autonomous of the others. The thread of the judging program is
integrated with a dynamic cached thread pool of Java codes. Besides, each dynamic cached
thread can be executed on its own and minus any meddling and deadlocks. As such, the
parallel judgment can enhance efficiency in the system execution. Likewise, the cached
thread pool of the system would call back and release the memory in order to streamline
memory functions when the thread is not being used for more than one minute.
3.3. System Sequence Diagram
The following sequence diagram in Figure 7 is used to highlight the communication of the
OJS’s entities for a specific case. The use case illustration shows the sequence of events
required for the entire OJS user to login, participate in a competition and submit codes for
further processing.

Figure 7: System Sequence Diagram

3.4. System Class Diagram

Below is a class diagram of the online judging system and provides a static outlook of the
project implementation. This diagram complies with the software architecture, sequence
diagram, and problem field, as elaborated in previous sections.

Figure 8: Class Diagram

4. System Implementation
 References
Barr, A. (2018). Secure Programming: Writing Testable Code. New Hampshire: Skillsoft
Ireland Limited

Coker, Z., Maass, M., Ding, T., Le Goues, C. and Sunshine, J., 2015, December. Evaluating
the flexibility of the Java sandbox. In Proceedings of the 31st Annual Computer Security
Applications Conference (pp. 1-10).

Chang, W.Y. and Kim, S.S., 2014. Development and application of algorithm judging
system: analysis of effects on programming learning. The Journal of Korean
association of computer education, 17(4), pp.45-57.

Homès, B., 2013. Fundamentals of software testing. London: John Wiley & Sons.

Knodel, J., & Naab, M. (2016). Pragmatic Evaluation of Software Architectures. Cham:
Springer International Publishing: Imprint: Springer https://doi.org/10.1007/978-3-
319-34177-4.

Mizuno, T. and Nishizaki, S.Y., 2014. Distributed Online Judge System for Interactive
Theorem Provers. In EPJ Web of Conferences (Vol. 68, p. 00016). EDP Sciences.

Nidhra, S. and Dondeti, J., 2012. Black box and white box testing techniques-a literature
review. International Journal of Embedded Systems and Applications (IJESA), 2(2),
pp.29-50.

O'Reilly, 2021. Chapter 10. Input/Output Facilities. [Online] Available at:

https://www.oreilly.com/library/view/learning-java/1565927184/ch10.html
[Accessed 3 March 2021].

Langr, J., Hunt, A., & Thomas, D. (2015). Pragmatic Unit Testing in Java 8 with JUnit.
Raleigh, Pragmatic Programmers.

Liang, H., Chen, C., Zhong, X. and Chen, Y., 2017, June. Design and implementation of
online automatic judging system. In IOP Conference Series: Earth and Environmental
Science (Vol. 69, No. 1, p. 012091). IOP Publishing.

Wirawan, I.M., Taufani, A.R., Wahyono, I.D. and Fadlika, I., 2017, October. Online judging
system for programming contest using UM framework. In 2017 4th International
Conference on Information Technology, Computer, and Electrical Engineering
(ICITACEE) (pp. 230-234). IEEE.