Collaborative Learning of

Programming using
Automated Grading System
Contents
1. Introduction
2. Challenges
3. Literature Survey
4. Motivations
5. Research Objectives
6. Conclusion
7. References

2
Introduction
Collaborative learning
 An approach where two or more people can learn or attempt to
learn together
 Knowledge is achieved by interaction of active members
 Interaction is done through face to face conversation or
computer discussions via online forums

3
Introduction(cont.)
Automated Grading System

 A software or application software

 Contains information of students and their grades

 Used for evaluation purposes

 Examples are WEB-CAT, Curator, Praktomat, ASSYST etc

4
Introduction(cont.)
Advantages of automated grading system:

 Reduce workload of faculties

 Save time and money

 Helps in the evaluation process

 Capable of executing text cases written by students and faculties

 Faster and efficient in information processing

 Assess time and space complexity of algorithms implemented by the

evaluated programs
5
Challenges
 In certain situation it can be very rough and strict

 Not an exact scoring system

 Not an accurate representation of the performance

 To find out the actual information of student’s gained

knowledge

 To find out which student needs more special attention

6
Literature Survey
 E. Stankov et al. proposed a new model for collaborative learning
of programming using source code similarity detection.[1]

 Introduced a hybrid approach for source code similarity

 Introduced a model for semiautomatic source code assessment [3]

 Introduced MENDO System[2]

 Proposed a new model for collaborative learning of programming

7
Literature Survey(cont.)
Hybrid approach for source code similarity detection

Set of
source code

EMAx tool
Parse tree

calculate metrics
Attributes

Data mining clustering method

Clusters

8
Literature Survey(cont.)
EMAx tool


Used for C++ source code analysis


Key modules of the tool:
1. Parser: It generates an abstract syntax tree (AST) for a given source code

2.Executor:It takes AST and gives a simulation view of the possible

execution of source code

3. Attribute extraction module: It creates a vector of integer values from AST

built by executor

4.Auxiliary module: It is used for handling user interaction, displaying

results, exporting data in various formats etc
9
Literature Survey(cont.)
EMAx tool

Fig 2: High architectural view of EMAx tool

10
Literature Survey(cont.)
Semiautomatic source code assessment

Set of source
code

Grades are assigned

Attributes
Data mining clustering method

Clusters of similar
source code
Fail in any test case pass all the test cases

Eligible for
Reject
reassessment

11
Literature Survey(cont.)
MENDO System

 Developed by integrating the previous models

 A web based e-learning system that enables communication,

learning form lectures

 Tasks can be solved with automated grading system

 An organization of competitions and programming courses

12
Literature Survey(cont.)
 Features of MENDO System:

MENDO

Tests
Training

Training Feedback
contents (video Results
(particular
lectures) user) (grades)

13
Literature Survey(cont.)
Collaborative learning of programming

 Data clustering method is used to enhance the features of MENDO system

 The clustering method is used on received correct solutions

 Newly submitted solution (incorrect solution) is assigned to one of the

clusters

 On user demand, correct solutions that are most similar to the submitted
solution will be provided

 A feedback is also provided to the user

14
Literature Survey(cont.)
Collaborative learning of programming

Fig 1. The new process of solving and grading tasks on MENDO

15
Literature Survey(cont.)
Clustering method

 An appropriate data clustering method (EM clustering) is

applied on the dataset

 An appropriate clusters of similar correct source code can be

obtained as a result

 Each correct solution is assigned to a cluster so that the dataset

can contained additional information

16
Literature Survey(cont.)
 10 clusters have been selected for experiment

 Main goal is to determine correct solutions against incorrect

solution submitted by a particular user and build dataset

 For each set of source code, two datasets can be obtained

i. One dataset can be obtained when seeking for 10 clusters
with EM
ii. Another dataset can be obtained when the number of
clusters was determined by clustering method

 To achieve that goal two Bayesian classifiers have been used

17
Literature Survey(cont.)
Instance Prob. of Prob. of Prob. of User Submission Correct
No. cluster cluster cluster No. NO. (Y/N)
0 1 2
1 0 0.001 0.999 1 1 N
2 0 0.002 0.998 1 2 N
3 0 0.002 0.998 1 3 Y
4 0 1 0 2 1 N
5 0 1 0 2 2 Y
6 0 0.001 0.999 3 1 N
7 0 0.002 0.998 3 2 Y

Table 1. The result of testing the first classifier on the test set format of 10 packages of
solutions

18
Literature Survey(cont.)
Inst. Prob. of Prob. of Prob. of Prob. User Submiss. Correct
No. cl. 0 cl. 1 cl. 2 of cl. 3 No. No. (Y/N)

1 0 0 0 1 1 1 N
2 0 0.002 0.996 0.002 1 2 N
3 0 0.002 0.990 0.008 1 3 Y
4 0 1 0 0 2 1 N
5 0 1 0 0 2 2 Y
6 0 0.001 0.996 0.003 3 1 N
7 0 0.002 0.988 0.010 3 2 Y
Table 1. The result of testing the second classifier on the test set format of 10
packages of solutions

19
Motivations
 To use automated grading system in various aspects like
attendance system, scoring system etc

 To enhance source code similarity detection technique

 To make the automated grading system compatible for different

programming languages

 To enhance the maximum probability of correct solution

method

20
Research Objectives
 Want to work for improvement of the probability of correct
solution

 Want to implement the automated grading system with

different clustering methods

 Want to make the automated grading system more efficient

21
Conclusion
 The automated grading system is based on source code
similarity detection, data mining and semiautomatic code
assessment

 Collaborative learning of programming model is based on

MENDO system

 Collaborative learning of programming model used data mining

clustering technique

 It is used to cluster the previously uploaded correct source code

and newly submitted most similar solutions

22
References
[1] E. Stankov, M. Jovanov, B. Kostadinov and A. Madevska
Bogdanova, “A new model for collaborative learning of
programming psing source code similarity detection” in IEEE
Global Engineering Education Conference (EDUCON), pp. 709-
715, 18-20 March 2015.

[2] B. Kostadinov, M. Jovanov and E. Stankov, “A new design of a

system for contest management and grading in informatics
competitions”, in ICT Innovations 2010, Web proceedings, pp.
87-96, 2010.

23
References
[3] E. Stankov, M. Jovanov, A. Madevska Bogdanova and M.
Gusev, “A new model for semiautomatic student source code
assessment”, in Journal of Computing and Information
Technology(CIT), vol. 21, no 3, pp. 185-192, 2013.

[4] E. Stankov, M. Jovanov, A. Bojchevski, and A. Madevska

Bogdanova, “EMAx: Software for C++ source code analysis”, in
Olympiads in Informatics, an International Journal, vol. 7, pp.
123-131, 2013.

24
Thank you

25