An experimental application of Portable Education Portfolios (PEPS) to allow for learning across educational environments:

PROJECT MANAGEMENT STRATEGY

The case of KommGame and openSE

www.opense.net

Partners:
Sociedade Portuguesa de Inovao, SA Prof. Augusto Medina Jos Carvalho (Project Coordinator) e-Mail: josecarvalho@spi.pt URL: http://www.spieurope.eu/

Aristotle University of Thessaloniki Ioannis Stamelos e-Mail: stamelos@csd.auth.gr URL: http://www.auth.gr/home/

Tampere University of Technology Imed Hammouda e-Mail: imed.hammouda@tut.fi URL: http://www.tut.fi/public/

Universidad Rey Juan Carlos Jess M. Gonzlez-Barahona e-Mail: jesus.gonzalez.barahona@urjc.es URL: http://www.urjc.es/

Ecole pour lInformatique et les Techniques Avances (EPITA) Olivier Ricou e-Mail: ricou@lrde.epita.fr URL: http://www.epita.fr/

Free Knowledge Institute Wouter Tebbens e-Mail: wouter@freeknowledge.eu URL: http://www.freeknowledge.eu

Partners:
University of Maastricht / UNU-MERIT Rdiger Glott e-Mail: glott@merit.unu.edu URL: http://www.maastrichtuniversity.nl/ http://www.merit.unu.edu/

University of Oxford Ross Gardler e-Mail: ross.gardler@oucs.ox.ac.uk URL: http://www.ox.ac.uk/

The Open University Patrick Mc Andrew e-Mail: p.mcandrew@open.ac.uk URL: http://www.open.ac.uk/

logo

linkSpace Management Services Gesellschaft m.b.H (representing the European Learning Industry Group) Elmar Husmann e-Mail: huselmar@de.ibm.com URL: http://www.elig.org

An experimental application of Portable Education Portfolios (PEPS) to allow for learning across educational environments:

The case of KommGame and openSE

Document subtitle
Deverashetty veerendra Kumar Terhi Kilamo Goduguluri Veerakishore Imed Hammouda

Department of Software Systems Tampere University of Technology

October 2011

open educational Framework for computer science Software Engineering

1. KommGame
1.1. BACKGROUND
Many online communities use different types of reputation systems to rate, or simply comment, on the contributions and content added by others. The web2.0 is riddled with social reputation aspects in all kinds of services. Reputation itself can be something as simple as a favoriting a users post or rating an online activity or topic with points. These simple models of reputation can be combined to a more thorough and robust model. Farmer and Glass talk about different reputation models in their book [1] and list the variants as: Rating, Favorites and Flags, This-or-That Voting, Points, Reviews and Karma, which is an aggregate of different reputation types. In the educational context the best place to apply reputation is in e-learning, where most of the activities happen online. In [2] the authors discuss the reputation system in the context of elearning. A web based reputation system called SocialX [3] was designed to support collaboration and social aspects of e-learning. In SocialX the students share and exchange of solutions, discuss solutions of exercises through a forum, and participate in project activities. These findings support the idea that social learning and reputation systems can promote learning and enforce the motivation of learner. In the context of Social Learning people can establish and leverage social connections and it helps to accelerate the distribution and sharing of information, content and guidance. People can improve on their performance, progress in their careers, and help others to learn and develop through a social learning approach.

1.2. DESCRIPTION
A social online learning environment intended to both free learners as well as on campus learners at Tampere University of Technology, KommGame, was developed to study the use of reputation as a motivational driver. In KommGame reputation was added as a component to a collaborative online learning environment used to enable learners to work with and learn software development

open educational Framework for computer science Software Engineering

methods mainly in the context of open source (OS). The pedagogical focus of KommGame was thus to support the teaching of community driven software development practices in a safe but realistic environment. Open source relies on an active developer community. As the point of KommGame was to mimic the infrastructure of a real OS community the learning required the students to contribute to the project and to form a course community. The goal was to create a learning environment that would motivate students to make more contributions. The motivational factor was enforced by the addition of a reputation system. As the environment was offered online it was open to any interested learner via the OpenSE initiative. The main focus being in the community driven software development which also supported a large variety in participating learners. The reputation model chosen for the environment was the karma mentioned in [1]. The main activities in OS development were the ones set to gain reputation to the learners. Each activity gained the learner additional karma. The formula for calculating the karma was chosen as shown by the equation 1:

where n corresponds to the total number of contributions. fk is the weight function corresponding to contribution type. Favorites is the number of like bookmarks a content author gets. Weekly Quality Tokens corresponds to the number of time the particular participant was selected as the best quality contributor of the week by the rest of the members of community. The equation sets each contribution a particular weight in the community decided by the course moderator. The final karma value of the participants is the sum of weight times of each contribution. A course on OSS was organized at Tampere University of Technology. KommGame was utilized on the course. The main goal of the course was to give a practical experience of OSS development in an actual open source infrastructure which in turn should give students the ability to participate in real OSS projects. The course was open to both TUT students as well as free learners. Those learners who were participating in a university course met weekly during the week to discuss the week's progress and to vote on the most active learner. The course activities included a

open educational Framework for computer science Software Engineering

presentation and a collection of wiki pages on course topics, an small piece of software developed online as a community and contributing to an existing open source project. Each week the learners attending class would vote on a best contributor and a hat was given to the learner winning the vote. The last part was not compulsory. The karma model was used and it covered the following activities: 1.Creating or editing Wiki pages. 2.Reporting bugs, new feature and improvement requests. 3.Comment on bugs, new feature and improvement requests. 4.Closing bugs, new feature and improvement requests. 5.Thumbs up to wiki content page. 6.Adding hats for best quality contributor of the week. Implementation of the KommGame physically consisted of a content management system Dokuwiki, a bug tracking system Mantis, version control system SVN and a locally implemented karma engine and a graphical user interface. All learners were required to register to the system in order to participate in the game.

1.3. EXAMPLE SCENARIOS

For example, activities a user can do with the infrastructure include editing a wiki page, reporting a bug in bug tracking system, viewing the karma report and viewing an individuals profile. Add content: The system used to share the open content with community is a wiki. Every participant can add open content to the wiki system by providing authentication credentials. Fig 1, shows a wiki edit feature where user is adding content to a wiki page. The user clicks the edit button to add content to the wiki.

open educational Framework for computer science Software Engineering

Fig. 1. Add content to wiki and Favorites bookmark.

Once the content edited is saved, any other user can start editing the page by clicking the edit button again. If any community member likes the content of the wiki page then they can bookmark page as favorite. When some community member bookmarks a wiki page as favorite the author of the page gets karma. Reporting a new request: Fig 2 depicts the interface of the bug tracking system. In this interface we can view all the bugs that are reported so far. If the user wants to report a new bug then user has to click the link Report Issue which is marked with an ellipse in Fig 2. Afterwards the user will be able to report different kind of issues in that interface.

Fig. 2. Bug reporting system user interface.

Viewing the karma report: Participants can view the karma values of all the participants in a graph

open educational Framework for computer science Software Engineering

and table view using the Karma reporting system. Fig 3 shows the sample view of karma values and a karma graph. In the graph shows different category of users in different colors the type of user. Each vertical bar in the graph represents the score of each user of the system. Each vertical bar has two parts with different colors, the bottom part indicates the score from the previous week. The upper part indicates the score of the current week. The hat-like icon in some bars indicates that those corresponding users were the best contributors for some week in the past. The names in Fig 3 are blurred out to protect user privacy. The table view contains more details of the user like ID, username, real name and score. The graph and table view can be sorted based on participant's score, ID, real name and username. Each users corresponding profile page can be accessed through the hyperlinks in ID column of the table view.

Fig. 3. Karma reporting interface.

Viewing individuals profile: Participants can have a detailed view of activities done by a community member using the user profile system. Fig 4 shows the sample view of user profile. The left side table shows numerical figures of the participants contribution, on the right side we can see the blog that is maintained by the participant. This view is publicly available to anyone. When a course coordinator logs in two more buttons are seen Add a hat and Remove a hat. If the current user

open educational Framework for computer science Software Engineering

is the best quality contributor of the week, he will be given a hat. The remove button is used to eliminate human errors.

Fig. 4. Detailed user profile interface.

1.4. EVALUATION
Out of 24 registered students 15 students participated actively in the course. There were three rotations of the KommGame offered for free learners, one of them running together with the course.

1.5. TUT COURSE

There was a slow start in the project initially as students were not used to OSS development, but later as the project progressed there were more contributions from all students. This trend can be observed from the statistics. There were totally 63 issues reported in the bug tracking system, on average each student posted 4 issues. During the first week there were only 3 issues reported and the second week saw 12 reported issues. The count kept on increasing every week. When all of the contributions were evaluated it was found that there were 6 students who were more active than rest of the community. This kind of scenario is also very common in real open source project where 90% of the code is created by very few member of the project. On an average each student

10

open educational Framework for computer science Software Engineering

contributed 40 wiki edits. Most of the students made a similar level of contribution; the average karma value of all the students is 38.5. There were the few very active ones who played a key role in setting the road map for the project. Feedback was collected from the TUT learners. In table 1, the responses are summarized. The course and KommGame in particular gained fairly good feedback. As constructive feedback the students suggested there would be dedicated resources who would act as committers and take care of all the unanswered communications. In the current implementation committers were also participants of the course. Some students felt that the karma system resulted as competition factor rather than a motivational factor. It would be better if the significance of the karma system was explained better in the beginning of the course. Students wanted to modify the way karma is given to wiki edits. The current implementation of the course used centralised revision control system, but many students would have preferred a distributed revision control system instead. The decision making process was very slow during the community game. The students of the course suggested to include a voting system that would make the decision process easier and faster. Students felt that the project done during the course was too small. A simple project was chosen because the aim of the course is to practice open source development and not actual programming skills. The average grade of Community game was a bit over average at 3.65.
Table 1. Feedback questions and summary of response.

Feedback Question Give an overall grade to the course (0-5) What was the best part?

Result Average value is 3.55 1. Working in a open source environment. 2. Community game. 3. Presentation and discussions during presentation. 1. Dedicated committer resource. 2. Better governance framework. 3. Distributed revision control. 4. A more complex project. 5. Modify karma model for wiki edits. Average value is 3.65

What would you improve?

Grade the community game project (0-5)

11

open educational Framework for computer science Software Engineering

1.6. RECOMMENDATION
As a learning environment the KommGame presents a viable option for learning software development processes and community driven development online. Student feedback supports this. Online communities need cohesive factors that help the community to form toward a collaborative environment. The usage of reputation seems to act as such a factor. Some found the reputation as a competitive factor more than a motivational one. However, a study indicates that competition also works as a motivator for many learners. The usability of the system could be improved still and some of the development tools changed. For example many students suggested a distributed version control system over a centralized one. A better environment to enabling online discussion and a better flow of control would be beneficial. As an online environment, an easy method for allotting hats to participants without class participation should be added. Even if the KommGame is an online workspace it seems the learners require additional support and guidance. If a weekly or monthly class room setting is not possible due to the wide distribution of students geographically, a similar setting should be arranged via an online chat. In addition a voting system could be implemented to the online game. A central person or people need to be set as the project owners who take care of the project and keep the communication alive as well as take care of the commits. Further support and visibilty to get an active group of online learners within the same timeframe should also be thought of. So far there is no certification system to grant someone a certificate stating the skills reached for a beginner OSS programmer that could be added to the learners portfolio. In the future KommGame could act as a standard system to issue certificates of OSS basic skills to any learner around the world who has participated and contributed sufficiently. KommGame can also be seen as a tool for teaching software development methods in general, it is not tied to open source alone. OpenSE is framework which provides open source projects and internships to users. As KommGame could be used to provide basic OSS skills to any learner it would be better for openSE new users to in the

12

open educational Framework for computer science Software Engineering

future to gain basc competence on OS through it. KommGame can act as abasic tool to certify whether learners OSS basic skills within OpenSE.

13

open educational Framework for computer science Software Engineering

2. PEPS
2.1. BACKGROUND
The use of education portfolios has started in late 1980s.Portfolios provide tangible evidences of learner achievements. The goal of all education portfolios is to represent and certify different artifacts or learning activities achieved by learners. Portfolio techniques are generally used by the disciplines such as history and science to promote critical thinking which assess learner progress. These portfolios support life-long learning process (LLLP). LLLP is a framework of formal, nonformal, and informal learning activities.

Fig. 5. Framework of Life-Long Learning Process.

With the success of Open Source Software (OSS) development model, there is an increase in the number of job opportunities in the OSS market. It has also motivated learners interests towards open education. Furthermore, some open educational frameworks like openSE (open education framework for computer science and engineering) have emerged to provide different aspects related to open source to learners. Learners are participating in different learning projects and achieving recognitions from the reputation systems in open source education. Peps are portable education portfolios and can be used to collect these learning activities into one place.

14

open educational Framework for computer science Software Engineering

2.2. DESCRIPTION
Peps are portable education portfolios which are used to import learning activities from different learning spaces. They provide an authenticated way to import the learning activities of learners from separate learning spaces. It is a system to provide authenticated or certified details of learners to their portfolio.

Fig. 6. Pep system interacting with different learning spaces.

Peps can import all kinds of learning activities: formal, informal or non formal learning. So it supports LLLP. While importing the details from a learning space a pep system asks the learner to authenticate with their openID. Then pep imports learning activities from the given learning space. So the details obtained are authenticated and thus authenticity for evidences is provided. With the help of pep system user can create different views of his portfolio and can give access to others to view his portfolio. Learners can participate in different learning spaces and can have different credentials for different spaces. It is hence problematic for the learners to remember all the credentials of different learning spaces. This problem is solved by introducing the openID authentication. OpenID is an URL, usercentered, open and decentralized standard for authenticating users. With the help of OpenID

15

open educational Framework for computer science Software Engineering

users do not have to remember the multiple usernames and passwords. In order to login into a system, a new user always has to register and also to different sites. Single Sign On concept, means user logs into the system once and access to all the systems without giving login information again and again. As a solution to SSO, openID can simplify the users operation process and reduce the resource provider overhead. i.e., OpenID has the single sign on procedure to reduce redundant, multiple accounts and passwords. So the openID provides a secure and unified authentication mechanism to improve the anonymity of users.

2.3. ARCHITECTURE
From the architectural point of view the pep system can be divided into the following modules: 1. Pep GUI 2. Learning space module 3. Learning space GUI 4. openID module Figure 7 shows the architectural design of the learning space environment. User interacts with the learning space through a pep GUI. The learning space also provides a GUI for users to authenticate with their openID. If authentication is successful then the user is redirected to the PEP system and also the learning details from the learning space are saved into the PEP database.

Fig.7. Learning space environment Architecture.

16

open educational Framework for computer science Software Engineering

2.4. IMPLEMENTATION
Pep interacts with various learning spaces. In order to interact with learning spaces the user at least needs to have the following details: Address of learning space The administrator of a learning space has to provide an URL which locates the learning space. The user has to provide this URL to the PEP system for PEP to be able to locate the learning space. Identify the user The users have to associate openID with their learning space credentials. When the PEP system locates a learning space the user can authenticate with their openID. If the authentication process is successful the user gets recognized.

Type of web service The pep system provides a sample data web service model. The administrator of a learning space has to provide the learning activity details of users in the same format of the model.

2.5. EXAMPLE SCENARIOS

Example scenarios of activities the user can do with the infrastructure include associating openID to a learning space, login or signup interface to the PEP system, the interface to specify learning space location, the interface to authenticate with a learning space, and the interface to view the imported details. Associate openID: The user has to associate their openID to the learning space credentials. An

17

open educational Framework for computer science Software Engineering

interface is provided by the administrator of a learning space to associate openID.

Fig. 8. Associate OpenID.

Login or Signup interface to pep system: OpenID authentication is used as a login system for PEP. User has to authenticate with his openID. If user logs in for the first time, the data required for sign up is handled with the help of openID.

Fig. 9. Pep system Login interface.

Interface to specify learning space location: The user can get an URL of a learning space provided by administrator. The pep system provides an interface for the user to specify the URL of a learning space. Then the PEP system redirects to the interface of the learning space.

Fig. 10. Interface to specify learning space location by user.

18

open educational Framework for computer science Software Engineering

Interface to authenticate with learning space: An interface is provided to the user to authenticate with openID and then the details are imported to the PEP system. This interface location is specified or given by the user to the PEP system. The PEP system redirects to the specified learning space.

Fig. 11. User interface to authenticate with learning space.

Interface to view imported details: When the user successfully authenticates at a learning space the learning activity details of the user are saved to the PEP database. The PEP system provides a GUI named as PEP viewer which shows all the learning activities.

19

open educational Framework for computer science Software Engineering

Fig.12. Interface to view imported details.

2.6. CASE: openSE AND OSS COURSE OF TUT

OpenSE and the OSS course from TUT are two different learning spaces. These two learning spaces are considered when importing details and viewing then in the PEP system.

2.7. RECOMMENDATION
OpenSE is a learning space where users can contribute to different learning projects. It provides open source projects, mentored internships, and educational games to learners. It allows the users to learn from what others have learned and achieved. The user can learn alone or in collaboration with other learners in openSE space. Learners can get a badge by submitting their report on their learning experience.

20

open educational Framework for computer science Software Engineering

PEPS are a system that collects all the learning activities of users in an authenticated way. It acts like a certification system. And also users can create their own portfolios and give access for others to view them. If PEPS are associated with openSE then openSE can act as a central system where users can learn different activities regarding open source, can do open source projects, collect their activities from different learning spaces and certify users based on their performance etc.

21

open educational Framework for computer science Software Engineering

REFERENCES
[1] R. Farmer and B. Glass: Building web based reputation systems, page 72. O'Reilly Media / Yahoo Press, 2010. [2] S. Wang: Study on E-Learning System Reputation Service, Wireless Communications, Networking and Mobile Computing, 2008. WiCOM '08. 4th International Conference on, vol., no., pages.1-4, October 12-14. 2008. [3] A. Sterbini, and M. Temperini: Social exchange and collaboration in a reputation-based educational system, Information Technology Based Higher Education and Training (ITHET), 2010 9th International Conference pages. 201-207, April 29 2010-May 1, 2010.

22