Vol 10 Iss 1 Feb 2011

ITALICS Volume 10 (1) February 2011
Contents
Editorial – Stephen Hagan, University of Ulster

and Anne Morris, University of Loughborough
Ontology-Based E-Learning Personalisation for Disabled Students in

Higher Education – Julius Nganji, Mike Brayshaw, Brian Tompsett,
University of Hull 1
Virtual Learning Environments for Mobile Learning

Laura Crane, Phillip Benachour and Paul Coulton, Lancaster University 12
Generalised Diagramming Tools with Automatic Marking

Pete Thomas, Kevin Waugh and Neil Smith, Open University 22
Teaching Media and Information Literacy to Postgraduate Researchers

Andrew Whitworth, Steve McIndoe and Clare Whitworth,
University of Manchester 35
Bob’s Project Guidelines: Writing a Dissertation for a BSc in Computer Science

Robert Laramee, Swansea University 43
Building a Software Development Environment to Enrich Learning and Enhance

Employability – John Kerins, University of Chester 55
A Real-World Mobile Interaction Design Task

Praminda Caleb Solly and Paul Matthews, University of the West of England 64
Embedding Inquiry based learning into Programming via Paired Assessment

Sonya Coleman and Eric Nichols, University of Ulster 72
Emergent Requirements for Supporting Introductory Programming

Natalie Coull, University of Abertay Dundee and
Ishbel Duncan, University of St Andrews 78
Python for Teaching Introductory Programming: A Quantitative Evaluation

Ambikesh Jayal, University of Gloucestershire, Stasha Lauria, Allan Tucker and
Stephen Swift, Brunel University 86
Studying First Year Forensic Computing: Managing the Student Experience

David Salt, Harjinder Singh Lallie, Philip Lawson, University of Derby 91
FEBRUARY 2011 VOLUME 10 ISSUE 1
EDITORIAL BY STEPHEN HAGAN, University of Ulster & ANNE MORRIS, Loughborough

University
Welcome to the first issue of Italics 2011. This spectrum of papers carried in this issue covers Flexible Learning
and Learning Environments, Employability, the promotion of media and information literacy skills for postgraduate
students and support for novices undertaking introductory programming modules.
The paper by Nganji, Brayshaw and Tompsett, suggests a semantic approach to achieving reasonable
adjustments to the presentation of e-learning resources to meet the needs of disabled students, drawing on an
ontology of various disabilities encountered in higher education. The process aims to present disabled students
with tailored learning resources relevant and suitable for their individual and specific needs. The paper proposes
a viable assemblage of established architectural components which overall could deliver such a customised
Learning Environment.
The paper by Crane, Benachour and Coulton identifies and discusses infrastructural and sociological barriers
limiting access to learning environments via mobile devices. A survey of network coverage data, mobile widgets
and students’ experiences suggests that a lack of a dedicated mobile application or mobile website coupled with
inept network access impose significant constraints at this time.
The paper by Thomas, Waugh and Smith describes an approach to the generalisation of tools for teaching and
learning the skills associated with modelling with diagrams. The paper briefly describes the existing tools and one
approach to the automatic marking of diagrams. The authors report on their work to generalise both a marking
algorithm and a drawing editor in such a way that revision tools can easily be generated for new domains. They
also report on how they have incorporated their tools into their own institution’s Moodle-based Virtual Learning
Environment.
The paper by Whitworth, McIndoe and Whitworth details an open educational resource designed to develop
media and information literacy skills in postgraduate students. The authors describe the motivations for
undertaking the project, the model of media and information literacy education which they used to create the
materials, how the resource was created, and its evaluation using eight students. The outcome is the provision of
a stand-alone resource that can be completed in 7 to 10 hours of independent study by postgraduate students.
The resource is to be used by Manchester University in early 2011 but the authors are keen for other institutions
to adapt and use the resource and to receive feedback. Of particular interest is the use of the resource across
different disciplines. Links to the live resource are provided in the paper.
Laramee offers a template and guidelines underpinning the mechanics of writing a final year student project
report in Computer Science. These may serve as a useful starting point for students and new project supervisors
alike and are offered openly for adoption and use.
Kerins provides a reflective analysis of the utilisation of Higher Education Innovation funding to establish a
software development team within a department of computer science and information systems to grow industrial
links, and to effect synergy between students, academic staff and external clients in problem solving and in
forging solutions to real-world problems. The author outlines numerous benefits derived from the initiative
including the honing of the employability skills of participating students.
Solly and Matthews presents an overview of a real-world interaction design assignment where the focus was on
encouraging students to explore the conceptual design space to find creative solutions. By providing students
with background information extracted from case studies and with access to a domain expert, the students
developed a deeper understanding of the context. This assignment allowed for the integration of research and
teaching in a way that created an exciting environment where both students and teachers became co-learners.
The paper also discusses the benefits to the students of working with experts drawn from a range of non-cognate
disciplines.
The paper by Coleman and Nichols presents a case study on significant improvements in student attendance and
in assessment mark profiles obtained by pairing students within a first year algorithmic programming module.
Students worked together in pairs and were also assessed in pairs. The authors also captured the students’
views on working and being assessed as a paired unit and noted how the collaborative relationship and peer
support continued beyond the module of study and contributed positively to the overall student experience.
The paper by Coull and Duncan reflect upon the problems associated with learning and teaching first year
University Computer Science programming classes and review the various support tools and techniques which
have been developed to improve student success in this area. The authors propose ten requirements that a
support tool should satisfy and offer these to those interested in selecting among the support tools available or
moving to build a dedicated system.
Jayal, Lauria, Tucker and Swift presents an experimental study wherein the understanding and proficiency of
students studying introductory programming via Java is contrasted with an approach which taught basic
programming concepts and constructs via Python before moving to Java. The study reported a marked
improvement in the performance of those students initially exposed to Python over those using Java throughout
the module. The results as set out are an interesting contribution to the ongoing introductory programming
language debate.
The paper by Salt, Lallie and Lawson presents the experiences of students studying Computer Forensics at the
University of Derby, together with the views of the teaching and technical support staff. The authors strongly
suggests that there is a growing mismatch between students expectations and reality, an underdeveloped ability
to problem solve and communicate findings and unrealistic expectations of the job market. The authors suggest
that Universities need to appraise their activities honestly and within an ethical framework. In introducing digital
forensics courses they suggest that there is a need to rapidly assess the needs of our students and to ensure that
our practices and their expectations align with the reality of the employment market.
ONTOLOGY-BASED E-LEARNING PERSONALISATION FOR
DISABLED STUDENTS IN HIGHER EDUCATION
Julius T. Nganji Mike Brayshaw Brian Tompsett

Department of Computer Science Department of Computer Science Department of Computer Science
University of Hull University of Hull University of Hull
HU6 7RX HU6 7RX HU6 7RX
J.Nganji@hull.ac.uk M.Brayshaw@hull.ac.uk B.C.Tompsett@hull.ac.uk
http://www.hull.ac.uk/php/323567 http://www.hull.ac.uk/php/cssbct
ABSTRACT
The number of students with disabilities in UK higher education institutions increases every year. Delivering
education online is becoming increasingly challenging as institutions encounter some disabilities requiring
adjustments of learning environments. The law requires that people with disabilities be given equivalent
learning experiences to their non-disabled peers through “reasonable adjustments”. Educational institutions
have thus utilised assistive technologies to assist disabled students in their learning, but some of these
technologies are incompatible with some learning environments, hence excluding some disabled students and
resulting in a disability divide. To solve this problem, amongst other solutions, e-learning personalisation has
been used and more recently, this is also achieved using Semantic Web technologies such as ontologies.
Nevertheless, as ontologies are incorporated into learning environments little seems to be done to personalise
learning for some disabled students. This study, in order to bridge the gap, proposes a personalisation
approach based on a disability ontology containing information on various disabilities encountered in higher
education, which can be used to present disabled students with learning resources relevant and suitable for
their specific needs.
Keywords
E-learning, learning environments, Semantic Web, ontology, disability, disability divide, disabled students,
accessibility, personalisation, higher education.
1. INTRODUCTION
Today’s fast advancing information and communication technology (ICT) has changed the way in which we
live and work. Many activities in our daily lives are steadily being driven by such technology, including
distance and online learning. This is a global phenomenon as ICTs pervade many societies worldwide, with
poor and middle income countries also adopting and diffusing these technologies much faster than in high-
income countries (World Bank, 2008). No one wants to be left out of this information revolution. Coupled with
the Web, computers greatly enable learning and teaching and increase output. The benefits are potentially
enormous and hence, many educational institutions are employing these technologies which have greatly
driven the learning and teaching process. Teaching in higher education has thus progressed from the
traditional classroom method (e.g. chalk and talk) to embracing e-learning which involves the use of modern
technology in teaching and learning as the Web is increasingly being used by educational institutions
(Alzahrahni et al, 2008; Gonzalez, 2007).
Although these technologies have facilitated many tasks especially for people without disabilities, they have
also created an unequal access to such services for disabled people. Steyaert (2005) observes that this
digital divide for students with functional impairments is because:
“Web-based higher education is failing to transpose the basic accessibility notions from the physical to the
digital environment, hence, the threat of increasing exclusion”.
The fact that disabled students may be disadvantaged in education due to their disability could account for
poor performance, which necessitates “reasonable adjustments”. However, according to a 2009 research
report by the UK Department for Innovation, Universities and Skills (DIUS, 2009):
“Disabled students are slightly less likely to attain a good degree (first or upper second class) than those who
do not report a disability, and there is evidence to suggest that this persists even after controlling for a range
of other factors.”
ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 1

With the enactment of disability legislations in the UK such as the Disability Discrimination Act (DDA) 1995,
2005 and the Special Educational Needs and Disability Act (SENDA) 2001, educational institutions are
compelled to provide equivalent learning experiences to disabled students with their non-disabled peers. Also,
since the use of the internet grants access to very useful information that can facilitate learning, higher
education institutions need to ensure that students with disabilities have the same access to the internet as
students without disabilities (Wall & Sarver, 2003). Thus institutions need to provide disabled students with
support in their study through accessible learning resources suitable for their needs. Where these students
cannot naturally access these resources due to impairment, appropriate assistive technologies will need to be
employed.
The problem however arises when students cannot access learning materials even with the use of assistive or
adaptive technologies due to incompatibility (Unterfrauner & Weiermair-Märki, 2008). A study of the use of
assistive technologies has confirmed the need for further research in order to make assistive technologies
and learning environments compatible and hence resulting in a unified environment which is both assistive
and a good learning environment (Tompsett, 2008). Regarding the problems encountered by disabled
students when accessing learning environments and in line with Steyaert (2005), Tompsett (2008) identifies
existing challenges as the inability to “control and modify the university environment to accommodate the
needs of some students with rarer conditions”. This inability seems to arise from the fact that such learning
environments were designed inflexibly (Pahl, 2002).
For such systems that cannot be used by some people with impairments, adjustments will need to be made to
meet the needs of specific disabilities (Maguire et al, 2006) and this would often require the services of an
expert. Nevertheless, constant adjustment is costly and time consuming and could also impact negatively on
the student’s performance. The need therefore is for a system that will incorporate the needs of various
disabled students, which will be compatible with assistive technologies, being both assistive and a positive
learning environment, thus promoting inclusivity.
With the Semantic Web (Berners-Lee et al, 2001), there are now new opportunities to build flexible systems
that will meet the needs of disabled students. Disability aware systems could be designed using Semantic
Web technologies, leading to personalised environments that will enable disabled students to have relevant
learning resources and to work independently, with little assistance from a disability tutor. Personalisation of
learning environments therefore presents a solution to inaccessibility of learning environments since it
facilitates information presentation and retrieval. The Semantic Web also facilitates search and retrieval of
information and will be very helpful in personalising learning for disabled students. The Semantic Web in the
near future will conceivably be fully embraced in e-learning delivery, in the same way the current Web has
been employed in teaching and learning. This study therefore aims to personalise learning on the Semantic
Web for disabled students. It begins by briefly examining the need for disability inclusion in higher education,
proceeds to examine the underlying technologies of the Semantic Web and thence to e-learning
personalisation from whence the architecture of an ontology-based e-learning system for disabled students is
presented. This architecture will serve as a foundation from which e-learning personalisation systems for
disabled students using Semantic Web technologies could be built.
2. THE NEED FOR DISABILITY INCLUSION IN HIGHER EDUCATION

Due to several factors, including age, disease and accidents, the number of people with disabilities increases
every year. In the European Union, there are at least 74 million people with disabilities (Votis et al, 2009). This
increased number of people with disabilities is also reflected in higher education as more disabled people
progress to higher education studies. According to HESA (2009) statistics, in the 2007-2008 academic year;
about 27,465 students were known to have a specific learning disability out of the total number of 62,510
students with disabilities in UK higher education institutions. This was up from the 57,750 students with
disabilities in the 2006-2007 academic year. Amongst disabilities encountered in UK higher education are:
autistic spectrum disorder, blindness/visual impairment, deafness/hearing impairment, specific learning
difficulties, mental health/emotional difficulties and mobility difficulties which all need to be addressed as each
one of those affected by these disabilities is part of the society.
Inclusion in the society is a fundamental human right which needs to be respected by everyone. As key
services such as education, banking, travel and shopping to name a few, are increasingly being offered
online, not everyone is able to access these services fully. Since various people with disabilities access the
internet with different technologies, some disabled people need special technology to be able to access
information online. When these technologies are incompatible with online systems, disabled people are
disadvantaged. The consequence thereof is a disability divide (Hollier, 2007) which excludes disabled people
from the use of technology to access online information. In higher education, this gap is evident in the case
where students with special needs cannot study conveniently due to lack of “reasonable adjustments”.
When students study in classrooms, they interact with one another which could later continue online through
social networks. This is very important in social interactions and information sharing as students connect with

others elsewhere, which could be helpful in their studies. Accessible ICTs could therefore play a key role in
inclusion and hence participation in the society. The UK Department for Communities and Local Government
(DCLG, 2008) has suggested a correlation between ICT exclusion and social exclusion thus:
“Digital technologies pervade every aspect of modern society. However these opportunities are not enjoyed
by the whole of the UK population – for example, 17 million people in the UK still do not use computers and
the Internet and there is a strong correlation between digital exclusion and social exclusion”.
Some disabled people are also part of the 17 million people who are missing out of the benefits of ICTs and
hence, new initiatives to include them are necessary. With an increasing interest in using Semantic Web
technologies to design learning environments today (e.g. Nemirovskij et al, 2007; Vargas-Vera & Lytras,
2008), researchers and designers of such environments need to prevent exclusion by considering the needs
of disabled people from the onset. E-learning personalisation should therefore include special needs.
As earlier discussed, disability inclusion in higher education is a legal requirement under the DDA 1995, 2005
and SENDA 2001 and it is also morally correct to consider the needs of disabled people in higher education.
The problems faced by disabled students in higher education are further exacerbated by the vast amount of
information on the Web which in most cases is inaccessible to some disabled people. Information retrieval
using existing techniques are unable to exploit the semantic knowledge within documents which means they
are unable to return specific results (Shah et al, 2002), hence the Semantic Web is motivated by the fact that
ontologies can be used to overcome the limitations of keyword-based search (Castells et al, 2007). The
problems associated with difficulties in retrieving information from the Web can now be solved with Semantic
Web technologies. This is because “the Semantic Web makes information more meaningful to people by
making it more understandable to machines” (Ohler, 2008). When machines understand information, search
of information will return meaningful results. It is worth examining the architecture of the Semantic Web to
understand the technologies that facilitate information retrieval.
3. THE SEMANTIC WEB

Since its conception, the Semantic Web has received a lot of support from the international research
community who are engaging in active research in this domain. The Semantic Web has been defined by its
authors Berners-Lee, Hendler & Lassila (2001) as:
“The Semantic Web is not a separate Web but an extension of the current one, in which information is given
well-defined meaning, better enabling computers and people to work in cooperation”
The Semantic Web will therefore have more meaning, enhancing the current Web, and will also enhance
study for disabled students given the ability to facilitate information retrieval as earlier discussed. It is the fact
that “online content of websites, databases and other resources is accompanied by machine-readable
descriptions that add meaning to the content, and describe the structure and status of the knowledge of that
content” (Carmichael, 2008). Anderson and Whitelock (2004) earlier observed that:
“The capacity of the Semantic Web to add meaning to information, stored such that it can be searched and
processed, provides greatly expanded opportunities for education, simulation and real-time action anywhere
on the distributed network”.
Semantic Web technologies could revolutionize the way people with disabilities access learning
environments, by increasing accessibility and relevance of presented content and hence e-learning
personalisation greatly benefits from this technology. Intelligent agents on the Semantic Web could facilitate
reading for some dyslexic students or could transform information and present it in a format that could be
more suitable for their needs. The Semantic Web has a layered structure as shown in Table 1.
Semantic Web technologies add more meaning to the Web especially with the use of ontologies (Horrocks,
2008). The word ontology has its origin from Philosophy where it refers to the subject of being or existence.
In Computer Science however, Gruber (2009) defines ontology as “a specification of a conceptualisation”.
Also noted Ibid is the fact that “ontologies are used for integrating heterogeneous databases, enabling
interoperability among disparate systems, and specifying interfaces to independent, knowledge-based
services”. Ontologies also play a vital role in personalising web content for both disabled and non-disabled
people. The Web Ontology Language (OWL) which superseded DAML+OIL is widely used today and is a
language for the Semantic Web developed by the W3C Web Ontology Working Group (McGuinness &
Harmelen, 2004). OWL is a revision of the DAML+OIL ontology language and has three sub languages: OWL
Lite, OWL DL and OWL Full. OWL Lite supports users needing a classification hierarchy and simple
constraints. OWL DL supports those users who want the maximum expressiveness while retaining
computational completeness and decidability. OWL Full is meant for users who want maximum
expressiveness and the syntactic freedom of RDF with no computational guarantees.
Table 1. Layers of the semantic web

Trust: This layer focuses on issues of trust on the Semantic Web.
Logic and Proof: This is an automatic reasoning system on top of the ontology layer to make new
inferences.
Ontology vocabulary: The ontology layer provides description for properties and relation between these
properties and resources.
RDF + rdfschema: The Resource Description Framework (RDF) layer enables the representation of
metadata about web resources. It uses URIs to identify web-based resources but also has a graph model
for describing relationships between resources.
XML + NS + xmlschema: The XML layer with its namespace and schema ensures the use of a common
syntax on the Semantic Web. This layer allows structuring of data on the web but does not communicate
the meaning of the data.
Unicode and URI: This is responsible for encoding characters and uniquely identifying resources on the
internet. Unicode is the standard for computer character representation while URIs provide the standard for
identifying and locating resources.

Figure 1: ADOOLES ontology class hierarchy
Ontology editors like Protégé, developed at Stanford University, have been designed to support ontology
modelling via the protégé-frames or protégé-OWL editors. This Java-based platform supports several plugins
amongst which is OWL Viz which makes use of GraphViz for graphically displaying ontologies as seen in
Figure 2. We used Protégé to develop an ontology which could serve as a knowledge base for personalising

learning for disabled students in higher education. The ontology has concepts related to disability, ability,
assistive mechanisms and services, which all form the main classes as seen in Figure 1.
The ontology, Abilities and Disabilities Ontology for Online LEarning and Services (ADOOLES) incorporates
some of the concepts contained in the ADOLENA ontology developed and used by Keet et al (2008) to
enhance the National Accessibility Portal (NAP) and holds vital information regarding various disabilities
encountered in the UK higher education as graphically represented in Figure 2. The ontology will be used for
e-learning personalisation.
The ADOOLES ontology has been designed to incorporate concepts in higher education related to various
disabilities and can be used to integrate learning resources and disability-related information systems. This
ontology contains relevant information for people with disabilities as well as those without disabilities, although
the focus here is on disabled people. Some of the concepts used in the HESA statistics for higher education
relating to disabilities have been incorporated, together with some concepts from the ADOLENA ontology.
ADOOLES uses the range of disabilities described in the HESA statistics for disability such as specific
learning difficulty, blind/partially sighted, deaf/hearing impairment, wheelchair user/mobility impairment, mental
health difficulties, unseen disability and autistic spectrum disorder (Figure 2). A Services class is also
introduced in the ontology, which is lacking in the ADOLENA ontology.
Figure 2: Class Disability of ADOOLES ontology
4. E-LEARNING PERSONALISATION
With the exponential growth of the World Wide Web (WWW), it is increasingly becoming difficult to access
information online. One approach to solve this problem is through web personalisation, which makes it a lot
easier for individuals to access relevant information. Over the years, personalisation of learning environments

has become an important topic in e-learning research. The need for personalisation stems from the fact that
learners have differing needs and preferences. One of the reasons why different learners have different needs
and preferences is because of disability (Nevile & Treviranus, 2006). According to Vallet et al (2005),
personalisation is all the more necessary for situations such as when there are large amounts of available
content, the user has short time, when there are imprecise needs and when new content is available. Web
and Semantic Web personalisation for disabled students could involve:
Content personalisation: where content relevant to the needs of disabled students are displayed to them.
Such needs could also be obtained through the students’ browsing behaviour or their input through forms.
Jeevan and Padhi (2006) have reviewed research in content personalisation which can shed more light on
this type of personalisation.
Link personalisation: where relevant links related to the special needs of disabled students are displayed.
This could be based on their preferences and profile (Longpradit et al, 2008) arising from their disability or
past browsing history.
Navigation personalisation: where the e-learning system is designed to appear in an easy order which
could be easily understood by disabled students and hence easy to browse. Thus, navigation personalisation
facilitates the use of the system (Wang and Yen, 2010). The system’s usability and accessibility requirements
should take into consideration the need to ease retrieval of information and to facilitate browsing through
simple navigation. This will also help improve the usability of the system.
Presentation personalisation: the look and feel of the content would need to be different for each student,
depending on their specific need. Good colour combinations for instance, will be necessary for people with
some form of visual impairment such as colour blindness. This type of personalisation could alleviate
impairments in some group of people such as the older people (Kurniawan et al, 2006). Media could be
semantically enriched with ontologies (e.g. Huang and Eze, 2005), facilitating conversion into formats that are
suitable for specific disabilities. An image for instance could be semantically annotated using RDF which
could be understood by a machine, enabling assistive technologies such as screen readers to interpret their
meaning to a visually impaired user. In the same light, text could be converted into sound for blind users using
a translation component, which could also be beneficial to some dyslexic students who may prefer sound than
too much text. The system under implementation should follow established guidelines such as Web Content
Accessibility Guidelines (WCAG 2.0) by the Web Accessibility Initiative (2008).
In order to personalise learning environments for disabled students in higher education, the following types of
personalisation could be employed:
Rule engines: This is rule-based, allowing rules to be specified (Gao et al, 2010). In e-learning, the
administrator may define some rules, achieved through a management component, which define what content
will be shown for a particular user. For a system to personalise services, the administrator may for instance
define rules which will turn off graphics and present only a text version of the system to a user with visual
impairments or could activate the translation component which will convert the speech to sound for a blind
student or sound to text for a student with hearing difficulties.
Simple filtering: In this type of personalisation, a predefined group of system users such as disabled
students determine the content that would be displayed. For instance, students with disabilities may have
personalised e-learning environment that would enable them to access content in formats that are preferable
for them, based on their specific needs.
Content-based filtering: In this type of personalisation, the content of objects is analysed to determine the
user’s interests. Thus items are personalised based on similarity of content (Ghauth and Abdullah, 2010).
Collaborative filtering: In collaborative filtering (e.g. Leung, Chan and Chung, 2006; Papagelis &
Plexousakis, 2005), the preferences of a group of individuals who have similar interests as they search the
web are used to predict similar content that may be of interest to them. This is then presented to them as they
access the system. A group of students with dyslexia for instance, could be presented with content
appropriately marked and suitable for their needs.
Much work has been done on e-learning personalisation on the current Web (Alonso et al, 2005; Mor &
Minguillon, 2004). As Semantic Web technologies are increasingly being incorporated into learning
environments, there is need for personalisation to be implemented using Semantic Web technologies. Various
researchers have developed systems for personalising learning on the Semantic Web (e.g. Dolog et al, 2004;
Wen & Brayshaw, 2007; Huang et al, 2006; Ghaleb et al, 2006), but very little has been done in the area of e-
learning personalisation for disabled students on the Semantic Web (e.g. Schmidt & Schneider, 2007,
Tzouveli et al, 2008). The work by Schmidt & Schneider (2007) focused on dyslexia, but there are a range of
disabilities which all need to be addressed in designing learning environments. Additionally, this has focused
on school children whereas in higher education, the requirements for building learning environments for
disabled adults would differ.

4.1 Ontology-Based E-Learning System Architecture
In Semantic Web technologies, ontologies could serve as a knowledge base which could be queried with a
reasoner (e.g. Java Theorem Prover) to obtain meaningful results. This could be applied in e-learning
personalisation to retrieve relevant content appropriate for specific disabilities. We have developed an
architecture to personalise online learning for disabled students in higher education using ontologies defined
in the Web Ontology Language (OWL). Our architecture is similar to that developed by Goudos, Peristeras &
Tarabanis (2006) to recommend various public services to citizens who are entitled to them in that a reasoner
(e.g. Java Theorem Prover) is used to query an OWL file which serves as a knowledge base, while the
reasoner invokes the web server (e.g. Apache Tomcat) to return a web page with the results as seen in
Figure 4. In the application developed by Goudos, Peristeras and Tarabanis (2006), users gain access to the
services through a login form on the user interface which collects various information including educational
level, family status, degree and area in which they live in order to recommend appropriate benefits and also
present documents needed for applying for such benefits.

Figure 4: The application architecture used by Goudos et al (2006)
Nevertheless, in our system whose architecture is presented in Figure 5, content specific to education is used.
Additionally, we have made provision for various components that are vital to the functioning of the system as
follows:
Information translation and presentation component: this component enables the disabled student to
obtain content suitable to their specific needs. When the student interacts with the learning environment
through the user interface, they are initially accessing general content. The system identifies the student as
one with special needs and retrieves learning resources specific and suitable for their disability. This is
enhanced by the translation module which translates and presents the resources to the student in an
appropriate format. The translator for instance could collect text and translate them to speech for blind
students or can semantically annotate images, facilitating understanding and description of the visual
resource.
Knowledge representation component: this component represents domain knowledge in a format that
could be easily understood by the system. Also, the reasoner can easily query the knowledge base and obtain
results which could be used to personalise the learning environment. The component also provides a means
for assistive technologies to interact with the learning environment, facilitating compatibility.
Information retrieval component: this component is responsible for interacting with the knowledge
representation component, collecting information and transmitting to the information translation and
presentation component. This is mainly made up of the reasoner which queries the knowledge base,
obtaining some results and the web server which invokes the reasoner, obtaining web pages with the results
and returning them to the student.
Management component: this component has a visual interface which enables administrators such as
lecturers, the VLE manager and systems developer to manage the student’s learning and to solve some
problems they may encounter.

Figure 5: Architecture of personalised e-learning system for disabled students in higher education
4.2 System Functionality

Much of the functioning of the system has already been described under the various components but suffice it
to summarise that when a student logs in from the interface and identifies with a specific disability, the
reasoner queries the disability ontology which is an OWL file; while the web server invokes the reasoner a
JSP page is created, returning content specific to the student’s disability after interaction with the translator
module. This flexible system will allow the student the freedom to work on their own without requiring much
intervention from a disability tutor, but would also be compatible with assistive technology when this is
required.
4.3 Pedagogic Benefits of the System
The proposed system has some pedagogic benefits which may be similar to those of other e-learning
systems. Amongst these benefits are:
Alternative formats of learning materials: with this being an online system, learning resources can be
provided to students in various formats (audio, video, slides, text, etc.), which could be suitable to their needs
as well as their preferred learning styles.
Accommodation of different learning styles: the system allows for materials to be presented in various
formats which therefore accommodates different learning styles. Auditory and visual learners could benefit
from audio, video, as well as textual materials. Additionally, the system could translate information into
appropriate formats to suit the learner. For instance, a learner may use the system to convert textual
information into an audio format which might be more suitable to their needs.
Accessible learning resources: given that the learning resources are accompanied by machine-readable
descriptions implies they will be more meaningful and also adhering to accessibility and usability standards
increases the accessibility of the resources. They could also be easily retrieved through search. This system
will also enable students to access learning materials at anytime and anywhere, which would be convenient
for them to study at their pace.
This personalised system also makes it easier for students to learn, cutting down on potential confusion (Kim,
2007).

4.4 Ethical Issues
In conducting this research, we have taken into consideration various ethical issues. Implementation of this
research has been made within an awareness of disability particularly through a diversity and equality
training. There are no safety concerns with this research; during the testing of the system, consent will be
sought from disabled participants before they take part in the research, just like in other research.
Nevertheless, if evaluation of the system is done heuristically, disability experts would be used.
5. CONCLUSION AND FUTURE WORK

We have proposed an ontology-based e-learning personalisation architecture for building e-learning systems
for disabled students in higher education. The e-learning system presented here should help solve some
disability-related problems encountered by disabled students in higher education, particularly inaccessibility of
learning environments and incompatibility with assistive technologies. Students will thus be able to have
learning resources suitable for their needs when they access the system.
The next phase will be the development of a diagnosis inference engine which will employ dynamically the
ontologies described here in an automated process that will seek to marry learners’ needs with potential
learning solutions which we look forward to being able to present in the future. We will evaluate the suitability
of our ontology as a representation of disability following a “gold standard” based approach (Dellschaft and
Staab, 2008), in which the ADOOLES ontology will be compared with a gold standard, taking into
consideration the WHO International Classification of Functioning, Disability and Health (ICF). The
implemented system will be evaluated heuristically during which experts will judge the system’s compliance
with recognised heuristics.
6. REFERENCES
Alonso, F., Lopez, G., Manrique, D. & Vines, J. M. (2005). An instructional model for web-based e-learning
education with a blended learning process approach. British Journal of Educational Technology, Vol. 36,
No. 2, pp. 217-235.
Alzahrani, S., Ayesh, A., & Zedan, H. (2008). Regionally Distributed Architecture for Dynamic e-Learning
Environment (RDADeLE). Proceedings of the 2008 Conference on Human Systems Interactions. Krakow,
Poland, pp. 579-584.
Anderson, T. and Whitelock, D. (2004). The Educational Semantic Web: Visioning and Practicing the Future
Education. Journal of Interactive Media in Education, 2004(1).
Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The Semantic Web. Scientific American, Vol. 284, No.5, pp.
34-43.
Castells, P., Fernández, M and Vallet, D. (2007). An Adaptation of the Vector-Space Model for Ontology-
Based Information Retrieval. IEEE Transactions on Knowledge and Data Engineering, Vol. 19, No.. 2,
February 2007, pp. 261-272
Carmichael, P. (2008). The semantic web and ‘web 3.0’. In: Selwyn, N. (ed.) Education 2.0? Designing the
web for teaching and learning. [Online] Teaching and Learning Research Programme, Available:
http://www.tlrp.org/pub/documents/TELcomm.pdf [Accessed: 7 January 2011].
DCLG. (2008). Delivering Digital Inclusion: An Action Plan for Consultation. Department for Communities and
Local Government (UK). [Online] Available:
http://www.communities.gov.uk/documents/communities/pdf/1001077.pdf, [Accessed: 10 March 2009].
Dellschaft, K. and Staab, S. (2008). Strategies for the Evaluation of Ontology Learning. In Buitelaar, P. and
Cimiano, P., editors, Ontology Learning and Population: Bridging the Gap Between Text and Knowledge,
pages 253{272. IOS Press.
Dolog, P., Henze, N., Nejdl, W. & Sintek, M. (2004). Personalisation in distributed e-learning environments.
Proceedings of the 13th International World Wide Web conference on Alternate track papers & posters.
New York, USA, pp. 170-179.
DIUS, (2009). Disabled Students and Higher Education. Department for Innovation, Universities and Skills.
[Online] Available: http://www.dius.gov.uk/~/media/publications/D/DIUS_RR_09_06, [Accessed: 5 October
2009].
Gao, M., Liu, K. and Wu, Z. (2010). Personalisation in web computing and informatics: Theories, techniques,
applications, and future research. Inf Syst Front, 12:607-629.
Ghaleb, F., Daoud, S., Hasna, A., ALJa’am, J.M., El-Seoud, S.A. and El-Sofany, H. (2006). E-Learning Model
Based on Semantic Web Technology. International Journal of Computing & Information Sciences, Vol. 4,
No. 2, pp. 63-71.
Ghauth, K.I. and Abdullah, N.A. (2010). Learning materials recommendation using good learners’ ratings and
content-based filtering. Education Tech Research Dev, 58:711–727

Gonzalez, C. (2007). Variation in lecturers’ experiences of teaching undergraduate on-campus courses using
the web. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007.
Nanyang Technological University, Singapore, pp. 333-338.
Goudos, S. K., Peristeras, V., & Tarabanis, K. (2006). A Semantic Web Application for Matching a Citizen’s
Profile to Entitled Public Services. Proceedings of the 2nd International Conference on e-Government
(ICEG 2006). Pittsburgh, USA, 2006.
Gruber, T. (2009). Ontology. Entry in the Encyclopedia of Database Systems, Ling Liu and M. Tamer Özsu
(Eds.), Springer-Verlag.
HESA, (2009). Students and Qualifiers Data Tables. Higher Education Statistics Agency. [Online]Available
http://www.hesa.ac.uk/index.php?option=com_datatables&Itemid=121&task=show_category&catdex=3,
[Accessed: 31 October 2009].
Huang, W. and Eze, E. (2005). Multi-media Semantics Contextualisation for knowledge-oriented e-Learning.
Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT’05).
Hollier, S.E. (2007). The Disability Divide: A Study into the Impact of Computing and Internet-related
Technologies on People who are Blind or Vision Impaired. Ph.D. Thesis., Curtin University of Technology.
Horrocks, I. (2008). Ontologies and the Semantic Web. Communications of the ACM, Vol. 1, No. 12, pp. 58-
67.
Huang, W., Webster, D., Wood, D. & Ishaya, T. (2006). An intelligent semantic e-learning framework using
context-aware Semantic Web technologies. British Journal of Educational Technology. Vol. 37, No. 3, pp.
351-373.
Jeevan, V.K.J., Padhi, P. (2006). A selective review of research in content personalization, Library Review,
Vol. 55 No.9, pp.556-86.
Keet, C.M., Alberts, R., Gerber, A., & Chimamiwa, G. 2008. Enhancing web portals with Ontology-Based
Data Access: the case study of South Africa's Accessibility Portal for people with disabilities. Fifth
International Workshop OWL: Experiences and Directions (OWLED'08). 26-27 Oct. 2008, Karlsruhe,
Germany.
Kim, W. (2007). Starting Directions for Personalized E-Learning, in H. Leung, Q. Li, F. Li, and R. Lau (Eds)
Advances in Web Based Learning – ICWL 2007, pp. 13-19, Springer-Verlag (Heidelberg), 3-540-78138-2
Kurniawan, S.H., King, A., Evans, D.G. and Blenkhorn, P.L. (2006). Personalising web page presentation for
older people. Interacting with Computers, Vol. 18, Issue 3, pp. 457-477.
Leung, C.W., Chan, S. C. & Chung, F. (2006). A collaborative filtering framework based on fuzzy association
rules and multiple-level similarity. Knowl Inf Sys, 10(3): 357-381.
Longpradit, P., Hall, W., Walters, R., Gilbert, L., Gee, Q., and Wills, G. (2008). An Inquiry-led Personalised
Navigation System (IPNS) using multi-dimensional linkbases. New Review of Hypermedia and Multimedia,
Vol.14 Issue: 1 pp. 33-55
Maguire, M. et al. (2006). Design of a virtual learning environment for students with special needs. Human
Technology. Vol. 2, No. 1, pp. 119-153.
McGuinness, D.L., & Harmelen, F. (2004). OWL Web Ontology Language Overview, W3C Recommendation.
World Wide Web Consortium. Available from http://www.w3.org/TR/owl-features/, Last accessed 10
September 2009.
Mor, E. & Minguillon, J. (2004). E-learning personalisation based on itineraries and long-term navigational
behavior. Proceedings on the 13th International World Wide Web conference. New York, USA, pp. 264-
265.
Nemirovskij, G., Egner, M., T. & Eberhard, H. (2007). SWAPS: Semantic Web Approach for Personalisation of
Study. Proceedings of the Seventh IEEE International Conference on Advanced Learning Technologies
(ICALT , 2007). Niigata, Japan, pp. 711-712.
Nevile, L., & Treviranus, J. (2006). Interoperability for Individual Learner Centred Accessibility for Web-based
Educational Systems. Educational Technology & Society. Vol. 9, No. 4, pp. 215-227.
Ohler, J. (2008). The Semantic Web in Education. Educause Quarterly, Vol. 31, No.4, pp. 7-9.
Pahl, C. (2002). Managing evolution and change in web-based teaching and learning environments.
Computers and Education. 40: 99-114.
Papagelus, M. & Plexousakis, D. (2005). Qualitative analysis of user-based and item-based prediction
algorithms for recommendation agents. Engineering Applications of Artificial Intelligence, 18: 781-789.
Shah, U., Finin, T., Joshi, A., Cost, R.S. and Matfield, J. (2002). Information retrieval on the semantic web.
CIKM '02 Proceedings of the eleventh international conference on Information and knowledge
management.
Schmidt, A., & Schneider, M. (2007). Adaptive Reading Assistance for Dyslexic Students: Closing the Loop.
Proceedings of the 15th Workshop on Adaptivity and User Modeling in Interactive Systems. Hildesheim,
Germany, ISBN: 978-3-86010-907-6 (2007).
Steyaert, J. (2005). Web-Based Higher Education: The Inclusion/Exclusion Paradox. Journal of Technology in
Human Services. Vol. 23, No. 1, pp.67-78.
Tompsett, B.C. (2008). Experiencias de enseñanza a estudiantes de informática con
discapacidad en Universidades del Reino Unido, Technologías de la
Información Y las Comunicaciones en la Autonomía Personal, Dependencia Y Accesibilidad (2008),
Fundación Alfredo Brañas, Colección Informática Número 16, pp.371-398.
Tzouveli, P., Schmidt, A., Schneider, M., Symvonis, A., & Kollias, S. (2008). Adaptive Reading Assistance for
the Inclusion of Learners with Dyslexia: The AGENT-DYSL approach. Proceedings of the IEEE
Conference on Advanced Learning Technologies (ICALT 08). Cantabria, Spain, pp. 167-171.
Unterfrauner, E. & Weiermair-Märki, C. (2008). User Requirements for Adult Learners with Special Needs in
Accessible Lifelong Learning. Paper presented at ilearning Forum, Paris, France.
Vallet, D., Mylonas, P., Corella, M., A., Fuentes, J., M., Castells, P., & Avrithis, Y. (2005). A Sematically-
Enhanced Personalisation Framework for Knowledge-Driven Media Services. Proceedings of IADIS
International Conference on WWW/Internet. Lisbon-Portugal.
Vargas-Vera, M. & Lytras, M., D. (2008). Exploiting Semantic Web and ontologies for personalised learning
services: towards Semantic Web-enabled learning portals for real learning experiences. Int. J. Knowledge
and Learning, Vol. 4. No. 1, pp. 1-17.
Votis, K., Lopes, R., Tzovaras, D., Carrico, L., & Likothanassis, S. (2009). A Semantic Accessibility
Assessment Environment for Design and Development for the Web. In: Universal Access in Human-
Computer Interaction. Applications and Services.Springer, Berlin, Germany, pp. 803-813.
Wall, P.S. & Sarver, L. (2003). Disabled student access in an era of technology. Internet and Higher
Education, Vol. 6, No. 3, pp. 277-284.
Wang, M. and Yen, B. (2010).The Effects of Website Personalization on User Intention to Return through
Cognitive Beliefs and Affective Reactions. PACIS 2010 Proceedings.
Web Accessibility Initiative. (2008). Web Content Accessibility Guidelines 2.0. [Online] World Wide Web
Consortium. Available: http://www.w3.org/TR/WCAG20/ [Accessed: 25 October 2010].
Wen, L. & Brayshaw, M. (2007). An Individualized E-Learning Web Services Prototype. International Journal
of System and Information Sciences Notes, Vol. 1, No. 1, pp. 29-33.
World Bank, (2008). Global Economic Prospects: Technology Diffusion in the Developing World. [Online] The
International Bank for Reconstruction and Development/The World Bank. Available: http://www-
wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2008/02/15/000310607_200802151105
11/Rendered/PDF/42097optmzd0REVISED0GEP020081PUBLIC1.pdf [Accessed: 5 November 2009].

VIRTUAL LEARNING ENVIRONMENTS FOR MOBILE LEARNING:
CONSTRAINED BY INFRASTRUCTURAL AND SOCIOLOGICAL
BOUNDARIES?
Laura Crane Phillip Benachour Paul Coulton

School of Computing and School of Computing and School of Computing and
Communication Systems Communication Systems Communication Systems
InfoLab21, South Drive InfoLab21, South Drive InfoLab21, South Drive
Lancaster LA1 4WA Lancaster LA1 4WA Lancaster LA1 4WA
l.crane@lancaster.ac.uk p.benachour@lancaster.ac.uk p.coulton@lancaster.ac.uk
ABSTRACT
This paper describes and discusses two issues which limit the delivery of mobile learning from assisting in
disseminating course and module information to higher education students. The concept of delivering
information to support learning is designed to augment their engagement with their subject areas and
ultimately enhance their learning experience by allowing for increased flexibility in their access to learning
materials. The paper concentrates on both the infrastructural and sociological issues associated with providing
VLE access from a mobile or flexible position. Using mobile widgets, network coverage data and student’s
responses to understand the potential benefits and limitations of using mobile devices to access information
from the VLE. Discovering the majority of students are without a dedicated application or mobile website,
coupled with inept network access this paper investigates the apparent constraints of a promising method of
disseminating information to learners.
Keywords
Mobile Learning, Virtual Learning Environments, Student Engagement, Flexible Learning
1. INTRODUCTION
Mobile learning or ‘M-Learning’ holds the promise to enhance and optimize learning by utilizing pre-existing
information systems and deliver them in a contextual, more meaningful manner. Mobile devices have become
increasingly popular and a number of applications have been developed to aid teaching and learning
strategies (Cheng et al, 2008).The modern or nomadic learner travels interminably between locations and
utilizes mobile technology to access information and learning materials from anywhere and at any time. This
revolution of pre-existing concepts of space, time and educational content into personalized, interminably
connected, and dynamic learning environments is currently in rapid development. Yet, it is the transmission
and delivery of these contemporary learning environments which are imperative to the successful uptake and
adoption of mobile learning. Subsequently, educators and technologists must design delivery systems which
are both effective and appropriate for the students.
With mobile phone penetration in Western Europe currently measured at 130% (Ofcom, 2008) the potential of
mobile devices lies in not only the popularity of the technology, but in the unique properties possessed by the
mobile phone or device. Although originally expressed as an outgrowth and consequence of e-learning, it was
soon defined as being ‘spontaneous, private, portable, situated, informal, bite-sized, light-weight and context
aware’ (Traxler, 2010).Possessing such inherent properties means that education can be delivered and
realized for digital native and nomadic students.
Unfortunately, mobile learning does have its limitations. Deliberations over these shortcomings (Shudong &
Higgins, 2005) cite a number of inherent weaknesses of why mobile learning disappoints on its massive
potential. Most notably, access to internet connections and personal boundaries are mentioned as potential
problems in the rapid adoption of the technology. With the delivery of learning objects being as imperative as
the content itself, discovering the next major development in technology enhanced learning is essential to the

benefit of everyone involved in the education process. Therefore investigation into the confines and
boundaries which exist is important in developing future mobile learning ideas and models.
2. FLEXIBLE LEARNING AND MOBILE LEARNING

Flexible learning is defined as an approach to the education process which is learner centric, impartial to time,
space and the methods of learning and teaching in an efficient networked environment. (Moran & Myringer,
1999). In drawing inherent characteristics from both the distance-based education and campus based
environments; with the concept of the learning materials being delivered to the student using appropriate
technology combined with the need for social interaction and community based networking has crafted a
prerequisite status for flexible learning. Flexible learning is a movement away from a situation in which key
decisions about learning dimensions are made in advance by the instructor or institution, toward a situation
where the learner the range of options from which to choose with respect to these key dimensions. (Collis &
Moonen, 2001). The advancement of information technology and more specifically mobile technologies
ensures an excellent tool for enhancing flexibility and has the potential to accommodate the needs of diverse
groups of students (Choy et al, 2002).
Although it has been more difficult to create a fully comprehensive definition of mobile learning; several
attempts have already been proposed. Originally defined as its differences from traditional and ‘tethered’ e-
learning (Traxler, 2005) the ever developing technology with high mobile device penetration within society has
made the definition increasingly difficult. Although O’Malley (2003) encompasses mobile learning as “Any sort
of learning that happens when the learner is not at a fixed, predetermined location, or learning that happens
when the learner takes advantage of learning opportunities offered by mobile technologies”. As a result,
mobile learning can be considered a part of flexible learning. Perhaps mobile learning can offer the context
dimension to flexible learning. Further, mobile learning can be considered an application example of a
disruptive technology where users are interrupted regularly, where a user will receive information on the move
and within a context. This suggests that mobile learning has more control over the learner, whereas for flexible
learning, the user (learner) has more control in choosing study modes and methods.
The figure below demonstrates the characteristics or features for both flexible learning (Gunn, 2000) and
mobile learning (winters, 2006). As demonstrated the obvious parallel is of the allowance for personalization in
both time and special dimensions for accessing and adhering to learners varying learning schedules and
styles. For mobile learning to become a legitimate solution to the requirements posed by flexible learning, the
limitations which could hamper the delivery of information must be investigated and understood.
Figure 1 – Characteristics of Mobile & Flexible Learning
3. VIRTUAL LEARNING ENVIRONMENTS

The rapid developments within information and communication technology; coupled with students encounters
and subsequent expectations with online environments create new possibilities for educational experiences.
As information technology has undoubtedly formed unparalleled opportunities for social communication and

peer-construction of knowledge, higher education institutions have aimed to align themselves with these
technological advancements and adopted this ‘Webagogy’ (Pettenati et al, 2000).
.
One major archetype of this is with universities virtual learning environments now becoming the natural
medium for most ‘digital native’ students accessing academic support such as announcements or course
materials. With Web 2.0 being as pervasive as it is to Generation Y, with instantaneous communication now
an integrated expectation in their daily lives, it is only a matter of time before this level of interaction is
expected in an educational context. Although virtual learning environments themselves are perceived as a
modern example of the Web‘s influence on educational activity, they have existed in other, more primitive
forms since compulsory formal education began to be introduced in the US and Europe. This ranges from
archaic distance learning using primitive forms of the mail system, the introduction of the television and most
recently now the internet. Therefore, education has always utilized current trends in technology to support
learning. Arguments into the role of institutions and teachers are to facilitate engagement in the materials
presented including ‘navigational aids’ for students to further their understanding of a particular subject
(Pachler et al 2010).Currently, most universities have a comprehensive virtual learning environment which
already utilizes a profusion of technologies in the process of course management and effective delivery of
learning materials. With Web 2.0 functionality now being an intrinsic and progressive element in most
students’ learning environments; there is now an expressed need to mobilize this sharable and interactive
content. It is clear that the latest versions of learning environments often attempt to replicate aspects of other
social media; however it seems that both teachers and learners are supplementing their activities with tools
that sit outside of these institutionally sanctioned environments. One clear example of this vacant replication is
shown in Figure 2. Such a tool acts as a private and social web space to record and share reflections on
student learning, explore learning outcomes and aspirations.
Fig 2. MyPlace: Educational Social Web Based Software
In addition, such a platform encourages higher education students to think about their future careers, and
supports personal development. Even with a blogging feature directly available for the students, many still
utilize social networks and blogging sites away from the university itself. Social software is perceived
differently within an educational context, compared with ordinary personal usage and this discourages
student’s adoption (Small & Morgan, 2008). The use of blogging in an educational context or as part of an
application built within an educational establishment is something that students have not really adopted. Even
the use of RSS or Twitter as technologies where students can get updates or share educational experiences
as part of or built-in within virtual learning environments is not something that students prefer to be engaged
with. This issue is one of the major motivations for this study and raises subsequent hypotheses, in which the
need for the pervasive nature of a ubiquitous environment is necessary in education. Traxler (2010) argues
that students no longer need to connect and consume information at the expense of real life, but can do so as
part of real life as they freely move around the world, using their own personal devices to engage with people,
ideas, and information, perhaps using these same mobile devices to produce, consume, and store content
and conversation. This can be achieved in a mobile network environment or within a wireless campus
network. A wireless network can be based on different technologies such as the work presented by Wang, Ci
Zhan, and Xu (2007) Here, a mobile learning environment based on applying wireless sensor networks to
context-aware learning enabling people to learn at any time and at any place was proposed and described .
What is important is that a cost- effective wireless infrastructure is available to support a learning environment
especially within a university campus. This is one of the main promises for mobile and wireless technology to
support and assist learners, allowing students to efficiently engage with information; and subsequently the

ease of use aims to increase adoption and participation in the learning process. As the Horizon report (2010)
argues the possibilities for mobile devices in learning environments as a whole are endless. The devices
portability allows for the mobile phone to become a store of reference materials and learning experiences, as
well as general-use tools or fieldwork, where they can be used to record observations via voice, text, or
multimedia. Therefore teamed with the known benefits of the VLE, the experience of learning could be
heightened to the benefit of students.
4. INFRASTRUCTURAL CONSTRAINTS
The communication and data transfer possibilities created by mobile technologies can significantly reduce
dependence on fixed locations for work and study, and thus have the potential to revolutionize the way we
work and learn (Peters, 2007). Of course; this entirely depends upon the accessibility and range of wireless
access available to both the staff and students. Higher education institutions have identified the student’s
requirement for mobility by providing wireless LAN access on campus, using such initiatives as ‘Eduroam’
which provides a free, secure, and internationally available roaming access service for the education
community. Unfortunately, with the developments in mobilizing access to education, nomadic learners have
now been transformed into mobile learners, and’ Eduroam’ is effective, but does not entirely encompass the
needs of mobile or flexible learners.
.
Currently, wireless access is configured for nomadic learning practises, with ‘hotspots’ being favoured within
busy, communal buildings or lecture halls. Figure 3 represents the data collected from a focus group of
students regarding their routes around the campus and the network access which is readily available for all
members of the university.
Fig. 3. Chloropleth demonstrating nomadic access and popular campus routes
From the three separate maps of campus above, it can be seen that the wireless LAN access is limited to
buildings. This also conveys a lack of wireless access around the open areas which are used for communal
use by the students, as well as the housing area (accommodation) and therefore a large area which students
use as part of their daily routes on campus.
Image ‘a’ is the university’s own publication which denotes the availability of wireless access across the
campus. These are mostly buildings in which most educational and research activities take place, such as
lecture theatres, laboratories or seminar rooms. Image (b) is a representative simulation which provides a

visualization of access and distribution of the wireless network across the campus. Again, these are closely
matched the first image; but allow for a more realistic radius and range of the provided access. The third
image is a representation of the most popular routes and paths used by the students; but layered over the
chloropleth diagram. This shows the students most popular routes around campus only converge with
wireless access as they walk parallel to the networked buildings. Therefore the access to mobile learning
using wireless LAN is limited due to the infrastructural boundaries which are aimed at nomadic learning, rather
than mobile learning. This creates situations which gaps in access to learning services are present, forcing
students to adapt their learning around a system, rather than the information systems encompassing the
working patterns of the learner; defeating the objective of flexible learning.
The most obvious alternative for access is using the mobile internet itself, although cost can be especially
large for the considerable proportion of overseas students. With an average rate of £1 a megabyte for both
uploading and the downloading data, strategies must be implemented to allow cost-effective and continual
retrieving methods for a fully pervasive learning framework. Of course cost efficiency is an issue, but so is
quality of access. Table 1 demonstrates the quality of access from five separate university campuses
according to each of the major mobile phone service providers in the UK for accessing email, browsing
internet pages and viewing video content. The mean values represent the calculated average value of
coverage provided by each of the mobile network operator’s freely available coverage data.
Campus Mean of Coverage Quality
Campus 1 0.84
Campus 2 0.92
Campus 3 0.84
Campus 4 0.92
Campus 5 0.92
Campus 6 0.96
Campus 7 0.88
Campus 8 0.92
Campus 9 0.88
Campus 10 1
Campus
0.908
Mean
____________________________________________
Table 1 – The average 3G network coverage from ten UK campus based higher education institutions
Although each of the network providers possessed a relatively high level of coverage for each of the ten
university campuses, unfortunately only one campus environment had a full level of coverage for 3G access.
One apparent answer to the problem would lie in higher education institutions purchasing network capacity
from a mobile network operator to provide custom and higher quality mobile services to end-users. Again, the
downside would be significant cost, both in the setup and maintenance of an information distribution channel
which subsists without any expense. But this must be considered and costs for such a system should be
calculated.
Apart from the obvious infrastructure of communications access, the other issue is that of the access of the
content for the virtual learning environments. The major challenge to the utilization of such mobile learning
channels as RSS feeds for mobile learning is that of the paradox which arrives when using feeds directly from
a secure virtual learning environment. With RSS feeds being inherently mechanisms intended to freely inform
the user of recent content modifications, and authorization being the procedure to restrict access to
information (Crane & Benachour, 2010). This pre-existing information infrastructure poses an obstruction in
the access to mobile learning due to security policies of each institution. Although the wireless access is a
major factor in constraining mobile learning, the virtual learning environments structure is also important in

accessing information to support learning. This of course renders the mobile learning experience dependent
on the VLE.
5. SOCIOLOGICAL BOUNDARIES
Although current network access can limit the usage of a mobile virtual learning environment; if the students
feel it is an infringement upon their personal space, the concept of a mobile VLE for supporting learning can
be somewhat invalidated due to not understanding personal contexts of time and space.
For many students, a mobile phone is the ultimate personal device, serving a multitude of functions in their
day to day lives and holding important information. Being the definitive personal device means there is a
strong sense of personal space and limitations to the extent academic practises should intrude. Certain
sociologists even argue that with the evolution of a mobile connected society, “the distinction between public
and private domains should be dispensed with since nothing much of contemporary social life remains on one
side or the other of the divide” (Sheller & Urry, 2003).
In order to attain more information surrounding the sociological aspects which affect the necessity for mobile
learning; a survey was conducted from a collective of 66 students, from a total of twenty different higher
education institutions; with the entirety answering positively to both using a university maintained virtual
learning environment and owning a mobile phone.
Fig. 4. Student’s responses towards VLE & online social networks
From this investigation, the majority of students do not have a dedicated mobile VLE either by a dedicated
application or by a mobile website. Further to this, most of those who answered believed they would use a
mobile VLE; yet the appropriateness of has a larger, more comprehensive and apprehensive range of
responses.
One other boundary which has arisen during the last few years is the use of social networks and Web 2.0 by
higher education institutions to informally improve student engagement away from the VLE. Figure 5 shows
the responses to the integration of VLE and social networks in the delivery of learning content and information.

Fig 5. Student’s responses towards intrusion and integration of VLE & online networks
Interestingly, there is a lack of positivity in the incorporation of external social networks, with the majority of
those who answered finding it less comfortable being connected in this domain. Although a student having
their own phones being transformed into a learning form seems acceptable; the concept of social networks
doing the same seems relatively unacceptable.
6. DESCRIPTION OF THE STUDY

A study was performed using a mixed group of ten first year undergraduate students over a six week period
for a User Interface Design module. Supplementary to this information will be daily revision tips for the
upcoming exams, to act as inducement to sustain active interest in the widget. This time of year was selected
specifically due to it being the last module of the academic year, and therefore the student’s participation and
engagement is notoriously lower than in any of the previous terms. Each of the focus group was equipped with
a Nokia 5800 XpressMusic mobile phone, to keep as their own for the duration of the study. The option of
installing their own Sim card was presented to the students, so they could use the phones as they would their
own. Half of the students had a dedicated university managed RSS Widget installed on their devices, and the
remaining half had a Twitter client readily available to receive information about the module content and
revision tips. No mention of the evaluation criteria or surveying strategies were informed to the students to
ensure they had no real insight into what type of feedback or opinions they held at this stage. This was done
so that they would not be distracted about the type of feedback they would give and focus mainly on the use
of the applications. The participants were informed that if they had any issues or opinions regarding the
project to film them via the phone’s video camera, to gain a realistic and in situ insight on the student’s usage
during this study. If complications developed regarding the use of the widgets or accessing the updates,
assistance was available daily either face to face or by email. The lecturer and module convener leading and
orchestrating the learning material for the module was asked to post exactly the same educational content,
announcements and updates on both the existing virtual learning environments RSS outputs and externally
online via Twitter.
The RSS widget was created in Aptana Studio using a compound of XML, HTML and JavaScript for
deployment and installation on the devices. In order to reduce variation, the colour schemes were emulated
like the Twitter widget, which was a standard client widget; from the Ovi store, the online marketplace and
repository for widgets and mobile applications, similar to that of the Android Market or the Apple Apps store.
Other than this difference, the most obvious was that of RSS requiring a mechanism which would allow
students to respond to any updates they receive. This would provide the RSS service with a half-duplex
system instead of its original simplex setup. This function was created by embedding a form into the widget
which in turn, passes their email and comment into a SQL database; notifying the lecturer or module convener
if any comments have been received. To make certain this was not tipping the balance; this messaging
service only allowed for 140 characters to be sent, in parallel to Twitter messages. Figure 6 shows visual
demonstrations of the consonant widgets for both Twitter and RSS respectively.

Fig.6: Demonstration of Consonant Widgets
7. STUDENT EXPERIENCES
In order to measure the student’s experiences of the mobile learning trials, the participating users were asked
to attend a short, informal debriefing to induce some insights into the mobilized virtual learning environment
content. The students were provided with a blank SWOT analysis and asked to fill in the strengths,
weaknesses, opportunities and threats to supporting mobile learning using the VLE. Figure 7 is the
aggregated results of both the RSS and Twitter group’s thoughts towards the strengths, weaknesses,
opportunities and threats of the system they had experienced.
Fig.7: SWOT analysis constructed by student focus group
The students stated a range of issues, benefits and limitations of the mobile supporting technology. Most
notably, the wireless and network connectivity, teamed with the economic costs which they cited as a threat to
their adoption of the mobile delivery of VLE information. None of the students questioned touched on, nor
discussed the intrusion or disturbance of their personal boundaries. Alternatively, the students mentioned the
aggregation of social network into their mobile delivery channel as an opportunity for the technology. As seen,
there were some differences in the responses between both the official RSS channel and that of the twitter,
social networking channel, which include all four sectors of the SWOT analysis. Ultimately, the problem of cost
efficient wireless access was the most apparent consistent problem.

8. DISCUSSION & CONCLUSION
Investigating the limiting factors in mobile learning will be of interest to both educational technologists and
educators themselves. From the widgets constructed and the subsequent investigation into information
delivery in a mobile learning domain; it can be easily argued that a mobile VLE could be efficiently utilized.
Overall all the students responded positively about the mobile widgets position and the appropriation of their
use during the trials. Although there is no clear leader, students generally responded more positively to the
RSS widget, rather than the Twitter application. As far as limitations and drawbacks of the mobile study, the
data sets recruited and tested were from one department and may have had existing knowledge or
background of a technical nature. Conceivably, using another set of students from an arts or humanities
subject field may provide differing results on the strengths and weaknesses of a mobile VLE information
system. Again, the focus groups used were not interviewed or chosen on any existing or previous mobile
learning background, and may have been exposed to similar systems prior to the study itself. Once more, all
the students were provided with identical mobile phones, although some may have been familiar with the
handset or manufacturers interaction menu system previous to the study. Several directions and research
areas can be considered and investigated from the results of this paper. One major issue for further review is
the coverage and infrastructure of the universities wireless network. The need to create cost effective mobile
information systems directly for students could be considered inane without the necessary internet access
available. Furthermore, the constrains which appear to exist in utilizing externally available Web 2.0 should be
discouraged or discarded due to the student’s own personal space. Ultimately, this paper is only a foundation
to the discussion and discovery of the apparent limitations of mobile learning channelling VLE information.
Identifying and answering the issues raised are imperative for both student engagement and future education
and technological advancements.
9. ACKNOWLEDGMENT
The authors wish to acknowledge the support of Nokia for the real-time hardware which was used for the
development of this project. Again, appreciation must be directed towards Lancaster University’s Information
System Services and Learning Technology Group for their on-going support.
10. REFERENCES
Cheng, G. D., Chang, C. K., & Wang, C. Y. (2008). Ubiquitous learning website: scaffold learners by mobile
devices with information aware techniques. Computers & Education,50 (1), 77-90.
Choy, Sarojni C. and McNickle, Cathy and Clayton, Berwyn (2002) Learner expectations and experiences.
Student views of support in online learning. In: Guthrie, H. (Ed) Online learning. Research readings. National
Centre for Vocational Education Research, Adelaide, pp. 106-122.
Collis, B., and Moonen, J. (2001). Flexible learning in a digital world. London: Kogan Page.
Communication Market Report, OfCom. August 2008
Crane, L. & Benachour, P. (2010). Dissemination of learning services: using RSS for M-Learning. IADIS
Mobile Learning Conference. Porto, Portgual, (pp. 269-273).
Gunn, C. (2000). A guide to flexible learning principles and practices. Auckland, New Zealand: The University
of Auckland, Centre for Professional Development.

Johnson, L., Levine, A., Smith, R., & Stone, S. (2010). The 2010 Horizon Report. Austin, Texas: The New
Media Consortium.
Jones, N., Blackey, H., Fitzgibbon, K, & Chew, E. (2010).Get Out of MySpace!. Computers & Education, 54
(3), 776-782.
Pettenati, M, Abou Khaled, O, & Giuli, D. (2000). Technology and pedagogy; developing a webagogy.
Submitted to Journal of Interactive Learning Research.
Traxler, J. (2010). Will student devices deliver innocation, inclusion, and transformation?. Journal of the
Research Center for Educational Technology, 31 (2) 129-138.
Traxler, J. (2005). Defining Mobile Learning. Proceedings IADIS International Conference Mobile Learning
2005, Malta, pp 261-266.
Sheller, M. and J. Urry (2003) ‘Mobile Transformations of “Public” and “Private” Life’, Theory, Culture &
Society 20(3): 107–25.
Shudong W., & Higgins, M. (2005). Limitations of mobile learning. The JALT CALL Journal. (2), 3-14.
Small, G., & Vorgan, G. (2008). iBrain. New York: HarperCollins.
Pachler, N., Bachmair, B. & Cook, J. (2010). Mobile learning: structures, agency, practices. New York:
Springer.
M.C. Pettenati, O. Abou Khaled, D. Giuli Technology and Pedagogy; developing a Webagogy, submitted to
Journal of Interactive Learning Research, AACE, March 2000.
Peters, K. (2007). M-Learning: Positioning educators for a mobile, connected future. International Review of
Research in Open and Distance Learning, 8(2).
Traxler, J. (2010). Will student devices deliver innocation, inclusion, and transformation?. Journal of the
Research Center for Educational Technology, 31 (2) 129-138.
Wang, M., Ci, Li., Zhan, P., & Xu, Y. (2007). Applying wireless sensor networks to context-awareness in
ubiquitous learning. Third International Conference on Natural Computation.
Winters, N. (2006) 'What is mobile learning?', In M. Sharples (ed.) Big Issues in Mobile Learning: Report of a
workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative Nottingham: University of
Nottingham.

GENERALISED DIAGRAMMING TOOLS WITH AUTOMATIC
MARKING
Dr Pete Thomas Dr Kevin Waugh Dr Neil Smith

Department of Computing Department of Computing Department of Computing
Centre for Research in Computing Centre for Research in Computing Centre for Research in Computing
Open University Open University Open University
Milton Keynes, MK7 Milton Keynes, MK7 6AL Milton Keynes, MK7 6AL
6ALp.g.thomas@open.ac.uk k.g.waugh@open.ac.uk n.smith@open.ac.uk
ABSTRACT
We describe an approach to the generalisation of tools for teaching and learning the skills associated with
modelling with diagrams. The paper briefly describes the existing revision tools and our approach to automatic
marking of diagrams. We report on our work to generalise both the marking algorithm and the drawing editor
in such a way that revision tools can be easily generated for new domains. We also report how we have
incorporated our tools into our institution’s Moodle-based Virtual Learning Environment (VLE).
Keywords
automatic marking, computer-assisted instruction, computer science education, diagrams.
1. INTRODUCTION
Diagrams are used in many disciplines for communication, recording decisions and as a mechanism for
exploring alternatives. In some fields, formal syntax and semantics for diagramming have been defined. For
example, the Unified Modeling Language, UML, (Jacobson et al, 1998) in Computing is a well-defined notation
for recording design decisions in software development. The UML defines a range of diagram types for
different purposes. Similarly, there are also agreed diagramming standards in Electronics for circuit diagrams
and in Chemistry for molecular diagrams.
Diagrams are ubiquitous in teaching and learning and there are often associated modelling skills that students
should master. Often, students use off-the-shelf diagramming software that would typically be used by
practitioners. While they need such skills in practice, such tools can be daunting to learn. Additionally, they
often constrain the student to (a subset of) the syntax and semantics of the standard language and are not
specifically designed for teaching and learning. We would prefer to allow students to construct diagrams with
little constraint on what they draw and have a tool provide feedback on errors made in the syntax.
Integrating such tools into e-learning and e-assessment systems is problematic. E-assessment systems are
almost exclusively text-based with little support for diagramming. Quiz engines, for example, expect
responses to be textual, particularly if the response is to be automatically marked. While quiz engines can
present the student with a diagram embedded within a question and ask for a response based on hot spots,
for example, there are no facilities for entering a diagram created by the user. Even if there were such a
facility, it is likely that responses would be routed to a human for grading.
In an attempt to address these problems, we have been developing computer-based tools for teaching and
learning graph-based diagram modelling skills (Thomas et al, 2007a). We refer to these tools as revision tools
as they are typically used by students as formative learning aids to revise or practice skills already taught in a
module. These revision tools provide the student with a series of graded questions each of which requires a
diagram to be drawn. The tools incorporate a diagram marking engine that compares a student-generated
diagram with one or more model solutions and grades the student’s attempt (Thomas et al, 2008a; Thomas et
al, 2008b). The tools also provide graphical as well as textual feedback at a variety of levels of detail which
the student can use to improve their answers and hence their understanding. Our intention has been that the
revision tools should be used as formative learning aids for practicing the modelling skills taught in a module.
In our work, we seek a fully automatically generated grade rather than provide support to the human marker
seen in semi-automatic approaches to diagram assessment (Batmaz and Hinde, 2007; Higgins and Bligh,
2006) which are the most common approaches in this field. In this paper, we concentrate more on describing
the learning tools rather than the automatic marker whose high degree of effectiveness, when compared with
human markers, has been addressed elsewhere (Thomas et al, 2008b).

We have built two revision tools: one for entity-relationship diagrams (used in modelling databases) and one
for UML sequence diagrams (Thomas et al, 2008a); both are used in modules and have been well received
by students (Thomas et al, 2007b). They are stand-alone single-user applications.
This work has been a spin-off from our more general interest in understanding diagrams and in particular
imprecise diagrams (such as diagrams that contain errors as is often the case with student productions)
(Smith et al, 2004). In this work, we have recognised that there is a wide range of diagram types which have
similar characteristics to the entity-relationship and sequence diagrams that we have studied to date. This
suggested that we might be able to generalise this work so that the automatic marker and associated drawing
tool can be applied to a wider set of domains. In doing this, we would also be able to cope with the problem of
different notations for the same domain (the classic example is of entity-relationship diagrams for which there
are numerous notations).
If these revision tools are to be helpful to the teacher, we also need authoring tools for the creation of
questions, model solutions and mark schemes that can feed into the teaching tools.
Our work so far suggests that it might be possible to generalise the marking engine, drawing editor and
authoring tool to deal with a variety of different domains and notations. There are examples of generalised
diagram editors but these do not incorporate support for marking. For example, the CIDER project (Jarsen et
al, 2004) takes a description of a diagramming domain in the form of a formal syntax and generates an editor
for drawing diagrams in that domain but does not address automatic marking.
This paper describes how we have approached the problem of generalising our work to multiple domains.
Section 2 discusses the types of diagram that we aim to deal with. Section 3 summarises the significant
aspects of our bespoke revision tools to illustrate the embedded pedagogy. Section 4 summarises our
approach to automatic marking of diagrams. Section 5 addresses the specifics of generalisation and
describes a new tool for specifying diagram types that underpins our new diagramming and marking tools.
Section 6 discusses the exploitation of this work within the context of e-assessment and outlines our plans for
future work.
2. GENERALISATION OF DIAGRAM TYPES

Figure 1 shows examples of different types of diagram that we have been examining (Entity-Relationship,
UML Class, Biological Flow, and UML sequence diagrams, respectively). They are all essentially graph-based
diagrams characterised as sets of boxes (two-dimensional figures) connected by links (piecewise linear). The
Introduces 1 1..*
Hotel Room
0..1 * 0..1
Member
Borrows Owns *
*
Copy Book Guest

HeldAs 0..1
:UserInterface :Hotel :Room : Guest
checkIn(s)
g := identifyGuest( s)
HYPOTHALAMUS r := findAFreeRoom()
accept(g, r)
PITUITARY setOccupied(g)
LH
Oestrogens setRoom(r)
FSH
OVARY
Figure 1 Different types of diagram
links have adornments of a variety of types. Boxes can also have adornments in the form of attributes and
labels, for example.
At its simplest, an entity-relationship diagram (ERD) is a set of entities (represented as boxes) with
relationships between them. The relationships are either explicit in the diagram, being represented by lines
joining boxes, or implicit, being represented by the juxtaposition of entities. In some notations, attributes of
entities are drawn using other shapes that are connected to the entities by lines and constitute another type of
relationship in the diagram.
When one examines UML sequence diagrams (SDs), there is a great deal of similarity to ERDs in that there
are entities (now called objects) which are related by arrowed lines (representing the sending and receiving of
messages between objects). Sequence diagrams also have activations and different types of message. In
other words, a sequence diagram has a small set of types of things (objects and activations) related in
different ways (indicated by different types of arrows and lines).
However, sequence diagrams, as their name suggests, have one very different feature from ERDs: the
messages and their associated activations are in sequences (indicating the passage of time). Sequences are
represented on a diagram by the relative position of activations and messages, with time conventionally
progressing down a diagram from top to bottom.
In general, diagrams may contain several types of objects (see the sequence diagram in Figure 1, for
example) and several types of relationship (as found in UML class diagrams, for example). Relationships do
not have to be represented by drawn links; it is also possible to represent relationships spatially. Figure 2
shows an entity-relationship diagram (ERD) in which some objects are located wholly within another object. In
this case the notation represents the idea of subtyping (Parent and Child are subtypes of Person). We want to
allow assessment of any of these diagram types.
Figure 2 An ERD showing spatial relationships
A further aim is to produce a new question type for an online quiz in which the student would be presented
with a question that requires them to draw a free-form diagram as the answer. The student would be
presented with a drawing tool in the form of an applet with which to create a diagram. The student’s diagram
would be submitted to a marking engine which would compare it with one or more model solutions and apply
a marking scheme (a description of how marks are to be allocated). The marking scheme applicator would
also produce feedback that would be fed back to the student’s browser. This mechanism is illustrated in
Figure 3.
Drawing Tool Diagram Mark Scheme

Comparator Applicator
Applet running in
student’s browser Marking Engine running on server
Figure 3 Drawing tool and marking engine
Thus, there are three tools that have to be generalised: the drawing tool, the diagram comparator and the
mark scheme applicator. In our initial investigations we produced standalone applications which integrated the
three tools into a single revision tool.
However, there is another dimension to this work if the tools are to be effectively used. Suitable questions,
model solutions and marking schemes have to be devised. Therefore, we have built a prototype authoring tool
for this purpose (Thomas et al, 2008a).

3. REVISION TOOLS FOR DIAGRAMS
Our original revision tool was designed to support students learning the modelling skills associated with
ERDs. Figure 4 shows the user interface of this tool. There are three main panes. The left-hand pane contains
a question (in textual form); the top, right-hand pane is for drawing diagrams and the bottom, right-hand pane
is used for identifying feedback based on the model solution.
Figure 4 The ERD Revision Tool

The tool bar towards the top of the window contains a variety of tools for building and editing ERDs. There are
tools for marking a diagram (Mark button) and viewing the model solution (Show/Hide button); although
students are encouraged not to use the latter until they have exhausted all the other teaching elements built
into the learning tool. Once a diagram has been drawn, the student can ask for it to be marked and will be
given a grade and some elementary feedback. Further feedback, in diagrammatic form, can be requested in
which the student’s diagram is compared with the specimen solution and errors identified.
If this feedback is insufficient or the student is lacking in confidence about the answer they have produced,
they can examine the model solution by first revealing the solution using the Show/Hide button and then
clicking on its elements and receiving an explanation of why the chosen element exists and how it relates to
the question scenario.
An important aspect of the ERD revision tool is that the diagram editor allows the student to create syntactic
as well as semantic errors. This enables the tool to provide a wide range of feedback and can explain the
notation when it is applied incorrectly. It is the analysis of imprecise diagrams that distinguishes this tool from
other tools.
Figure 5 shows the user interface for the sequence diagram revision tool. It bears a deliberately strong
resemblance to the ERD revision tool: only the diagram element creation buttons on the right of the tool bar
differ. However, the SD tool allows more latitude in what can be drawn when compared with the ERD tool. It
contains a syntax checking mechanism for student use which not only identifies errors but also suggests ways
in which they can be rectified (not all errors can be automatically corrected as this depends on the student’s
understanding of the question).

Figure 5 The SD Revision Tool
4. DIAGRAM MARKING
The similarities between ERDs and sequence diagrams have encouraged us to look at other types of
diagram. We have examined biological flow diagrams, UML class diagrams and influence diagrams and found
considerable similarities. These were sufficient to convince us that our approach to diagram marking, in which
we analyse a diagram into what we have called its minimal meaningful units (MMUs) and then aggregate
these into more complex meaningful units (MUs) (Smith et al, 2004; Thomas et al, 2008b) before finding
similar structures in two diagrams to determine their overall similarity, is a viable approach in a range of
diverse domains. For example, in ERDs, the types of MMU are entities, relationships and subtypes.
The relationships between the objects in a diagram are the most significant features of the types of diagram
we are investigating. Our approach compares all MMUs of one type in one diagram with all MMUs of the
same type in the other diagram and determines a similarity value for each pair. An optimisation step
determines, for each type of MMU, the most likely overall match between the instances of that type of MMU in
the two diagrams. Using a similarity value allows for imprecise matching, for example when a student makes
a small error in their diagram.
If any MMU in the student diagram remains unmatched with an MMU in the model solution, the context of that
MMU is examined to see whether it could possibly be paired with an MMU in the model solution. Once the
student’s diagram has been matched with a model solution, the marking scheme is applied.
We normally specify marking schemes in terms of the MMUs in the model solution with marks allocated to
each MMU. Marks are awarded to each MMU in the student solution in proportion to the degree of similarity
between it and the MMU in the model solution with which it is paired. If a similarity is below a specified
threshold, no marks are awarded to that MMU.
We explored the quality of our marking algorithm in a series of experiments (Thomas et al, 2007b). These
have provided positive evidence that our approach determines marks which compare well with those awarded
by expert human markers. After these initial experiments we have continued to improve the marking algorithm
and Table 1 shows the comparison between the current version of the automatic marker and human markers
for a corpus of 394 ERDs using our latest algorithm.
The table shows that there was exact agreement in over 72% of cases. However, given that there is variability
in human marking, we feel that a better measure of agreement is where the difference between the automatic
and human-generated mark is no more than 0.5, namely 93% (the diagrams were marked out of 7 and both
the human markers and the auto marker marked to the nearest half mark.). No diagrams differed by more
than one mark.
Diff 0 0.5 1 1.5

No. 284 84 26 0
% 72.1 21.3 6.6 0
Cum’l 72.1 93.4 100 100
TABLE 1 COMPARING THE AUTOMATIC AND HUMAN MARKERS
Figure 6 shows good agreement between the automatic and human markers across the range of possible
marks.
Human vs Automatic Marking

Human v. Auto (Test
Marking (Test set N=394)
Set: N=394)
50
Hum an
Auto
45
40
35
30
Num ber
25
20
15
10
0
0 0.5 1 1.5 2 2.5 3 3. 5 4 4.5 5 5. 5 6 6.5 7
Mar k
Figure 6 Comparison of marks across the range
We also looked at the agreement between markers using the AC1 (Gwet 2001) and Kappa (Fleiss 1971) tests
for inter-rater reliability. We use Gwet’s measure as it accounts for chance agreement between markers.
Table 2 (which first appeared in Thomas et al, 2008b) contains three columns comparing the original marks
from the markers (unmod), the marks following moderation (mod - the gold standard) and the marks produced
by the automatic marker (auto).
Mod-unmod Mod-auto Unmod-auto

AC1 Kappa AC1 Kappa AC1 Kappa
Training 0.479 0.461 0.631 0.619 0.393 0.371
Testing 0.225 0.234 0.306 0.314 0.204 0.213
TABLE 2 INTER-MARKER AGREEMENT
Both statistical measures show a high degree of conformity between markers and, since the measures are all
above 0.15, we can reject the null hypothesis that the marks were awarded randomly. The results show that
the automatic marker agrees with the moderated marks far more strongly than the unmoderated human
markers agree with the moderated human marks. We conclude that the automatic marker marks better than
most human markers. Further analysis of the results shows that the automatic marker awards marks that are
neither consistently above nor below the humans’ marks. A fuller description of these results appears in
Thomas et al (2008b).
The advantage of this approach is that the algorithm can match broken or erroneous MUs in the student
diagram with MUs in the model solution to determine the most likely match. Then, having determined

matching MUs, the differences between the student-produced MU and the matching model solution MU are
the basis of formative feedback.
To generalise the marking approach involves a teacher identifying the MMUs in a new domain and then
specifying the values of a set of weights and thresholds that drive the marking algorithm. Weights determine
the relative importance of features on a diagram and are set by the teacher to match the emphasis required in
the pedagogy being employed. Among other uses, thresholds determine whether the similarities between
instances of MUs are sufficiently large for them to be considered the same and whether marks should be
awarded for an imprecise MU. (Our authoring tool enables the teacher to devise questions, model solutions
and mark schemes, and specify weights and thresholds.)
5. DIAGRAM TYPE SPECIFICATION

To generalise our approach to multiple types of diagram, we want to be able to specify the type of diagram
that specific instantiations of the automatic marking engine and associated drawing tool will operate upon.
This is achieved through a specification tool. The specification tool is designed to enable the teacher to
specify the notation of a type of diagram that will support their pedagogy. For example, a teacher may want
students to work with a non-standard notation or with a tool that constrains the students in some way.
5.1 Underlying Assumptions

The specification tool assumes that the type of diagram to be specified will consist of one or more instances of
the following types of drawing element (in accordance with the generalisation principles outlined above):
• Box;
• Link;
• Spatial relationship;
• Sequence.
A box is a two-dimensional figure (not necessarily rectangular) that can have attributes. An attribute is a text
string which is either drawn and shown attached to a box by a short line or included in a table. There can be
several types of box in a diagram, for example, in a UML sequence diagram there will be boxes representing
objects and boxes representing activations.
A link is a piecewise linear curve joining two other elements (often two boxes, but it can join a box to another
link or join two links). A link can have several types of adornment which can appear at different locations
along the line. Five types of adornment have been identified:
• A label adornment is a text string used to label (or name) either the relationship as a whole or a role
that the relationship can take part in. Labels can be either noun phrases or verb phrases (there are
substantial semantics associated with comparing labels as these play a central role in identifying and
distinguishing many types of relationships (Jayal and Shepperd, 2008));
• A range adornment, of the kind that appears in UML class diagrams, specifies the cardinality
(multiplicity) of a relationship;
• An icon adornment is a small picture denoting some attribute often associated with cardinality; for
example, a crows foot in an ERD specifies the notion of many;
• A line adornment represents the weight and format of the lines(s) used to represent a relationship;
• A string adornment is a text string which is neither a label nor a range.
We have found it beneficial to divide links into two categories: those that are recursive (they link a box with the
same box) and those that are not (they link two different boxes).
A spatial relationship is a relationship between two boxes represented by some property of their relative
positions, for example, that one box is totally enclosed within another box or one box is drawn vertically
beneath another box.
Sequences are collections of objects ordered by some spatial property. Therefore, the tool must be told the
type of objects that are in the collection and the nature of the spatial property on which the objects are
ordered. For example, in a sequence diagram, each object is associated with a collection of messages and
those messages are ordered by their position in the diagram reading from top to bottom.
5.2 The Diagram Specification Tool

The diagram specification tool enables the teacher to define the types of elements and their properties that
will appear on a diagram. For example, when specifying a particular type of link, the specification tool will

Figure 7 Specifying a relationship
reveal a diagram similar to the one in Figure 7.
In Figure 7, an abstract representation of a line that represents a link runs horizontally across the figure and is
composed of a number of buttons (in Figure 7, these are the buttons labelled Unset and Icon). At each end of
the link are four buttons located at positions where adornments can be placed. The middle section of the line
consists of three buttons also labelled Unset, which are associated with specifying the format of the line
(narrow, dashed and so forth).
Two labels can be located in the middle of the line, one above and one below the line, and the generic names
for those labels are entered into the text fields shown in the middle of Figure 7. These generic names appear
in the student interface (for example, Link Label towards the top of Figure 10). The reason for two centrally
located labels is that a relationship can be read in two ways depending upon which of its two entities is taken
as the subject; for example, a doctor attends many patients and many patients are attended to by a doctor.
Pressing a button at either end of the line reveals the menu shown in Figure 8 which allows the selection of
the type of adornment that can be placed at that button’s location.
Having chosen the type of adornment, the tool will, where appropriate, either provide a list of possible values
for that adornment (see Figure 9a) or allow the teacher to input the required values (see Figure 9b).
Figure 8 Choosing the type of adornment
Figure 9b Setting values for a

Figure 9a Choosing an icon range adornment
Through this mechanism, the teacher selects those values that each adornment can take. The lists of values
will be displayed in the drawing tool so that the student can select a particular value to appear in a particular
place on the diagram. It is possible, if required, to include distracter values in these lists as a further test of
student knowledge.

The pull-down menu labelled Connection Ability in Figure 7 allows the teacher to choose the kind of
connectivity between elements: (1) between boxes only, (2) between boxes and between boxes and links,
and (3) between any type of element.
The View button will then show the user interface that will, eventually, be displayed to the student in the
drawing tool as illustrated in Figure 10.
Figure 10, for an entity-relationship diagram using the Chen notation (illustrated in Figure 13), shows that the
student can whether or not to have a Cardinality at each end of the link and, in the case of the line format,
whether it should be a single or double line.
The properties for a box type are set using a form similar to the one shown in Figure 11. The box properties
include: the shape of the box, the standard size for the box, how links are attached to the box (variable or
Figure 10 The student interface for a link Figure 11 Choosing the properties of a box
fixed) and whether attributes, if any, should be displayed using a diagram element (shape) or recorded in a
table.
The specification of a spatial relationship requires just the type(s) of objects involved and the types of their
relationship together with a precision value to be entered. For example, if the relationship exists when one box
is contained in another box the user chooses the relationship Contains to exist between the objects of type
box and enters a value (as a percentage) indicating how much of the inner box has to be within the outer box
for the relationship to be deemed to hold. This is an example of how the tools will allow for imprecision in
drawing a diagram.
Figure 12 Other specification parameters

While the specifier has much to do with the eventual user interface of the drawing tool, it also allows the
teacher to specify other parameters, including certain weights and thresholds, that drive the marking process
(see Figure 12).
The main work in specifying a diagram type is deciding on the types of element to be included, the list of
possible values for these element types and deciding on the weights and thresholds to be used in the tool.
5.3 The Application of the Specification Tool
The specification tool has been successfully used to generate different versions of the ERD revision tool (that
support different notations). The entity relationship diagram in Figure 1 uses the OU’s ERD notation in which
cardinality is indicated by the presence or otherwise of a crow’s foot at the ends of the relationship, and
participation is denoted by small circles also at the ends of the relationship. In the Chen notation, see Figure
13, cardinality is denoted by labels (1 and 1,n) at the ends of the relationship and participation is denoted by
links drawn with one or two lines. The Chen notation includes the relationship’s label within a diamond shape
whereas in the OU notation the label is placed above the line.
Figure 13 An ERD using Chen notation
Using the specification tool we have successfully generated specifications for UML Class diagrams, UML
Sequence diagrams, top-level context diagrams and biological flow diagrams.
6. ONLINE QUIZZES
We have successfully incorporated our technology into the online quiz component of our institution’s VLE
(Moodle). The Open University has invested heavily in the development of the e-assessment features of
Moodle (being the lead developer). In particular, we have interfaced our own quiz module, OpenMark, to
Moodle (Openmark, 2010). This has meant that, in our implementation, a Moodle quiz can consist of question
types taken from both Moodle and OpenMark. OpenMark supports a much wider range of question types than
the base Moodle quiz and allows us to develop new question types with relative ease. To the user, there is no
apparent difference between questions that originate in Moodle or in OpenMark.
Figure 14 A drawing question type in OpenMark/Moodle
Following the scheme outlined in Figure 3, we have been prototyping a new diagram quiz question type for
OpenMark which presents the user with a question requiring a free-form diagram to be created. Clicking on a
button within the question (see Figure 14), the user launches a drawing tool (appropriately tailored to the
question via the specification tool) in the form of an applet. The drawing tool looks very much like the ones in
the ERD and SD revision tools (Figures 4 and 5).
Once the user is satisfied with their diagram, they press a button to submit the diagram to the marking engine.
In a formative setting, once the marking of the diagram is complete, the score for the diagram plus feedback is
returned to the user and displayed in their browser. See: Figure 15.

Figure 15 Question and feedback from the marking engine in OpenMark/Moodle
To support the use of the new diagram quiz question type our authoring tool has been developed to enable
the teacher to construct questions (text), model solutions (diagrams) and marking schemes in a form that can
be plugged into the OpenMark system. Figure 16 illustrates the question development process.
Define
Data Model File Define Question,
Diagram Type
Model Solution,
and Mark
Build Question File

OpenMark Solution and
Test Mark Scheme
File
Java
Drawing Marking
Applet Engine
Figure 16 Question development process using OpenMark and the specification and question authoring
tools
The first step is to define the type of diagram required for the question using the specification tool (assuming
such a specification does not already exist). This produces a data model file which is essentially a set of
parameters. The data model file is input to the authoring tool in order to specialise the authoring tool for that
diagram type. The teacher then builds one or more questions using the authoring tool whose output consists
of two files for each question: a question file containing the text of the question in OpenMark format, and a file
containing the model solution and marking scheme used by the marking engine.
The data model file also plugs into the drawing applet to specialise it for the type of diagram required by the
question. In this way, the output from the specification and authoring tools specialise the drawing tool and
marking engine.

7. CONCLUSIONS AND FUTURE WORK
In this paper we have discussed only the e-assessment part of our research into diagram understanding. To
date we have a framework for the kinds of diagrams we believe our approach is applicable to: relaxed graph-
based diagrams. We have working versions of two revision tools, one for ERDs and one for sequence
diagrams. The revision tools are used in formative assessment and make use of a marking tool which
compares favourably with human markers.
In this paper we have discussed our approach to generalising the revision tools so that it becomes possible
for a teacher to create a revision tool for a specific diagram domain and then devise suitable questions for
inclusion into the tool. We have built authoring tools to support the development of questions which are
specific to ERDs and sequence diagrams.
We have a prototype tool for specifying a range of notations for diagrams in domains where representing
relationships between objects is the primary purpose. We have demonstrated that the specification tool can
generate revision tools for different ERD notations in such a way that a generalised revision tool can be
specialised for a given notation. In particular the prototype tool has been used to generate quiz components to
edit and mark diagrams which have been embedded in a Moodle-based VLE which is being used by
undergraduate students. We have also used the tool to specify UML class diagrams and biological flow
diagrams.
While we could go on to develop these tools, our wish is to formalise our work using constraint based
grammars which will provide a basis for greater generalisations.
We have argued that a significant deficiency in current online assessment systems is their lack of support for
free-text and free-diagram question types. Therefore, we have incorporated our diagram teaching tools into
our local VLE (Moodle). Since Moodle is open source, this opens up the possibility that our diagram teaching
tools can be made easily available to the community at large.
8. REFERENCES
Batmaz and Hinde 2007 Batmaz, F. and C.J. Hinde. 2007. “A Web-Based Semi-Automatic Assessment Tool for
Conceptual Database Diagrams”. Proceedings of the Sixth IASTED International
Conference on Web-Based Education, Chamonix, France, March 14-17, 2007, 427—432.
Fleiss 1971 Fleiss J.L. 1971. “Measuring nominal scale agreement among many raters.” Psychological
Bulletin 76(5):378-382
Gwet 2001 Gwet, K. 2001. Handbook of Inter-Rater Reliability, Gaithersburg: StatAxis Publishing.
Higgins and Bligh 2006 Higgins, C. A. and B. Bligh. 2006. “Formative Computer Based Assessment in Diagram
Based Domains”. Proceedings of the 11th Annual Conference on Innovation and
Technology in Computer Science Education (ITiCSE, Bologna, Italy, June 26-28, 2006),
98—102.
Iizuka et al 2001 Iizuka, K., Tanaka, J. & Shizuki, B. 2001. “Describing A Drawing Editor By Using Constraint
Multiset Grammars”. In Proceedings of the Sixth International Symposium on the Future of
Software Technology (ISFST 2001), Zhengzhou, China. November, 2001.
www.iplab.is.tsukuba.ac.jp/paper/international/iizukia-isfst2001.pdf
Jacobson et al 1998 Jacobson, Ivar; Grady Booch; James Rumbaugh (1998). The Unified Software Development
Process. Addison Wesley Longman.
Jarsen et al 2004 Jansen, A.R., Marriott, K. and Meyer, B. 2004. “Cider: A Component-Based Toolkit for
Creating Smart Diagram Environments”. Springer Lecture Notes in Computer Science. 2980,
pp415-419.
Jayal and Shepperd 2008 Jayal, A. and Shepperd, M. 2008. "The Problem Of Labels In E-Assessment Of
Diagrams," ACM J. of Educational Resources in Computing, 8(4), 2008. Available from
BURA http://hdl.handle.net/2438/2441.
Openmark 2010 “OpenMark examples”, last accessed 28th October 2010
http://www.open.ac.uk/openmarkexamples
Smith et al 2004 Smith, N, P.G. Thomas and K. Waugh. 2004. “Interpreting Imprecise Diagrams”.
Proceedings of the Third International Conference in the Theory and Application of
Diagrams. March 22-24, Cambridge, UK. Springer Lecture Notes in Computer Science, eds:
Alan Blackwell, Kim Marriott, Atsushi Shimojima, 2980, 239—241.

Thomas et al 2007a Thomas, P.G., K. Waugh and N. Smith. 2007. “Computer Assisted Assessment of
Diagrams”. Proceedings of the 12th Annual Conference on Innovation and Technology in
Computer Science Education (ITiCSE, June 25-29, Dundee, Scotland, 2007), 25—29.
Thomas et al 2007b Thomas, P.G., K. Waugh and N. Smith. 2007. “Learning And Automatically Assessing
Graph-based Diagrams”. In Wheeler, S. and Whitton, N. (eds). Beyond Control: learning
technology for the social network generation. Research Proceedings of the 14th Association
for Learning Technology Conference (ALT-C, 4—6 September, Nottingham, UK, 2007),
61—74.
Thomas et al 2008a Thomas, P.G., K. Waugh and N. Smith. 2008. “A Revision Tool For Teaching And Learning
Sequence Diagrams”. Proceedings of ED-MEDIA conference (ED-MEDIA, 30 June—4 July
2008, Austria) 5454—5460.
Thomas et al 2008b Thomas, P. Smith, N. and Waugh, K.2008 “Automatically Assessing Graph-based
Diagrams”, J. Learning, Media & Technology, 33(3) pp249-267, 2008.

TEACHING MEDIA AND INFORMATION LITERACY TO
POSTGRADUATE RESEARCHERS
Andrew Whitworth Steve McIndoe Clare Whitworth

School of Education John Rylands University Library CWA
University of Manchester University of Manchester 41 Nutclough, Hebden
Oxford Road, Manchester, M13 9PL Oxford Road, Manchester, M13 9PL Bridge, W Yorks, HX7 8HA
andrew.whitworth@manchester.ac.uk steven.mcindoe@manchester.ac.uk clare.whitworth@manchester.ac.uk
ABSTRACT
This paper reports on a project funded by the HEA-ICS, which sought to create an open educational resource
to help develop media and information literacy skills in postgraduate research students. The resource uses a
distinctive and holistic approach to the teaching of these subjects, which brings together Bruce et al’s ‘Six
Frames of Information Literacy’ model (2007), Egan’s division of literacy into conventional, emergent and
comprehensive (1990), and Whitworth’s objective, subjective and intersubjective domains of value (2009b).
The paper describes the structure and methods of the course, and some results from the summer 2010
evaluation of the draft version. It explains how this evaluation has resulted in the final version of the materials
which are in place on http://madigitaltechnologies.wordpress.com/infoliteracy.
Keywords
media literacy, information literacy, postgraduate research, open educational resource, holistic.
Note
This is a slightly revised and extended version of a paper presented at the HEA-ICS annual conference in
Durham, August 2010. We would like to acknowledge the help that Anne Morris and our (anonymous)
academic and student reviewers gave us in undertaking this project.
1. INTRODUCTION
This paper reports on a project which took place at the University of Manchester and was funded by the HEA-
ICS. The objective was to create a resource which would help develop media and information literacy skills in
postgraduate research (PGR) students. We first describe the project background and the motivations which
lay behind it. We then explore the distinctive model of media and information literacy (hereafter, M&IL)
education in use in the materials, before reviewing how this model has been translated into practice within the
constraints of available resources.
Piloting of the course took place throughout May, June and July 2010, and we report on some results from
this evaluation, before explaining how these informed the final version of the materials. No attempt is being
made here to generalise about the design of IL materials (or online learning generally), but we do try to
establish whether the project met its objectives, which were to create an effective, stand-alone resource that
could be completed in approximately 7-10 hours of independent study by PGR students.
2. BACKGROUND: EMBEDDING M&IL

Information literacy is recognised as fundamental to the health of the information society. UNESCO endorse it
as a basic human right, essential to national development, active citizenship and economic well-being as well
as educational standards (Catts and Lau 2008, pp. 9-11). Its importance is due to a recognition that digital
tools cannot be “readily… assimilated to existing practices of representation and communication… they are
fundamentally changing what it means to communicate, make meaning, think, work and learn” (Beetham et al
2009, p. 14). Hence, traditional notions of literacy must now be enhanced by an additional understanding of
the ways technology changes how we absorb information from the environment, then use it to sustain our own
learning, and that of the communities, organisations and society of which we are part (Whitworth 2009a).

In 2002 the Roberts review, SET for Success, made recommendations for a more systematic approach to
postgraduate and postdoctoral research skills training. This aspect of PhD study is now considered part of the
general research environment in an institution and its departments, and assessed as part of the RAE (now,
REF). As Streatfield et al (2010, p. 231) say, the previous situation was that “there appeared to be an inherent
assumption that most people who had reached the postgraduate student level and beyond would have few
issues, concerns and needs for training in relation to the information they required for their research”.
This is at odds with the reality of postgraduate study. Although the 21st-century learner now exists within a
world saturated by media and ICT - even in developing countries, where there has been a recent, rapid
penetration of mobile communications - OfCom’s Media Literacy Audit (2008, cited in Beetham et al 2009, p.
23) found that “[e]nthusiastic take-up of new media by young people was not necessarily accompanied by an
understanding of how new media content is produced, i.e. by a capacity to read it critically, or play a role in
collaborative co-creation... Their confidence in using the internet is similarly not complemented by critical
thinking or appropriate care in use of web sites, potentially exposing them to risks relating to unsuitable
material or abuse of their personal information.”
These difficulties are compounded for postgraduate research (PGR) students. They must spend more of their
studies working alone, without the filtering that takes place in a classroom. They may often be older and/or
from countries in which either the Internet infrastructure is less developed, censored or otherwise not freely
available, or whose education systems, largely for cultural-historical reasons, may not emphasise critical
thinking. Many are returning to education after a long gap. In any case (Beetham et al 2009, p. 24, citing
JISC/British Library, 2008): “research-behaviour traits that are commonly associated with younger users –
impatience in search and navigation, and zero tolerance for any delay in satisfying their information needs –
are now the norm for all age-groups”.
Despite the recognised importance of M&IL, it has not proven easy to embed it within UK HE institutions,
either at undergraduate or postgraduate level. There are two main reasons for this: limited definitions of IL
(see §3); and difficulties in locating M&IL institutionally.
Beetham et al’s (2009) report developed from the project Learning Literacies for a Digital Age (LliDA), which
undertook an audit of practice across 15 institutions and also included insights from around 40 nominated
exemplars. Though there were many positive things drawn out by their review, they also identified (p. 63)
much scattered, inconsistent provision. Where practice was strong in certain departments or central services
(like libraries), these were “silos”, which found it difficult to disseminate their good practice, and/or make their
work relevant to different contexts. Streatfield et al (2010) reported similar findings. There are also disputes
over the ownership of the agenda. IL (here omitting the ‘M’) has “been strongly influenced by the idea that it is
the province of librarians” (Whitworth 2009a, p. 98). Yet this view does little to connect IL with other literacies
such as media literacy; nor does it easily make connections between the library’s mandate to offer support to
all disciplines, and the needs of students who must work within disciplinary traditions and meet expectations
that differ between subject areas. Also, there is the wide variety of individual research projects and theses to
contend with. Few supervisors become involved with, or are even aware of, the increasingly important
provision of IL skills; indeed, some respondents in the research conducted by Streatfield et al went so far as to
call supervisors a ‘lost generation’: “overtaken by advances in research information fuelled by ICT
developments, and who were not fully aware of the implications of some of these changes, so that they were
not well placed to guide the next generation of researchers towards appropriate help” (Streatfield et al 2010,
p. 237).
Moving from undergraduate study into PGR work has already been recognised as a difficult transition in terms
of information skills (JISC/British Library 2008, cited in Beetham et al 2009, p. 26). Policies vary across
institutions departments as to how PhD students are trained. Could M&IL classes be made compulsory?
Could upgrading be made conditional on performance in a skills unit that may be run in a library: if not, does
the training lack credibility? The supervisor serves as the personal contact between the student and the
expectations of an institution and/or a discipline, but there may be little integration between what a library or
exemplary department can offer, and the professional teaching development of research mentors, as well as
the academic development of their students. We address these questions of integration further below, in §5.4.
3. THE HOLISTIC MODEL OF INFORMATION LITERACY

There is not the space in this paper to discuss this model in detail. Detail will be found mainly in Whitworth,
McIndoe and Fishwick (forthcoming, 2011), Whitworth (2009b) and, in a more general way, Whitworth
(2009a).Nor is there scope to conduct a full analysis of all the many national and international models for
M&IL education, as outlined by Beetham et al (2009, pp. 30-38) and Lau (2007), although note that Whitworth,
2011 will provide a content analysis of a range of IL policy statements and guidelines with reference to the
criteria developed in this section.

However, it is possible to make some criticisms of the general form of M&IL provision. Egan (1990)
categorised literacy in three ways: conventional, emergent and comprehensive, and it is Whitworth’s argument
that these categories can also be applied to contemporary multiliteracies such as information, media or
environmental literacy (Whitworth 2009b, pp. 32-4). Broadly:
• A conventional view of any literacy treats it as skills which can be defined objectively, and then
applied to all situations consistently. The skills are quantifiable and measurable, and the general
policy objective is to increase literacy rates.
• An emergent view of any literacy treats it as personal and subjective, applied by the learner in local
contexts to meet specific needs. Skills are developed through a reflective understanding of these
needs and differ from person to person, and the general policy objective is to increase individual
appreciation of the benefits of literacy.
• A comprehensive view of any literacy sees it as intersubjective, recognising that it can be defined, and
thus wielded as a political tool, to exclude certain groups from informational spaces, and hence be a
source of power. Skills are developed through negotiation, critique and action, and the general policy
objective is to transform practice and challenge the status quo.
Whitworth (2009b, p.28) illustrates the relationship between these three perspectives with the diagram
reproduced as Figure 1. This also connects the model to the “six frames of information literacy” proposed by
Bruce et al (2007). Bruce et al derived the six frames model from research which showed the wide range of
approaches to information and IL education that were in use by their respondents. The five frames mentioned
in the diagram - content, competency, learning to learn, personal relevance and social impact - represent a
progression from functional skills (where to look for information, how to find it effectively), through more
individual and reflective approaches, to a critical, comprehensive view in which IL constitutes the skills needed
to transform the environments of communities. It is important to note that these latter skills depend on, and do
not supersede, functional, conventional ones. Yet the model also suggests that it is right to criticise the
tendency, also noted by Beetham et al (2009, p. 63), to reify and privilege the “skills agenda” and separate it
from “learning and living”. The three domains are interdependent. All three are essential to the development of
fully information literate learners, cognisant not just of how to find information, but how to evaluate it and then
produce and reproduce it, so that informational resources remain open, dynamic, accessible and of good
quality. Whitworth (2009b) shows how omitting any one of these three domains of value from the evaluation of
information will lead to pathologies, which may include counterknowledge (belief systems, which may be
widespread, but which have no scientific basis: Thompson 2008), relativism, or groupthink (the tendency of
individuals to subsume their cognition under that of the group or groups of which they are a part: Janis 1972).
Figure One: Domains of value and approaches to literacy

Hence the existence of a sixth frame of IL, the relational frame, where the aim is to “explore variation” (Bruce
et al 2007, p. 51) in the ways students engage with information. The aim is not to privilege one frame over the
others, but familiarise students with all frames, empowering them to select between techniques as appropriate
to given - and dynamically changing - situations and contexts, to evaluate their needs and performance
against a range of criteria that include not just subjective but objective and intersubjective values as well.
Teaching in the relational frame thus requires a holistic approach (Whitworth 2009b); it is this approach which
has driven the development of the M&IL resource for PGR students.

4. THE PROJECT
The School of Education in Manchester has run the course unit Media and Information Literacy since 2008 as
part of the MA: Digital Technologies, Communication and Education (www.MAdigitaltechnologies.com). This
unit has been designed around the holistic model (Whitworth 2009b; Whitworth et al forthcoming, 2011). It
was cited in the LliDA report as an exemplar of IL teaching, and the only exemplar to address media literacy
alongside IL (Beetham et al 2009, p. 42). Two of the authors of this paper have worked on the unit since it
began, A, Whitworth as the course leader and C. Whitworth as a teaching and development assistant.
In 2009, McIndoe began work on a JRUL project aimed at developing and supporting students’ information
skills at all levels within the University of Manchester, UG, PGT and PGR. As an early step in this work an
audit was conducted of extant practices across the university. This enquiry made the JRUL and the existing
M&IL team aware of one another’s active interest in this sphere.
The winning of funding from the HEA then catalysed a collaboration to convert the existing M&IL materials for
use by a generic PGR audience. The course as it stood was too complex to be useful in this context. It was a
full-semester course with an involved assessment task (compiling a portfolio of M&IL teaching activities),
aimed at students who would go on to teach M&IL skills to others, in a variety of educational settings. Hence,
the first practical objective of this project was to create a pilot version by significantly simplifying the course for
a generic PGR student audience. Materials were converted in March and April 2010.
A selection of M&IL practitioners was then approached to act as academic reviewers of the pilot version,
alongside PGR students from a range of disciplines. The pilot was complete by July 2010, and analysed in the
following ways:
• through e-mail feedback and follow-ups with academic reviewers (who for reasons of anonymity we
should not name directly, but who include some authors whose work has already been cited in this
paper);
• through a focus group conducted with the PGR student reviewers;
• through analysis of Moodle log files, checking records of reviewers’ actual as well as reported
engagement with the materials.
This analysis fed into the final version of the resource. Section 5 discusses the form of the resource in more
detail and presents some results from this analysis.
5. THE HOLISTIC MODEL IN PRACTICE
5.1 Syllabus
Applying the holistic model to the needs of PGR students has led us away from a focus on technology and
information searching skills. Certainly these form part of the M&IL resource, but in accordance with the model
illustrated in Figure 1, the skills are located in a wider setting. The general theme is the role of research, and
the researcher, in a media- and ICT-saturated environment.
The structure and content of the resource are as follows:
• Introduction: why six frames of IL?
• Content frame: What is research? The role of information. Where to look. Citation guidance.
• Competency frame: Information literacy procedures and standards. Credibility of information.
• Learning to learn frame: How information becomes knowledge. How our tendency to seek patterns
and support for existing beliefs can prevent us learning.
• Personal relevance frame: Reflecting on information searching. Scientific method and
counterknowledge.
• Social impact frame: Bad science (Goldacre 2009), and other examples of how the media use and
abuse information and research. Media literacy. The information commons, intellectual property and
enclosure (Hess and Ostrom 2007).
• Conclusion: objective, subjective and intersubjective value, and why each one matters.
Remember that the sixth, relational frame represents the course viewed as a whole; the aim being to have
students move between the other frames as appropriate.
In designing the resource we have been mindful of the need to address employability as much as students’
academic skills. Employability is part of institutions’ strategic vision for student and researcher development,
but nevertheless is rarely addressed in most IL schemes (Beetham et al p. 42). Yet only a limited number of
PGR students actually enter research as a profession following graduation, and by no means all of these do
so in HE (Jeffrey 2010) - and those that stay in HE may not all become researchers. Hence, the content of the
resource has more general application: being relevant to IL skills needed in ‘everyday life’ as well as in the
lab, classroom or research centre.
5.2 Form and methods
The intention was to design the resource such that it could be completed by students in around 7-10 hours of
work, whether in one go, or smaller chunks corresponding to the seven topics listed above, spread over a few
weeks. Each topic contains two audio files, averaging 5-6 minutes in length, which are the principal content
delivery method: there are also some optional readings, additional web-based resources, and three activities
which test participants’ understanding. Note that at this point, the question of how students’ work on this
resource will be accredited has not been resolved, even at Manchester: it is almost certain this will vary from
institution to institution.
Some justification is required. The project brief required the resource to stand alone, that is, not be part of any
wider programme of teaching, and which students could follow independently. Hence there are at present no
collaborative spaces such as discussion boards or wikis included in the resource. Instead, the emphasis is on
content, which may seem to give it a didactic feel. Yet if adopted by other institutions it may be that students
are guided by an instructor, supervisor, or a community of researchers: it may be that students will continue to
work on the resource independently. The resource should be viewed not as a finished course of teaching but
as a base on which may be overlaid other activities, forms of interaction, and so on. Ultimately it could be
embedded in other research support and development activities, such as group seminars or research
methods courses which are discipline-specific and more oriented towards developing communities of
researchers, though note that student reviewers (see §5.3) generally agreed that the resource could stand
alone without such support.
The resource was piloted as a set of Moodle materials, as this was the system in use on the original Media
and Information Literacy resource. However, this proved unsatisfactory when it came to releasing the
resource into the public domain. This is not due to problems with Moodle as such, even if VLEs like this have
been criticised as an “undead” technology (University of Wolverhampton 2009). Instead, the choice was made
for practical reasons, such as removing the need to download very large files (the ‘wrapped up’ Moodle
course coming in at some 140 Mb in size due to all the audio files). The final version of the resource was
created as a stand-alone web site using Wordpress. It can be found at
http://madigitaltechnologies.wordpress.com/infoliteracy. Guidance, for those interested in using the resource,
is available on the site.
There is certainly scope for adopting institutions to incorporate other, more personalisable and/or collaborative
technologies, such as e-portfolios or blogs, in developing research skills. The audio files could be
disseminated as podcasts via iTunes, with supporting downloads for non-audio materials. Again it is stressed
that these are decisions that can eventually be made by adopting institutions. The project specifications
allowed neither the time nor other resources to innovate pedagogically.
5.3 Observations from the pilot
Eight postgraduate students were used as ‘alpha testers’ on the pilot course. These were found by
advertisement and were offered a small fee (£30) as an incentive to follow the course and participate in a
focus group at the end, which was conducted by the development assistant (CW) and recorded. The students
came from several different countries (Uganda, Pakistan, Cyprus and China as well as the UK) and almost as
full a range of disciplines as could be expected considering the number (students came from three of
Manchester’s four faculties, arts, humanities and natural sciences: only the medical school was missing).
They were also diverse in terms of the stage of study: one student was a Masters’ (postgraduate taught)
student, one was about to start on a PhD in September 2010, and others ranged from first to third year PhD
students.
The focus of the evaluation, where the student reviewers were concerned, was on whether the resource met
their needs, and whether its pedagogical methods were effective. Although students were asked to keep
notes on their interactions if they wished, students were given no prior introduction to the resource except in
the most general sense. This was so, as far as possible, their interaction with it would reflect how future
students would first engage with it: that is, knowing it was about Media and Information Literacy, but nothing
else.
General observations made by the student reviewers included the following, which all reflected positively on
the value of IL training generally, and the specific usefulness of the resource:
I was a bit shocked to be honest to have gone through 5 years of university and [prior to completing
course] not actually have a good definition or know what information literacy was. (2nd year PhD
student)

After going through the course it’s much clearer what you mean by information literacy and why it’s
important in our lives. (PGT student)
Before I didn’t really have much idea about [M&IL], but after going through the whole course… I had a
much clear understanding. (1st year PhD student)
I was a little bit confused about what information literacy was. After listening to the audiofiles and
seeing the six frames I have a deeper understanding. It was very helpful. (about to start a PhD)
Student reviewers made more specific comments about particular resources, pointing out when these led to
dead links, or (in the case of some of the audio files) were considered too long: these problems were fixed in
the final release. Some students, particularly the non-native English speakers, found some of the audio
podcasts a little fast-paced, and this was compensated for by including full written transcripts alongside the
audio.
In the focus group, students agreed that the forecast of around 7-10 hours to work through the resource was
accurate, though one of the eight claimed it had taken her 3-4 days of time. These accounts were verified by
checking the Moodle log files. These data also helped us check whether students were viewing all 7 topics
and in what order; and whether supplementary resources were being viewed as well as the audio files.
It is interesting to consider the different behaviours of the different students on the course. This variety gives
support to the idea that the resource is flexible, and can be followed by students in a way that fits their own
preferred learning style.
• STUDENT 1 (PGT student): Topics viewed in order. 3-hour blocks on three consecutive days.
Looked at some, but not all, supplementary resources.
• STUDENT 2 (2nd year PhD): Looked at topic 1; topic 2 a week later; and then bits of topics 3-5
another week later. Did not look at topics 6 & 7.
• STUDENT 3 (final year PhD): Did one topic at a time with 2-3 day gaps between them. Looked at
almost all resources, including supplementary resources.
• STUDENT 4 (1st year PhD): Never looked at anything, though did give us general feedback about IL
needs in the focus group.
• STUDENT 5 (1st year PhD): Did topics 1-3 all on one day but no subsequent interaction.
• STUDENT 6 (2nd year PhD): Did topics 1-3 on two days then waited a month before returning to
topics 4-7. Looked at some, but not all, supplementary resources.
• STUDENT 7 (about to start a PhD): Omitted topics 2-3, and did the rest over two days. Was quite
selective with viewing of resources. Only listened to one audio file.
• STUDENT 8 (2nd year PhD): Did everything on one day and in a seemingly random order, picking
and choosing resources. Omitted two audio files.
Interestingly, students were satisfied that the resource worked on its own and did not need to be embedded
into a wider environment that contained tutors, workshops and so on. Admittedly this is not a fair test of the
comparison - for it to have been, two pilots would have been needed, one with the resource standing alone
and one with accompanying assistance, and compare student opinions at the end, perhaps with pre- and
post-tests. However, it does suggest, at least, that the resource can be effective when used in a stand-alone
manner.
Academic reviewers’ interaction with the materials was not monitored in this way as the interest with them was
more in the general content of the resource, its structure (around the six frames model) and tone. Again,
general feedback from these reviewers was positive - one reviewer going so far as to call the ‘six frames’
structure ‘inspired’ - but reviewers also criticised specific aspects of the material, for example where it was still
redolent of the original taught PGT course instead of being ‘stand-alone’, or when resources needed updating
to reflect recent publications or developments. These were altered in the final release.
5.4 Integration
For both reasons outlined in §5.2 there is a need to release the materials with an understanding of how they
can best be integrated into the research development work already under way in other locations. This paper is
an element of this support. This connects with a general need for universities to consider how they integrate IL
into other aspects of their teaching and research work. Seeing IL as the responsibility of the library can easily
lead to a dislocation here. Including some kind of IL awareness training in the induction of new academic staff
can help, but continuous support is more helpful: the problem is that such CPD work is usually voluntary
except for new academics (Beetham et al 2009, p. 55). Simple provision of new materials is very unlikely to
lead to their adoption, even if they are accompanied by guidance on integration.

An interesting potential solution to this problem has been tried by Vedvik Tonning et al (2010). Their paper
reviewed efforts in a Norwegian university, which brought central pedagogical support services together with
teaching staff to help them integrate IL into new and existing courses they were developing. In a similar way,
then, it may be that the M&IL course could be included both in teacher training for new academics and
ongoing support for supervisors, though once again there is an issue with compulsion. This can only be
resolved in the adopting institution or department, however.
5.5 Evaluation and follow-up
Researching the manner in which this resource is eventually adopted, whether at Manchester or elsewhere,
will be an interesting follow-up project. Will it appeal more to some disciplines than others? How will it be
adapted, and what changes will be made?
The evaluation of the pilot will also need to measure the impact of the resource on students’ behaviour,
though with such a holistic approach, such measures will not all be quantitative and/or easily seen. The focus
group and perhaps follow-up interviews after a few months, could use questions similar to those used by
Whitworth et al (forthcoming) in their evaluation of the original M&IL course: students have been asked before
the pilot started what their views are of information literacy, and further research could check whether these
have changed and expanded after completing the pilot.
6. CONCLUSION
Information literacy is important; that much is broadly accepted. But there remains a risk that it will turn into
another functional skill, with students’ success measured only against quantitative criteria such as how quickly
they can find information and from what (approved?) sources. Such a view will turn students into mere
information-processing machines. But the research student must be creative, autonomous and individualistic.
IL for PGR students must therefore pay respect to the particular nature of research work.
This is the approach we have taken in developing this resource with the help of the HEA-ICS Subject Centre.
The evaluation suggests that the resource can at least potentially meet needs in this area: only through
adoption and use of the resource can the impact of the materials, and of the holistic model of IL more
generally, be tested. Such follow-up work will be ongoing from this point, with the course coming into use in
Manchester in early 2011.
7. REFERENCES
Beetham, H., L. McGill and A.Littlejohn (2009). Thriving in the 21st century: Learning Literacies for the Digital
Age. Glasgow Caledonian University, Glasgow.
Bruce, C., S. Edwards and M. Lupton (2007). Six frames for information literacy education: a conceptual
framework for interpreting the relationship between theory and practice. In Andretta, S., ed., Change and
Challenge: Information literacy for the 21st century, Adelaide: Auslib (pp. 37-58).
Catts, R. and J. Lau (2008). Towards Information Literacy Indicators, UNESCO, Paris.
Egan, K. (1990). Romantic Understanding: The Development of Rationality and Imagination, Ages 8 – 15.
London: Routledge.
Goldacre, B. (2009). Bad Science. Harper, London.
Hess, C. and Ostrom, E., eds. (2007). Understanding Knowledge as a Commons. Boston, MA: MIT Press.
Janis, I. (1972): Victims of Groupthink, Boston, Houghton Mifflin.
Jeffrey, L. (2010). Generic versus subject-specific researcher training, paper presented at LILAC 2010,
Limerick, Ireland.
JISC/British Library (2008), Google Generation
http://www.jisc.ac.uk/whatwedo/programmes/resourcediscovery/googlegen.aspx
Lau, J., ed. Information Literacy: An international state-of-the-art report. UNESCO/IFLA, Veracruz, Mexico.
OfCom (2009), Report of the Digital Britain Media Literacy Working Group,
http://www.ofcom.org.uk/advice/media_literacy/media_lit_digital_britain/
Streatfield, D., D. Allen and T. Wilson (2010). Information literacy training for postgraduate and postdoctoral
researchers: a national survey and its implications, Libri 60, 230-240.
Thompson, D. (2008). Counterknowledge. London: Atlantic.
University of Wolverhampton (2009), The VLE is Undead, at
http://www.learninglab.org.uk/eventdetails.php?Plv=&Pid1=&Pid2=&id=15.
Vedvik Tonning, A. S., T. Rullestad and T. Skagen. (2010). Integrating information literacy within the university
curriculum: Cooperation between University of Bergen Library and the Centre for University Pedagogy. Paper
presented at LILAC 2010. Limerick, Ireland: Librarians’ Information Literacy Annual Conference.
Whitworth, A. (2009a). Information Obesity. Chandos, Oxford.

Whitworth, A. (2009b). Teaching information literacy within the relational frame: the Media and Information
Literacy course at Manchester, Journal of Information Literacy 3/2.
Whitworth, A. (forthcoming, 2011). Information literacy as instrumental progressivism? An international policy
analysis and critique. Paper forthcoming in Library Trends.
Whitworth, A., Fishwick, I. and McIndoe, S. (forthcoming, 2011): Teaching information literacy in a holistic
way, in Teaching Information Literacy Online, ed. T. Mackey and T. Jacobson, Neal-Schuman, New York.

BOB’S PROJECT GUIDELINES:
WRITING A DISSERTATION FOR A BSC. IN COMPUTER SCIENCE
Robert S Laramee
Robert S Laramee
Visual and Interactive Computing
Group
Computer Science Department
Swansea University, UK
r.s.laramee@swansea.ac.u
http://cs.swan.ac.uk/~csbob/
ABSTRACT
These detailed guidelines attempt to explain the mechanics of writing a third year project dissertation in
Computer Science. They serve as a useful starting point for students (and new advisers) in their third or fourth
year of study in the Computer Science Department of a UK-based university. Afterall, no one is born knowing
how to write a dissertation. Yet, there are certain elements, a commonality, that can be found in virtually all
good dissertations. We give our recommendations as to each section a good dissertation consists of as well
as what each section contains. These guidelines are generic and can be customized to fit most projects. The
guidelines are open and free to use at any university or institution without permission.
We believe that the writing can start with the abstract, however, in practice writing usually starts with the
background section. The abstract can be approximately 6-12 sentences. It is a difficult starting point, but it
forces the author to write down a concise description of what it is they’re researching and what the benefits
are. Chances are, if the author can’t write an abstract, then it is not clear in the author’s mind what the project
is about. Of course the abstract will be refined and updated over the lifetime of the project. The abstract can
concisely (1) identify the project topic, (2) identify the benefits and advantages that result (3) and if there is
novelty, describe the novelty of the presented work. If there’s a description of the project in a previously
existing project selection brochure then that can be used as a basis for an abstract. These guidelines are a
starting point only and are not a substitute for an advisor.
Keywords
project guidelines, computer science
1. INTRODUCTION (MOTIVATION)
[included in the Initial Document, Interim Document (if updated), and Final Dissertation]
Although the title of the starting section is “Introduction” it should really be Motivation. In one or two
paragraphs, introduce the topic. This is followed immediately with why this is a useful project, including
possible applications of the work. “What is this dissertation topic good for?” is really the question a good
introduction attempts to answer. After the motivation behind the subject is described, the benefits, and
advantages, and possibly novelty of the dissertation are presented. This is actually where a project could
start. It starts with an idea, and one key to evaluating how good an idea is being able to identify its benefits
and possibly novelty. Each benefit that reading this dissertation provides is identified very explicitly in a
bulleted list. Possible points to mention include:
• Does the presented work provide a software tool of some sort?

• Does the presented system provide interactive features?
• Who will potentially benefit from the work?
• Does the tool or system provide new insight in some way?
• What does the reader potentially gain from reading this dissertation?
Words like, contribute, benefit, advantageous, and possibly novel are used in this list. An explicit bulleted list
is used in order to make the subject black and white to the reader otherwise, a reader might not understand
the dissertation. The goal of the project has to be carefully considered in this context. Describe the goal of the
project here.

The presented work often builds on a previous system or algorithm. If so, your work may inherit benefits from
the previous work. Those inherited advantages may also be listed.
In order to achieve the amazing benefits your project has to offer, specific challenges must be overcome, both
conceptual and technical. Those challenges can be mentioned here.
The introduction section then concludes with how the rest of the dissertation is organized. Section 2 presents
previous work on this topic. Section 3 presents the project specifications. Section 6 presents the project
design. Section 7 presents our system or method starting with an overview. Section 8 presents an evaluation
of our results both from a performance point of view and memory point of view. Section 9 completes the
exposition with conclusions and future work.
1.1 Formatting:
The format of the dissertation can be very similar to this document: 11 or 12 point Times (or similar) font,
single column. Excessive white space is to be avoided. Some authors try to pad their dissertation with a lot of
unnecessary white space. Sometimes there are blank pages, or pages that are only half occupied with
content. This is a waste. Don’t start every new section or subsection on a new page in order to make the
documentation appear longer. The goal of a dissertation is not to produce the thickest document possible, but
rather to present good quality. If it looks like too much padding has been used, this may even reduce the final
score of the dissertation.
1.2 On LaTeX:
The best software for writing papers, dissertations, theses, and books is LaTeX. LaTeX produces the highest
quality, most professional-looking output. See Kopka and Daly for an excellent LaTeX reference guide Kopka
and Daly (2004). This is one reason why virtually all conferences and journals provide LaTeX templates for
paper submissions. You can use the LaTeX source from this guideline document if you think it would help.
Simply contact the author. Also, http://www.latex-project.org/ is a great online-guide to LaTeX.
1.3 Project Titles:

Regardless of the title of your dissertation, it will only be referred to using two to three words. For example,
“Marching Cubes: A High Resolution 3D Surface Construction Algorithm” is referred to simply as ”The
Marching Cubes” paper Lorensen and Cline (1987). “Visual Analysis and Exploration of Fluid Flow in a
Cooling Jacket” is referred to as “The Cooling Jacket” paper Laramee et al. (2005a). On one hand the title of
your project is ideally simple enough such that it can easily be remembered and referred to with two or three
words. On the other hand, a title is also long enough in order to convey some sort of meaning and description
to the reader. This problem can usually be resolved using a title:subtitle combination, like in the case of
Marching Cubes Lorensen and Cline (1987) (no pun intended).
2. BACKGROUND
[included in the Initial Document, Interim Document (if updated), and Final Dissertation]
This section describes the research the author has done in order to prepare for the project. The background
section can be broken down into two sub-sections: (1) related work and (2) previous systems.
2.1 Related Work

The related work section demonstrates to the reader that you have done your homework (research), reviewed
the previous literature, and now are ready to present your contribution based what has been previously
published.
One of the difficult aspects of the related work section is choosing the proper scope. There is some
subjectivity in choosing which books or papers to refer to and also importantly, which previous literature not to
refer to. This is something an advisor is able to help with. In short, each previous publication gets a one
paragraph description. We note that there is literature related to these guidelines Laramee (2009, 2010a).
2.2 Citations
Any figure, image, or equation that is taken from another source must be cited. Content and terminology from
other sources must also be cited. For more information about citations and their use, see:
http://www.plagiarism.org/. Click on the “How to cite sources” link.
When referring to previous work use names, not only numbers. So, instead of writing, “In Jobard and Lefer
(1997), an algorithm for the creating of evenly-spaced streamlines is presented.”, write, “Jobard and Lefer
present an algorithm for creating evenly-spaced streamlines Jobard and Lefer (1997).” The names are much
more helpful to the reader than the numbers alone. Also, the previous literature was written by real people.

Writing the names also shows a certain respect and appreciation for the previous work that numbers alone do
not.
References should be accurate and complete, i.e., with page numbers etc. A paper without complete and
correct references can leave a bad impression on the reader and detract from a dissertation or paper’s
credibility. Errors can be interpreted as sloppy and also be a sign of hastiness.
2.3 Previous Systems (or Similar Applications)

If you are building a piece of software, i.e., an application, then there’s a good chance that other closely-
related applications already exist. This section describes those applications. For each previous system the
following information is given:
1. the name of the system or application,
2. the URL of the system,
3. a screen shot of the application, and
4. which platforms the application runs on,
5. the duration time of the trial license, e.g., 30 days
6. a one or two paragraph description of the application including:
• why the application was written and
• for whom (target users) it was written.
Note that each screen shot is accompanied by a caption, including a citation to the source of that screen shot,
e.g., where the software comes from ( including company name, web page author, web page title, URL, and
last access date, etc). Some examples are given in the references section of this document.
2.4 Data Characteristics

If your project process a specific or important data set(s), then a description of that data is an important part of
the project. Some very helpful questions to answer are: Where does the data come from? Who produced the
data? Why was the data collected in the first place? In other words, what are the fundamental questions (or
knowledge) that we are trying to answer with the data?
A technical description of the data is also very helpful. What is the size (or resolution) of the data set? For
example, how many samples does it contain? What is the range (minimum and maximum values) of the data
samples? What are the characteristics of the data? Is it time-dependent? Is it abstract data or scientific data?
Data is called “scientific” if every data sample has a given position in space, i.e., (x,y,z) coordinates. Abstract
data has no position associated with its data samples, e.g., financial data describing stock transactions such
as buying or selling. How big is the file that stores the data? What format is the data file in? Provide a
complete description of the data file format.
3. PROJECT SPECIfiCATION
[included in the Initial Document, Interim Document, and Final Dissertation]
The Project Specification Section is broken down into two sub-sections: the feature specification and
technology choices.
3.1 Feature Specification
Recall the five stages of software development: (1) requirements specification, (2) software design, (3)
implementation, (4) testing, and (5) documentation, for both programmers and users.
This section is all about stage one–documenting the requirement specification. The requirement specification
describes the functionality of your system. What features is the software to have? What will it be capable of
doing? The feature set is divided into two categories: (1) must-have features and (2) optional features. Must-
have features are the minimum set of requirements the system must have in order to be useful whereas
optional features are those that bring the level of the application up to outstanding quality. Each feature in
both the must-have and optional features sub-sections is presented in a numbered (enumerated) list.
3.2 Technology Choices

This section outlines which tools and options are available for your implementation. Three basic decisions
usually have to be made:
• Which programming language to use.
• Which GUI library to use. Some possible choices are
1. The Java Swing Library (free, cross-platform),
2. The Fox Toolkit (free, open-source, cross-platform) van der Zijp,
3. QT (cross-platform, both commercial and free, open-source editions available) Blanchette
and Summerfield (2004); Trolltech,
4. FLTK (free) Spitzak,
5. GLUI (free) Rademacher and Stewart,
6. wxWidgets (free, cross-platform) Smart,
7. wxWinows (free, cross-platform) Neifer,
8. GTK+ (open source, cross-platform) Team etc.
One of these GUI libraries is suitable for your application for sure. There’s a very nice description of how to
use the wxWindows library in chapter 18 of Horstmann Horstmann (2003). (The wxWidgets library is formally
known as wxWindows.)
• Which other (if any) libraries are used, e.g., OpenGL, DirectX, etc.
After describing the possible options, why you have chosen those options is given. The best tools are: (1)
open source, (2) cross-platform, and (3) provide a forum in which to ask questions, e.g., a mailing list. It’s nice
if the description of technology choices includes a screen shot of each GUI option.
4. PROJECT PLAN AND TIMETABLE

[included in the Initial Document, updated in the Interim Document, and Final Dissertation]
This section describes your project plan. The project is broken down into smaller tasks (divide-and-conquer)
and these interim milestones are written down into a table. There may already be a series of deadlines
outlined in your course handbook or project syllabus that can be included in the table:
1. initial project document deadline: Semester 1 Week 3,
2. presentation deadline Semester 1, late November,
3. project review just before the Christmas holiday,
4. interim document deadline: Semester 2 Week 2, and
5. demonstration and viva deadline: Semester 2, Week 10.
These are the absolute minimum milestones. Then, each feature described in the Project Specification
Section (Section 3.1) is included in the table. See Table 1 for an example. The table describing the project
plan and timeline can have three columns. In the first column is a number, in the second column is a
description of the feature to be implemented and in the third column is a date by which the feature is to be
implemented.
Sample Project Timetable

Description Date
1. Initial project presentation 19 Nov
2. File Opener: User opens data file via the File menu 1 Dec 07
3. User is able to view a 2D graph of time (x-axis) vs amplitude (y axis). 8 Dec 07
4. project review just before the holiday, 15 Dec 07
5. etc... 15 Jan 08
Table 1: A sample table with interim deadlines in this project. The timeline is broken down week-by-week. For each week
there is some measure of progress.
Construction of Table 1 is easy. Simply cut and past each feature listed in the Feature Specification Section
(Section 3.1) and place it under the description column in the table. Then add a completion date. Features
that are too big to be completed in one week can be broken down into smaller pieces that can be
accomplished within one to two weeks.
4.1 Time Management

We recommend working on your 3rd year project for approximately ten hours per week during your third year.
The project is worth 30 credits, or 25% of your third year. Also, the marks from your third year carry twice the
weight as the marks from your second year. Thus the project can be thought of counting for 60 credits
towards your final degree.
4.2 An Aside on Proofreading
In the ideal world, the dissertation is proof read by someone outside the project before submission to look for
typographical errors. Fresh new eyes that don’t necessarily know the topic can be good for spotting these
things, at least in our experience.
5. THE INTERIM DOCUMENT

The interim document, due the third Monday of semester two, summarizes the project’s progress up to that
point in time. The goal of the interim document is to provide the reader a summary of progress since the initial
document. Conceptually, the interim document is an extension of your initial document. However, if you
submit something that looks too much like your initial document, you will not earn the 25% (formerly 30%) of
the credit that this portion of your project is worth. Therefore, a table is included in your interim document
stating exactly how it is different from your initial document. Table 2 shows an example.
Your table includes every section and subsection of your dissertation that has changed, been updated, or
introduced (new sections and sub-sections) since the initial document. If a section has not changed,
Section Summary of Interim Document Status

1. Introduction This section has been updated with corrections since initial document.
2. Background This section has been updated and expanded (since the initial document).
3. Related Work This section was updated and expanded to include new literature.
4. Previous Systems More previous systems were added.
5. Project Specification This section has been updated and expanded with more detail.
6. Project Design This is a new section (not featured in initial document).
7. Project Implementation This is another new section.
8. ...
Table 2: A sample summary of the interim document indicating what has been done since the initial document was
submitted.
such as the Introduction, it is not included in the interim document. You may use the minutes of meeting notes
as a history of the project because they document progress that has been made since the initial document
Laramee (2007b).
The table of contents in this document indicates which sections are included in the initial document, interim
document, and final dissertation. This is to be used as a very general guideline and can be customized to fit
most projects. Also read the description of the interim document in your Computer Science Department’s
Course Handbook.
The interim document also requires an updated time table. The time table shows your temporal plan for the
remainder of the project. See Section 4 for a sample time table.
6. PROJECT DESIGN
[included in in the Interim Document, and Final Dissertation]
This section corresponds to the second stage of the software development process Sommerville (2007):
software design. Your design is most likely object-oriented. The design starts with a list of classes and their
responsibilities. This is a very important starting point (the list of classes and descriptions). It forces the
programmer to start thinking about implementation. After identifying a list of candidate classes and writing a
short description of each of them, e.g., 3–4 sentences, the list is reviewed with the project supervisor before
proceeding onwards with the description of relationships between them.
6.1 Design Diagrams

The design also depicts the relationships between the classes through class hierarchies and collaboration
diagrams. See also Laramee et al. Laramee et al. (2005b) for some examples of this.
Some types of diagrams that can be included in your design are described here. Process diagrams that
show different stages of processing along with input and output data in between stages (Figure 1).
System diagrams that identify different sub-systems in your software design and the relationship between
them.(Figure 2). In modern, object-oriented software development, more time is spent on software design
Wirfs-Brock et al. (1990) in order to make software more robust, increase code re-use, facilitate maintenance,
and make it easier to extend. The design of software is based on object-oriented methodology.
Figure 1: This figure shows an example of a process diagram. For example, the CFD process is iterative and can be
pipelined into modeling, simulation, and visualization stages. Rounded boxes represent input/output data while processing
stages are depicted as rectangles. All of the figures in this section were made with a program called Xfig
(http://www.xfig.org). Inkscape (http://www.inkscape.org) is another free and popular program for creating nice diagrams.
Figure 2: This is an example of a software system showing some of its subsystems. This example shows a schematic of
the design for a visualization system. Only the major subsystems are illustrated.
The design of your system can use the notation of Wirfs-Brock et al. Wirfs-Brock et al. (1990). An example is
shown in Figure 2. A semicircle with an arrow pointing to it represents a contract. A contract is a subsystem or
class interface with other classes or subsystems. It represents the set of services that a subsystem or class
provides. Figure 2 illustrates the different subsystems and the relationships they have with one another.
Class hierarchy diagrams are also very useful (Figure 3). Figure 3 shows different streamline rendering
options displayed in the class hierarchy in which they were designed and implemented. The hierarchy,
following UML notation Fowler (2003), illustrates the is-kind-of relationship between classes. At the top of the
hierarchy we have an abstract base, Streamline Renderer that describes the behavior and contains the
interface that all streamline rendering objects implement.
Composition diagrams are also very useful (Figure 4). Figure 4 shows the class relationship between the
major components of a texture-based flow visualization subsystem, again using UML notation. However,
rather than showing is-kind-of relationships as in Figure 3 we show composition, a variety of aggregation (is-
part-of relationship). With composition, the part object may belong to only one whole, further,
Figure 3: This is an example of a class hierarchy diagram. This example is from a class hierarchy of streamline and
geometric flow visualization options. UML notation is used.
The parts are usually expected to live and die with the whole Fowler (2003). For example, the Texture Stack
object is part of the Texture-Based Flow Visualizer object and the relationship is one-to-one. Furthermore, an
instance of Texture Stack may be in an instance of Texture-Based Flow Visualizer but not the other way
around. This is indicated by the black diamond shape arrow.
6.2 Figures and Images

Diagrams may be created with free drawing packages called Inkscape (http://www.inkscape.org) or Xfig
(http://www.xfig.org). Figures, images and their captions are “self-contained”. By self-contained, we mean that
the reader is able to look at any figure or image, read its caption, and gain a basic understanding without
having to read the main text. Enough details are provided in the caption such that the reader is not required to
read the whole paper in order to understand an image or figure. Many readers will make a first pass through a
document simply by flipping through the pages, looking at the images and captions, and already gain an
impression. This first pass may already influence the reader’s opinion. If the reader appreciates the images
and can gain an understanding of them from the captions, they may form a favorable first impression.
When a potential reader picks up any reading material, whether it be a newspaper, magazine, or book, they
first flip through it to look at the pictures. If they find an interesting picture, they read the caption. If they find
interesting pictures and captions, they may then go on to read the main text.
Pedagogic figures are good to include in your dissertation. The author will make his job easier if makes all of
his images, e.g., screen shots or frame buffer images, the same resolution, perhaps through a user setting.
Also, a user setting that changes the background color to white is very useful. Images with a white

background are best for papers in our opinion. We refer the reader to Tufte Tufte (1983) for further stylistic
guidelines.
Figure 4: This is an example of a composition diagram. For example, the major components of the texture-based flow
visualization design. Here aggregation, or is-part-of, relationships are shown.
7. IMPLEMENTATION
[started in the Interim Document and completed in the Final Dissertation]
Section 7 is the main content of the dissertation and thus occupies the most space. It is broken down into two
sub-sections: the must-have features and optional enhancements.
7.1 Basic Implementation (Must-Have Features)

This sections describes the must-have features implemented from the specification (Section 3) and how they
were implemented. The implementation is based on your model from Section 6. The name of Section 3 could
be the name of your system or algorithm (if it has a name, e.g. Image Space Advection (ISA) Laramee et al.
(2004)). Section 7 can also be called simply Implementation. Section 7 contains screen-shots of your
application as well as accompanying descriptions. Some important questions to be addressed in this section
are:
1. What are the different features and options your program offers?
2. How does the application work?
3. What data structures are used in the implementation?
4. Were there any features that were particularly difficult to implement? Why?
The reader then knows how your system (or algorithm) operates after reading this section. The reader is
given enough detail such that they can operate or implement the system themselves.
7.2 Enhancements (or Extentions)

After the must-have features of your system are presented in Section 7 a follow-up Section may be added that
includes enhancements or extentions to the must-have features. This section presents the optional features
that were first specified in Section 3. The must-have features describe the system necessary in order to get a
passing note for your project. The extensions describe the features that are necessary in order to obtain a first
on the project.
7.3 Commenting Your Code

The source code of your project is documented according to the guidelines set forth in a separate document
Laramee (2010b), a copy of which the reader asks for at the start of the implementation phase. For projects
written in Java, javadoc can be used to comment the source code:
http://java.sun.com/j2se/javadoc
For projects written in just about any language, e.g., C++, Java, etc doxygen:
http://www.doxygen.org
may be used to comment the source code. Doxygen or javadoc is then used to produce HTML output of your
code comments. Doxygen provides the added benefit of automatically generated diagrams Laramee (2007a).
This HTML output will be inspected by the project supervisor and whoever else may assess the project.
7.4 Coding Conventions

The source code of your project also follows the code commenting standards outlined in a separate document
Laramee (2010b), a copy of which the reader asks for at the start of the implementation. Following these
coding conventions yields several important benefits including, but not limited to: (1) higher code
legibility, (2) fewer bugs, (3) higher code quality, and (4) ultimately a better project. The source code of your
project will actually be inspected to see if it conforms with the conventions. In the end, following these coding
conventions raises the final score of your project.
8. TESTING AND EVALUATION

[included in the Final Dissertation]
Evaluation is a very important part of your project (or any project). A very important question to ask right from
the beginning is, “How is this project going to be evaluated?” Some important topics discussed in the
evaluation are:
• Testing: How is your software going to be tested? What are the different test cases? What sources of
data will be used to test your system?
• Performance: How well does your application perform? How fast (or slow) is your application? How
memory efficient (or inefficient) is your algorithm or system?
8.1 Results
In this section, the nicest images of your system or algorithm are presented. Your system is applied to
interesting data sets and the insight provided is described in addition to a concise description of the data sets.
The characteristics of the test data sets is given. Some possible things to describe are:
1. Where did the data (what domain) come from?
2. How large the are the data sets? What is their resolution in the case of structured data?
3. What are the spatial and temporal dimensionalities of the data?
4. Is the data structured or unstructured?
5. It the data multi-resolution or adaptive resolution?
Hopefully, you’ll be able to show your algorithm being applied to data sets from another domain, e.g., the
visualization of simulation data, in order to demonstrate that your software actually works.
8.2 Performance Analysis

In most areas of computer science like visualization and computer graphics there’s always a trade-off
between quality and speed (or memory and speed). Generally, the faster the performance, the lower the
quality or the higher the memory requirements. The algorithm or system being presented likely has some free
parameters that can adjusted to trade off quality for speed and vice-versa or to trade off memory for
performance and vice-versa. Essentially, every parameter whose value you are not sure of should be a user-
defined parameter–an option that can actually be changed by the user at run time (not just compile time).
A table of performance times can report both aspects, optimized for speed, i.e., exploration mode, and
optimized for quality, i.e., presentation mode. The best and most honest reports of performance times will
show when the algorithm or system reaches its limits, e.g., when it starts to run slow due to perhaps large
data sets. Algorithms also usually trade off memory for speed. In other words, faster algorithms often require
more memory, basically because more calculations are pre-processed and stored.
If your project is on the subject of visualization, remember, visualization is used for analysis, exploration, and
presentation. For presentation, the image quality must be at its highest, and thus performance probably at its
slowest. The highest quality is used to generate the images for the paper, presentation, and videos. For
exploration, the algorithms and systems are tuned for the fastest performance times possible. All optional
enhancements may be turned off (unless your algorithm always runs at interactive frame rates).
See Laramee and Bergeron Laramee and Bergeron (2002) and Laramee et al. Laramee et al. (2003, 2004)
for good examples of detailed performance analysis that trades off between performance and quality or
memory and speed.
9. CONCLUSIONS
The Conclusions Section just summarizes the presented work and lists the resulting advantages. Remind the
reader why the presented work is so good and what’s new. Basically, it’s like a concise, one page summary of
the abstract and introduction sections.
10. FUTURE WORK

The future work section is about possible extentions to the presented work.
11. TURNING IN SOURCE CODE

[included in Final Dissertation]
We recommend turning in your source code electronically when submitting your interim and final documents.
Your source code can be simply placed online, on your local computer science web server. As an example,
instructions on how to do this at Swansea University are given on the CS 124 (Computers and Society) web
page:
http://cs.swan.ac.uk/˜csbob/teaching/cs124-tutorial/
All undergraduates have accounts created when they start at University and have access to their own web
space. Within their home directories they will have a folder called public html. Any files/web pages they put in
there will be available on the web and can be accesses from:
http://cs-sol.swansea.ac.uk/˜username
where ˜username is the students username e.g. cs123456
The best possible thing a student can do is to place the doxygen HTML output onto their web server space
Laramee (2010b, 2007a). See Sections 7.3 and 7.4 for more on this topic. Some students turn in an electronic
copy of their source code and application on CD at the end of their project. We do not recommend printing out
your source code at any point since it wastes too much paper. Plus there’s no guarantee that anyone will look
at it.
12. ACKNOWLEDGMENTS
This section mentions any people that may have contributed in some way, e.g., with the implementation or
with discussions or feedback. It is tradition to mention your advisor here at minimum. The Acknowledgements
Section also thanks any agency or organization that may have contributed funding to the presented research,
e.g., the EPSRC, KPlus, the NIH, the NSF, etc. An example of a long and complicated acknowledgements
section can be found in previous work Laramee (2004).
Thanks to Daniel O’Sullivan and Nicholas Hammett for providing the impetus behind these guidelines. These
guidelines are the result of approximately four years of third-year project supervision in the Computer Science
Department at Swansea University. We thank the following project students for their valuable discussions and
input: Essam Farrag, Nicholas Hammett, Marc Harry, Kingsley Hau, Daniel O’Sullivan. Matt Reid, Ferdinand
Vesely, James Walker, and Ho Yuen Wan.
Questions, comments, suggestions for improvement, contributions to these guidelines, or any other feedback
is not only welcome but encouraged. If your department or organization uses these guidelines, we’d like to
hear about it. Please contact Robert S. Laramee.
13. APPENDIX–BCS ACCREDITATION

The following text is taken from from the Computer Science Departmental Handbook at Swansea University.
Project Requirements
The following requirements for projects to be acceptable for BCS Accreditation is extracted from information
supplied by the BCS. If you have any doubts as to whether your project meets the requirements please
discuss it with your supervisor. For projects to meet the requirements for exemption, they must be passed at
the first attempt to gain the award, with no condonement and no referral. All projects must involve a
professional approach. The project should be real in the sense that the product is for users other than the
author. The report should demonstrate an appropriate level of professional competence in the practical
development of a suitable application, tool or similar product. The report on the project should include:
• elucidation of the problem and the objectives of the project;
• an in-depth investigation of the context / literature / other similar products;
• a clear description of the stages of the life cycle undertaken;
• a description of the use of appropriate tools to support the development process;
• a description of how verification and validation were applied at all stages;
• a critical appraisal of the project, indicating the rationale for design / implementation decisions,
lessons learnt during the course of the project, and evaluation (with hindsight) of the product and the
process of its production (including a review of the plan and any deviation from it);
• in the case of group projects, a clear indication of the part played by the author in achieving the goals
of the project;
• references;
• appendices technical documentation.
The specific criteria applicable to projects acceptable for full exemption from the BCS Professional
Examinations is that:
• it involves at least 150 hours of individual student effort;
• the task should be to develop an IT solution to a practical problem, which would include the
production of a piece of software (which may be interpreted as some or all of a specification, design
or implementation of software);
• it exhibits a structured approach to information systems practice, involving a number of stages in the
life cycle;
• the product exhibits the attributes of quality, reliability, timeliness and maintainability;
• it must involve the production of a professional report as described above;
• it must lead to a description of the process and of the product;
• it must contribute significantly to the overall award classification.
14. REFERENCES
J. Blanchette and M. Summerfield. C++ GUI Programming with Qt 3. Prentice Hall in associate with Trolltech
Press., Upper Saddle River, New Jersey 07458, 2004. Bruce Perens’ Open Source Series (free for
download).
M. Fowler. UML Distilled: A Brief Guide to the Standard Object Modeling Language. Object Technology
Series. Addison-Wesley, third edition, September 2003.
C. Horstmann. Computing Concepts with C++ Essentials. Wiley, 3 edition, 2003.
B. Jobard and W. Lefer. Creating Evenly–Spaced Streamlines of Arbitrary Density. In Proceedings of the
Eurographics Workshop on Visualization in Scientific Computing ’97, volume 7, pages 45–55, 1997.
Helmut Kopka and Patrick W. Daly. A Guide To LaTeX. Addison–Wesley, 4 edition, 2004.

R. S. Laramee. Bob’s Concise Introduction to Doxygen. Technical report, The Visual and Interactive
Computing Group, Computer Science Department, Swansea University, Wales, UK, 2007a. (available online).
R. S. Laramee. How to Write a Visualization Research Paper: A Starting Point. Computer Graphics Forum
(CGF),29(8):2363–2371, 2010a. (available online).
R. S. Laramee. How to Write a Visualization Research Paper: The Art and Mechanics. In Eurographics
Education Papers 2009, pages 59–66. Eurographics, 2009. (available online)
R. S. Laramee. Interactive 3D Flow Visualization Using Textures and Geometric Primitives. PhD thesis,
Vienna University of Technology, Institute for Computer Graphics and Algorithms, Vienna, Austria, December
2004. (available online).
R. S. Laramee and R. D. Bergeron. An Isosurface Continuity Algorithm for Super Adaptive Resolution Data. In
Advances in Modelling, Animation, and Rendering: Computer Graphics International (CGI 2002), pages 215–
237. Computer Graphics Society, Springer, July 1-5 2002.
R. S. Laramee, C. Garth, H. Doleisch, J. Schneider, H. Hauser, and H. Hagen. Visual Analysis and
Exploration
of Fluid Flow in a Cooling Jacket. In Proceedings IEEE Visualization 2005, pages 623–630, 2005a.
R. S. Laramee, M. Hadwiger, and H. Hauser. Design and Implementation of Geometric and Texture-Based
Flow Visualization Techniques. In Proceedings of the 21st Spring Conference on Computer Graphics, pages
67–74, May 2005b.
R.S. Laramee. Bob’s Concise Coding Conventions (C 3 ). Advances in Computer Science and Engineering
(ACSE), 4(1):23–26, 2010b. (available online).
R.S. Laramee. Bob’s Minutes of Meeting Protocol. Visual and Interactive Computing Group, Department of
Computer Science, Swansea University, 2007b. (available online).
R.S. Laramee, B. Jobard, and H. Hauser. Image Space Based Visualization of Unsteady Flow on Surfaces. In
Proceedings IEEE Visualization ’03, pages 131–138. IEEE Computer Society, 2003.
R.S. Laramee, J.J. van Wijk, B. Jobard, and H. Hauser. ISA and IBFVS: Image Space Based Visualization of
Flow on Surfaces. IEEE Transactions on Visualization and Computer Graphics, 10(6):637–648, November
2004.
W. E. Lorensen and H. E. Cline. Marching Cubes: a High Resolution 3D Surface Construction Algorithm. In
Computer Graphics (Proceedings of ACM SIGGRAPH 87, Anaheim, CA), pages 163–170. ACM, July 27–31
1987.
M. Neifer. Looking Through wxWindows. http://www.ibm.com/developerworks/library/l-wxwin.html [March
2009].
P. Rademacher and N. Stewart. GLUI User Interface Library. http://www.cs.unc.edu/˜rademach/glui/
[December 2007].
J. Smart. wxWidgets. http://www.wxwidgets.org/ [February 2008].
I. Sommerville. Software Engineering. Addison-Wesley, London, 8 edition, 2007.
B. Spitzak. The Fast, Light Toolkit (FLTK). Easy Software Products. http://www.fltk.org/ [December 2007].
The GTK+ Team. The GTK+ Project. The GNOME Foundation. http://www.gtk.org/ [January 2009].
Trolltech. QT. Trolltech ASA. http://trolltech.com/products/qt [December 2007].
E. R. Tufte. The Visual Display of Quantitative Information. Graphics Press, 1983.
Jeroen van der Zijp. The Fox Toolkit. http://www.fox-toolkit.org [December 2007].
R. Wirfs-Brock, B. Wilkerson, and L. Wiener. Designing Object-Oriented Software. Prentice-Hall, 1990.

BUILDING A SOFTWARE DEVELOPMENT ENVIRONMENT TO
ENRICH LEARNING AND ENHANCE EMPLOYABILITY
Dr John Kerins
Department of Computer Science
and Information Systems
University of Chester
Chester CH1 4BJ
j.kerins@chester.ac.uk
ABSTRACT
There is an increasing trend towards employer engagement in Higher Education. It is recognised that the
workplace represents a legitimate learning environment in the modern knowledge economy. Higher Education
Innovation Funding is used to foster mutually beneficial links between universities and industry and
commerce. This paper provides a reflective analysis of the utilisation of this funding to establish a software
development team within a department of computer science and information systems to form links with the
regional economy, and to effect synergy between students, staff and external clients. This forms the basis of a
stimulating environment which provides real-world scenarios and brings current skills and knowledge into the
Department. It provides contemporary case studies, offers valuable support to the regional economy, and
enables students to acquire skills to equip them for employment. There is now scope for further curriculum
enhancement and new multidisciplinary initiatives to support applied research.
Keywords
employer-responsive provision, knowledge economy, knowledge-transfer, software development, work-based
learning.
1. INTRODUCTION
There is current concern about motivating and retaining students in Higher Education (HE) and in designing
programmes that will provide them with the skills to function effectively in demanding working environments.
Employer links are increasingly seen as a vital part of university provision, as knowledge generation is
acknowledged to play a significant role, particularly in advanced technical environments. Successfully
integrating provision is not without its challenges, however. This paper presents an account of a recent
knowledge exchange initiative within computer science and information systems, which was co-ordinated by
the University’s Research and Knowledge Transfer Office. This utilised HEIF 4 funding, which is designed to
support and develop a broad range of knowledge exchange activities which result in economic and social
benefit to the UK (HEFCE, 2008). The aim was to establish links with regional businesses and thereby foster
industrial contacts, enhance the curriculum, broaden the Department’s perspective and improve student
employability. The initiative has established firm foundations which promise opportunities to strengthen
regional links and to engage a wider range of students in stimulating, real-world activities to complement
theoretical study.
There is significant debate about the role of work-based learning in HE: Gray (2001), evaluates its
advantages, while Nixon, Smith et al (2006) point up tensions that can arise between HE providers and
employers. There are also questions about the quality of placements as pressure increases to allow students
access to employers (Little and Harvey, 2006). Mason, Williams et al (2003) also challenge the assumption
that employability skills can be developed in HE and transferred into employment. However, it is clear that
there is a marked shift towards recognition of the potential role of the workplace in many HEIs’ provision. The
Quality Assurance Agency for Higher Education’s (QAA) report acknowledges a number of HEIs’ revised
mission statements to accommodate collaboration with business and employers (QAA, 2010). There are
perceived imperatives driving this movement: the HEFCE report (CHERI and KPMG, 2006) identifies the trend
towards high performance management and high performance work practices in organisations featuring
complex job designs, devolved lines of responsibility, and reliance upon access to key business information.
This is consistent with the view that developing countries must rely increasingly upon intellectual capital to
maintain their competitiveness (RAE, 2007). It is also recognised that knowledge creation is not the preserve

of HEIs, and that significant learning can take place within working environments (CHERI and KPMG, 2006).
Successfully integrating the educational needs of individual learners with the requirements of employers is
complex and challenging. Potential conflict can arise from differences in values, traditions and practices
between academia and industry. Research focusing on new conceptual and methodological insights into
workplace learning considering a range of professions including accountancy, nursing, finance and education
is addressing the differences, if any, between work-based learning and professional and vocational education
(Pachler and Evans, 2008).
While vocational training in areas such as education and medicine has well established tradition and
practices, the expansion of employer-responsive provision involves a wider range of organisations in the
design, delivery and assessment of learning programmes (QAA, 2010). This raises questions about how to
establish and maintain the quality of provision. While the QAA reinforces the notion that HEIs are responsible
for upholding the quality of their awards, there is an argument that HEIs should strike a balance that militates
against becoming too risk averse, and encourages collaborative provision and effective employer
engagement.
Within this general context of increasing synergy between HEIs and industry, The Royal Academy of
Engineering adds an engineering perspective which stresses the importance of real industrial experience in
graduate recruitment from employer and student perspectives (RAE, 2007). The report highlights the value of
academic staff’s familiarity with industry problems and of challenging case-study material from industry. These
requirements are presented amid fears that an increasing shortage of engineers may have a significant
impact upon the economy’s competitiveness in the future. Indeed, HEFCE acknowledge HE’s role in
encompassing the wider view of the future of the knowledge society and its need for social cohesion as well
as economic prosperity in successfully balancing the requirements of industry and the needs of learners
(CHERI and KPMG, 2006).
Interestingly, the demand for people in the workplace to acquire new skills through partnerships with
academic institutions is raising questions about provision in Germany. A recent newspaper article (From Kick-
about Football Hero to Graduate) addresses a perceived need for mature employees with limited formal
academic qualifications to be allowed access to higher education (Boes, 2010).
It is within this broader context of seeking to engage learners in a purposeful, practical environment focusing
on work-related skills that this paper reflects the challenges of shaping skilled and confident graduates, from a
relatively small department, capable of adding value to regional organisations. It describes how recent activity
within the Informatics Centre, based in the Department of Computer Science and Information Systems,
demonstrates the potential to forge new methods of linking the curriculum to the external world to bring
benefits to staff, learners, clients and partners.
The Informatics Centre was established some years ago to provide IT support with data monitoring projects in
public health. It was recognised that this provided scope to engage students in real-world software
development projects and thereby enhance their insight into the demands of working in the software industry
(Kerins, 2007). This paper reports on the establishment of a software development team in the Informatics
Centre in response to an initiative to exploit knowledge exchange funding, and evaluates its benefits to
teachers, learners and external clients. Most of the latter represent Small and Medium Enterprises (SME) who
stand to gain a great deal by acquiring new technology to enhance their business processes.
This paper illustrates examples of these benefits and describes how continued involvement in the initiative
accumulates knowledge, which enables the Informatics Centre team to offer an improved and more informed
service to its clients. It also analyses the potential to enhance the computer science and information systems
curriculum by providing not only part-time employment opportunities but also valuable case-study learning
materials: the Informatics Centre has generated a developing repository of project deliverables which can be
exploited by group and individual learners. The challenge is to build interactive interfaces to this material so
that it is accessible across relevant programmes. It can also be made available to partner colleges where a
significant number of students study foundation degrees before attending the university for a final year. A
critical feature of the development is its link with the demands of regional businesses and the opportunity to
link software design and development tasks to contemporary business requirements. This provides students
with skills and insights which will enhance their employability and their preparedness to meet real-world
information processing demands within the regional economy.
2. UTILISING KNOWLEDGE EXCHANGE RESOURCES

Innovation funding has been made available to stimulate the exploitation of knowledge from institutes of
further and higher education by regional businesses (HEFCE, 2008). The underlying premise is that staff and
students within these institutions possess considerable know-how and expertise which can be used to provide
valuable knowledge as well as skilled and creative personnel to regional enterprises. There are particular
benefits for SMEs which lack the resources to invest in, or access, scarce skills such as software design and
development.
The innovation funding initiative complements a range of existing services whose aim is to foster relationships
between the University and regional employers. The University has a well-established Department of Work
Based Learning which has pioneered programmes designed to establish these links and to give students
valuable experience of a practical work placement to enhance their employment prospects upon graduation.
This environment has supported research into work-based learning methods, particularly approaches to
assessment (Brodie and Irving, 2007). Organisations such as Business Link (BusinessLink, 2010) also aim to
match individuals to employers for fixed-term paid placements which are of mutual benefit to both partners.
Within the Informatics Centre, the benefits of these initiatives were acknowledged. It was also recognised that
the experience acquired by individual students remained with them, while their relationship with the employer
remained external to the Department. Consequently, knowledge of real-world demand and of the tools,
technologies and skills needed to meet these requirements was not brought back into the University where it
could be used to consolidate existing knowledge. Moreover, by acting individually, students could learn a
great deal from professional staff within the placement organisation, but they were potentially exposed to
undertaking tasks with very limited professional or technical support.
The proposal was to establish a small team of software developers who would work collectively on client
solutions. The team would consist of students recruited from programmes within the Department who would
be supervised by academic staff. The aim was to pay the students just as they are paid for projects
undertaken through schemes such as Business Link. There were clear benefits in that the team would offer
supervision, and students could work collectively on tasks to share learning and responsibility. As students
often do some part-time work during their programmes of study, the prospect of working on software
development projects provided an excellent opportunity to engage in tasks that complemented their academic
programmes.
2.1 Establishing a Software Development Team

A specialist department within the University handles knowledge transfer funding, and proposals for projects
are evaluated by committees formulated within specialist areas based largely on faculty interests. In August
2008 the Informatics Centre was granted funding to establish a team of four student software developers
whose activities would be co-ordinated by two members of academic staff.
The team, including one experienced post-graduate student, contained a blend of skills, expertise, knowledge
and experience, and it worked initially on establishing an infrastructure to support development. This included
the adoption of SharePoint as a repository for documentation, communication and resources (Leon, Tynes et
al, 2007). The first projects were requests for web-based systems generated from within the University. The
team also responded to a speculative enquiry to the Department for support in utilising a business intelligence
application from a regional multinational company, and this led to the completion of a number of related
projects with the organisation.
2.2 Extending the Portfolio

A significant impact on the work of the team was the demand for knowledge transfer initiatives brokered by
the Regional Development Agency whose brief was to provide financial support for small businesses in key
sectors of the economy. The Agency released periodic funds for which regional SMEs could bid. The scheme
provided innovation vouchers to successful bidders, which could be spent with a suitable knowledge provider.
Over a period of some eighteen months, the team successfully bid to undertake a significant number of
innovation voucher projects. These were predominantly web-based applications in which SMEs from areas as
diverse as telecommunications, on-line sales and business consultancy services required new or substantially
overhauled systems to enhance their business processes.
3. EXPANDING THE TEAM

The University implements a Work-Based Learning (WBL) placement scheme for all second-year
undergraduate students. Six-week placements are co-ordinated by a centrally managed unit at the end of the
academic year. The Informatics Centre was able to offer three software development positions. The academic
component of this scheme is handled by WBL supervisors, which enabled staff within the Informatics Centre
to focus on co-ordinating development tasks (Kerins, 2007). This also enables the critical issue of assessment
to be handled by independent supervisors (Brodie and Irving, 2007). At the end of the placement there was
sufficient work to offer part-time contracts to each of the WBL students. Further expansion towards the end of
last academic year led to the appointment of a specialist designer from the Department of Multimedia.

It became evident that a number of clients required web applications to extend the effectiveness and
efficiency of their businesses. As most team members were learning web development techniques as part of
their academic programme, they were able to utilise and significantly enhance their expertise in the field.
4. TECHNICAL ADVANCES
In being exposed to external clients, the team was required to meet a range of user specifications. As
businesses recognised the ways that new technologies could enhance their processes, efficiency, and means
of communicating with the outside world, they sought to use knowledge providers to help them realise these
goals. Equally, as the team acquired knowledge and experience from their developing portfolio, they were
able to advise clients of the kinds of solution that might best meet their business requirements.
Web application development was implemented principally in .NET (Microsoft, 2010) and the team
successfully developed key software components such as a content management system enabling users to
exercise significant control over the material in their web sites. Users were offered facilities such as the
functionality to manage news updates and associated images on their sites. Developers undertook relevant
research to expand their own capability in order to integrate functionality such as shopping cart and on-line
payment facilities into user applications. They also acquired the expertise to integrate features including
search engine optimisation and interfaces to social networking sites into client systems. Noticeably, in a fast-
changing environment where these technologies are becoming more commonplace, there are increasing
expectations as to what clients might regard as basic requirements.
The team has also embarked on projects that require systems to be tailored to more specific business tasks.
Examples include a bespoke mail ordering system, a software version of a business board game and
promotion of products using animation software.
5. MANAGEMENT AND INFRASTRUCTURE

In addition to building a team and delivering client solutions, a significant task has been the establishment of
an infrastructure to support this development. This becomes more important as the volume and scope of
activities increases. In the summer of 2009, innovation funding was made available to support the promotion
of one of the members of the team to a full-time post as project manager. This was in recognition of the
demand for day to day management of the work of the team in the light of an increased project load. The
funding was linked to anticipated growth in output, with the assumption that income would lead to self-
sufficiency.
5.1 Quality Assurance

The appointment of a project manager enabled essential management functions such as the handling of
project costs to be put in place. Significant effort was also devoted to the design and implementation of a
quality assurance system, with appropriate documents and practices, to establish standard approaches to
project development. This was adopted as an internal project which entailed relevant research into ISO
standards (BSI, 2008) to inform the design of project templates. Standard templates for system specifications,
risk analyses, testing documentation, sign-off and post-project technical reports have been established and
are now being refined to maintain an effective and efficient quality assurance system.
An electronic infrastructure has been built in parallel with the growth of the team’s activity. SharePoint has
been configured to support access to and sharing of documentation and communication across the Centre,
and within individual projects. The quality assurance team took responsibility for structuring the SharePoint
site: the overall vision of how projects should be managed and how their deliverables should be classified,
made accessible and archived was planned coherently. Work is currently underway on developing models
representing the structure and function of the Centre so that information about processes and procedures can
be made more readily accessible to current and future members of the team. This documentation also
represents a potentially valuable learning resource.
5.2 Knowledge Management

Since the recruitment of the software development team, the Informatics Centre has evolved into a dynamic
environment supporting an increasing range of roles and responsibilities. As identified in Section 4, a
significant amount of technical work has been undertaken and the pressures of completing these diverse
projects has meant that very often team members have moved fairly quickly on to new tasks. The notion of
managing knowledge raises complex questions about the nature of tacit and explicit knowledge and
organisational learning (Hislop, 2009; Prusak and Matson, 2006). Some of these issues are of particular
interest to the Informatics Centre because the team is composed largely of students who are likely to leave at
the end of their academic programme. However, this risk has been mitigated by the recent full-time
appointments of two team members and the progression of some members of the team to part-time
postgraduate study.
Issues surrounding the development of technical and managerial expertise have been identified as an area for
investigation, with a focus on how best to identify and protect knowledge resources within the Centre. This is
an active area of applied research which will be reported on when further progress has been made.
6. CURRENT ACTIVITIES
As the team’s portfolio of completed work has expanded, opportunities to respond to requests for support
within the University, and from external enquiries, have increased. Consequently, the Centre was able to
recruit two more students at the beginning of the last academic year. The team has completed or is working
on over 40 small to medium sized projects to date.
In addition to software development tasks, the Centre has provided training in web application development to
a group of computer programmers from a regional branch of a national organisation. Selected members of the
team composed learning materials from examples of their work. Their experience using a range of
technologies and techniques, such as AJAX to support dynamic web interaction (Garrett, 2005; Shklar and
Rosen, 2009; Welling and Thomson, 2009), enabled them to enhance the provision with informed, timely
insight.
7. BENEFITS OF THE INITIATIVE

A great deal has been achieved over a relatively short period and it is worth reflecting on the benefits of the
team’s activities as well as looking forward to how the success can be consolidated and built upon in the
future. Two principal benefits are potential enrichment of the curriculum, and enhancing employment
opportunities. The environment also supports the potential for applied research.
7.1 Enriching the Curriculum

Currently there is significant demand for knowledge transfer support in the expanding field of web application
development as SMEs in particular recognise the improvements in business processes that current
technology offers. This has underpinned the engagement in projects supported by the Regional Development
Agency and has exposed the team to a rich variety of business applications that require increasingly
sophisticated software solutions to facilitate business development.
The Centre has already built up a considerable repository of material in terms of system specifications,
system models, test reports, post-project technical reports and completed software solutions. Some of this
material has been integrated into a Level 6 Systems Analysis and Design module where design models have
been introduced as exemplars and a completed project has been used as a scenario to simulate requirements
gathering and system modelling. There are also plans to base an experiential learning module for a group of
some ten students in the Centre. It is too early to evaluate these initiatives but it is anticipated that these
activities will be expanded to enable a wider interaction between the team and computing students.
In addition to the explicit documentation associated with completed projects, the expertise and knowledge
built up by the team constitutes a reservoir of ability and understanding that offers valuable insight into the
challenges of applying the principles of software development to client requirements. The relevance of this
explicit and tacit knowledge is sharpened by the fact that it has been acquired by students’ peers. This helps
to set standards and aspirations and reinforces the sense that academic programmes can be linked to
rewarding roles in employment. The importance of this last point is stressed by the Royal Academy of
Engineering (RAE, 2007).
7.1.1 Future Directions

While these resources have been built up, as yet there is no suitable infrastructure to link them to the
curriculum. An immediate task is to plan and implement processes to facilitate this interaction. The aim is to
provide access to both explicit and tacit knowledge. SharePoint provides the architecture to support the
former. By enabling access to relevant parts of the site, project documents can be made available for
analysis. There is scope to expand this facility however; it is anticipated that these materials could form the
basis of an interactive on-line portal supported by automated learning technology. In this respect the explicit
knowledge within the repository could be utilised independently by learners who could be set individual or
collective tasks based upon case studies and the associated interactive learning environment. Equally, this
facility could be accessible to colleges offering foundation degrees in partnership with the University.
A bi-monthly seminar series has been instituted at which team members present technical reports on some
aspect of their work. Students from all programmes within the Department are invited to these events, which

are also videoed. This affords some access to the members of the team and there are plans to make the
videos accessible within the context of the proposed learning repository. In developing the pedagogical
benefits of the Centre there is a need to foster more personal links between members of the team and the
wider student body. By assigning time and resources to this task, there is potential to tap the experience of
students actively working in the team to inspire learners in computing-related subjects as well as those in
areas such as business and management.
7.2 Enhancing Employment Opportunities

The expansion of the volume of projects undertaken has enabled the Informatics Centre to host a number of
work-based learning placements. These comprised software developers from Computer Science and
Information Systems, a graphics designer from the Multimedia Department, and two Marketing students from
the Business School who were given the task of working on advertising the work of the Centre. Interestingly, a
Law student was also placed to undertake legal investigation of legislation surrounding Intellectual Property
Rights and the protection of data. These examples illustrate the emerging possibilities of interacting with other
disciplines and enabling students to work with each other in a stimulating environment focused on real
demands from the outside world.
Clearly, those students who have been recruited to work on a part-time basis for the team have been afforded
an excellent opportunity to acquire valuable skills that will equip them to make a worthwhile contribution to
potential employers. They are able to derive huge benefit from face-to-face contact with clients as well as
being exposed to the demands of working through the software development lifecycle and meeting agreed
deadlines. They have also been introduced to the rigours of software standards, and are familiar with the
demands of working within a quality assurance framework. However, there is still significant scope to improve
these processes, and the aim is to work towards externally auditable standards. The posts have also required
them to learn a great deal about software tools as they have sought to meet individual sets of requirements.
This experience complements their academic programmes and gives them insight into how these principles
are applied to challenging business processes.
Since the project began, four team members have progressed to external software development posts.
Additionally, three members of the team have been appointed full-time and a further three members have
retained part-time contracts with the Centre, which they integrate with external development work. Five
current students are also working with the team.
7.3 Supporting Applied Research

The work of the development team has already entailed active research in so far as individuals have had to
find out how to do things in order to meet immediate requirements. Particularly in the evolving field of web
application engineering, clients’ aspirations to interface with social networking sites and to integrate
multimedia capability into their systems, for example, have confronted developers with new challenges. They
have become adept at searching relevant discussion boards and consulting software engineering sites
(stackoverflow, 2010; Microsoft, 2010; DevGuru, 2010) to seek techniques that can help them solve novel
problems. Team members have also researched relevant standards to inform their planning and
recommendations for the implementation of quality assurance policies and practices (BSI, 2008).
These technical and management areas pose questions that developers must address in a problem-solving
manner as an inherent part of their role. They also raise issues that bear further analysis. Members of the
software team are principally concerned with developing solutions and they have very limited resources to
investigate issues in a wider context. However, their activities serve as a very useful environment in which
tools, techniques and policies can be analysed independently by dissertation students whose work can inform
the direction of the Centre and its activities. Knowledge resources are being analysed with a view to
optimising how they are utilised. This is of particular importance given the potentially limited time that some
developers may remain with the team. As knowledge is accumulated within the Centre, the team becomes
better equipped to apply it when building new solutions. This learning by experience is an important part of the
educational process for everyone involved in the enterprise.
Critically, the formation of the software development team has established an environment in which students
can learn by doing. Members recruited to the team have significant responsibility to deliver quality solutions.
This provides a valuable grounding in the acquisition of professional standards of practice that will enhance
their attractiveness to employers. It will also impact upon their ability to make a valuable social and economic
contribution upon graduation. There is potential to generate more opportunities for students to benefit from
these interactions. This will either be through active involvement in developing solutions, by taking up work-
based learning placements in the Centre, or by studying recent case studies in relevant modules and
accessing the evolving body of materials available through the range of projects undertaken.

8. RISKS AND LIMITATIONS
Notwithstanding the benefits of the work of the Informatics Centre in helping to enrich the computing and
information system curriculum, there are clear risks inherent in the undertaking and there are limitations in
what has been achieved.
In establishing a framework to support the development environment, Informatics Centre staff drew upon the
assistance of internal services such as human resources, finance and the knowledge transfer and research
office in order to utilise their facilities. This helped in recruitment procedures and in drawing up contracts with
clients, for example. Nevertheless, much of the activity was pioneering within the institution and probably
reflects the shift within Higher Education towards a greater level of engagement with the external community.
One of the principal risks concerns the ability to successfully meet clients’ requirements. There were clear
risks from the Institution’s point of view: failure to meet acceptable standards would damage the University’s
reputation. This was mitigated initially by the fact that a small, mixed team with a range of complementary
skills was assembled. Suitable costing models also had to be devised to ensure that work was completed to
acceptable budgets.
As the team has gained experience, it is better able to scope projects and has a better understanding of the
capabilities and limitations of both individuals and the team as a whole. Equally, there has been investment in
strengthening internal procedures to make the Centre more robust: by devising risk policies for projects to
increase awareness of potential issues such as the impact of new development on client systems, for
example. There is a need to be mindful of the students’ workloads, and to avoid placing them under undue
stress, particularly at assessment periods. There is an inherent risk in being reliant upon highly skilled
individuals whose expertise cannot easily be replaced and this must be borne in mind in managing project
resources.
There are some obvious limitations to what has been achieved. A relatively small number of students have
been given the opportunity to work directly for the team. While work-based learning placements can be
offered to a few additional students, ways must be devised in which the resource can be used to engage a
wider range of students. Apart from making project deliverables available as a repository of learning materials,
there is scope to use project specifications for experiential learning scenarios, as discussed in Section 7.1.
9. STUDENT FEEDBACK AND POTENTIAL DEVELOPMENT

As a part-time employee, each student was subject to a series of three probationary reviews in accordance
with prevailing HR practise. The appraisals were designed to assess progress against a range of generic
attributes such as Communication, and Team Work and Motivation, for example. The students also included
comments on their development during their time in post. Of nine students having completed the process, five
acknowledged the employment benefits of their role in their appraisal, and all of them expressed a wish to
enhance their skills, knowledge and experience through their project work.
These sessions also revealed a desire for more opportunities to work with and learn from other team
members. These observations highlight the scope for continued improvement in the management of the
environment and its members. As the Centre evolves, there will be potential to analyse its teaching, learning
and employment benefits more formally and to seek ways of building upon the foundations reported in this
paper.
A significant challenge is to harness the many benefits of engaging with external clients to attract and retain
students so that they can acquire relevant skills to equip them to provide high quality software engineering
skills to the Region, and beyond, upon graduation. Furthermore, the software development environment can
be made accessible to aspiring university students in schools and colleges to show them how they can
acquire specialist vocational skills that can lead to stimulating careers. At the other end of the HE spectrum,
students can be encouraged to return for postgraduate study where they can develop their expertise, and
enhance the services and solutions that the Centre can offer to its clients.
Currently there are many questions about the funding of Higher Education and the role of employers in
shaping the curriculum. This paper reports on an initiative to enhance and exploit software development
expertise within an academic department of computer science and information systems to forge links with
regional businesses, and to give students key roles in establishing and maintaining the environment. The
paper has shown that there are many positive benefits in promoting active learning and it has discussed the
potential to expand the environment and to make it the dynamic focal point of an enriched curriculum.
Significant progress has been made in a relatively short time and there is confidence that the model can be
refined and further integrated into computing-related programmes. This will improve the commercial relevance
of current programmes and provide students with skills, experience and insight which will increase their

readiness to find worthwhile employment. The opportunities afforded participants so far, as discussed in
Section 7.2, suggest that this is proving successful.
These developments have taken place amid the shift towards employer-responsive provision. The initiative
offers an interesting perspective on this trend and it provides a platform for further collaboration and
experimentation. Unlike specialist centres such as the Centre for Excellence in Professional Learning from the
Workplace (CEPLW, 2010) and the Centre for Excellence in Professional Placement Learning (Ceppl, 2010),
the scope of activity reported here does not extend to formal research in work-based learning. Nevertheless,
there are common aims in enhancing learning and seeking to integrate real-world professional demands into
the curriculum. As partnerships with external organisations continue to be developed, they may also begin to
play a more influential role in shaping the curriculum.
The integration of QA principles is important in developing professional standards and eventual ISO
recognition would help affirm the value of the Centre’s work in providing sound learning experiences for its
computing graduates.
10. REFERENCES
Boes, N. (2010). Vom Bolzplatzhelden zum Bachelor. Frankfurter Allgemeine Zeitung. Frankfurt.
Brodie, P. and K. Irving (2007). Assessment in work-based learning: investigating a pedagogical approach to
enhance student learning. Journal of Assessment and Evaluation 32(1).
BSI (2008). Quality management systems - Requirements. CEN, Brussels.
BusinessLink (2010). Practical advice for business. Retrieved 19th May, 2010, from
http://www.businesslink.gov.uk/bdotg/action/home.
CEPLW (2010). Centre for Excellence in Professional Learning from the Workplace. Retrieved 20th
November, 2010, from www.wmin.ac.uk/ceplw.
Ceppl (2010). Centre for Excellence in Professional Placement Learning. Retrieved 25th November, 2010,
from www.placementlearning.org.
CHERI and KPMG (2006). Towards a strategy for workplace learning, HEFCE.
DevGuru (2010). DevGuru Home. Retrieved 19th May, 2010, from http://www.devguru.com/.
Garrett, J. J. (2005). Ajax: A New Approach to Web Applications. Retrieved 12 January, 2011, from
http://www.adaptivepath.com/ideas/essays/archives/000385.php.
Gray, D. (2001). A Briefing on Work-based Learning, LTSN Generic Centre.
HEFCE (2008). Higher Education Innovation Fund. Retrieved 14th January, 2011, from
http://www.hefce.ac.uk/econsoc/buscom/heif/.
Hislop, D. (2009). Knowledge Management in Organizations A Critical Introduction 2nd Edition. Oxford
University Press, Oxford.
Kerins, J. (2007). Collaborative Initiatives and Work-based Learning Opportunities: A Case Study. ITALICS
6(1).
Leon, W., W. Tynes, et al. (2007). Microsoft SharePoint Server 2007. John Wiley & Sons Ltd, Indianapolis IN.
Little, B. and L. Harvey (2006). Learning Through Work Placements and Beyond. Retrieved 12th October,
2006, from http://www.hecsu.ac.uk/cms.
Mason, G., G. Williams, et al. (2003). How Much Does Higher Education Enhance the Employability of
Graduates?, HEFCE.
Microsoft (2010). Microsoft Developer Network. Retrieved 17th May 2010, from http://msdn.microsoft.com/en-
gb/default.aspx.
Microsoft (2010). The .NET Framework. Retrieved 19th May, 2010, from http://www.microsoft.com/net/.
Nixon, I., K. Smith, et al. (2006). Work-based learning: illuminating the higher education landscape, The
Higher Education Academy.
Pachler, N. and K. Evans (2008). Turning Workplace Learning on its Head. Education Today 57(4).
Prusak, L. and E. Matson, Eds. (2006). Knowledge Management and Organizational Learning A Reader.
Oxford University Press, Oxford.
QAA (2010). Employer-responsive provision survey: A reflective report, The Quality Assurance Agency for
Higher Education.

RAE (2007). Educating Engineers for the 21st Century. Retrieved 17th May 2010, from
http://www.raeng.org.uk/news/publications/list/reports/Educating_Engineers_21st_Century.pdf.
Shklar, L. and R. Rosen (2009). Web Application Architecture 2nd Edition. John Wiley & Sons Ltd,
Chichester.
stackoverflow (2010). Stack Overflow Home. Retrieved 18th May, 2010, from http://stackoverflow.com/.
Welling, L. and L. Thomson (2009). PHP and MySQL Web Development 4th Edition. Addison-Wesley, Upper
Saddle River, NJ.

A REAL-WORLD MOBILE INTERACTION DESIGN TASK
Praminda Caleb Solly Paul Matthews

Department of Computer Science and Creative Department of Computer Science and Creative
Technologies Technologies
University of the West of England University of the West of England
Frenchay, Bristol BS16 1QY Frenchay, Bristol BS16 1QY
praminda.caleb-solly@uwe.ac.uk paul2.matthews@uwe.ac.uk
http://www.cems.uwe.ac.uk/~pcalebso http://www.cems.uwe.ac.uk/~pmatthew
ABSTRACT
This paper presents an overview of a real-world interaction design assignment where the focus was on
encouraging students to explore the conceptual design space to find creative solutions. By providing them
with background information extracted from a real research study with access to a domain expert, we were
able to assist with enabling a deeper analysis of the context, which is often difficult to do within the resource
limitations of an academic course.
Also by engaging the students in a real case-study we were able to explore the potential for further developing
the work as a cross-disciplinary collaborative research project. Integrating research and teaching in this way
creates an exciting environment where both students and teachers become co-learners.
Keywords
Interaction design, mobile, mental health, anxiety, practice, hci.
1. INTRODUCTION
Mobile interaction design involves the development of interfaces, functions and user feedback for applications
running on mobile devices. To maximise the quality of interaction and ensure that an application meets user
needs, a user-centred design process should be followed which helps developers understand the usage
domain and user motivations. Through a collaborative process between the client, end-users and designers,
and as an outcome of user research, requirements can be determined for building prototype applications. In
the learning situation the provision of realistic contexts for design helps provide crucial exposure to the levels
of detail and nuanced user characteristics that designers will meet in the real world. Providing students with
the opportunity to develop practical design skills can help to bridge the gap between theory and practice. The
challenges of teaching interaction design have been discussed by Sas et al. (Sas, 2006) and include giving
consideration to the fact that as design is mostly a heuristic process, it requires a more reactive and flexible
approach by educators. It is important to encourage students to find alternative ways of thinking about the
material and develop the habits of critical thought.
This paper describes an assignment developed for a module in Interaction Design at Level 3 for the B.Sc.
Web Design at the University of the West of England (UWE) which takes these challenges into consideration
when defining the design brief. The proposed design brief was written to engage students with a real-world
scenario and give them the grounding to go from conceptual design through to a high-fidelity prototype using
an iterative user-centred design process. The assignment brief included background research evidence for
the design idea and high level requirements that students could then use to design and develop a high-fidelity
interactive prototype. The application to be designed was an interactive mobile solution that would enable
students to obtain support for social anxiety in relation to university work (presentations, seminars etc).
This application for the assessment is the outcome of an ongoing collaboration between the Department of
Psychology and the Department of Computer Science and Creative Technologies at UWE.

2. DEVELOPMENT OF THE ASSIGNMENT
The learning objectives for this Interaction Design module include developing cognitive skills with respect to
evaluating and designing interactive technologies appropriate to the context of use and analysing and
prioritising human, environmental and social requirements.
The module also has a strong research element requiring students to seek primary and secondary information
beyond the teaching delivered in class.
In the teaching of interaction design our focus is to engage students in human-centred design and as such
learn how to consider the close connection between the functionality of what they are designing and the end-
users’ needs and context, with a view to developing applications that are usable and useful, as well as
desirable.
Our goals for assessments at this level therefore include engaging students in work that not only provides a
realistic context to help them develop their skills for translating user needs into tangible design features but
also provides them the opportunity for creativity and to produce work that they can use to showcase their
skills. Students are encouraged to use their assessment work as part of their portfolios (Example Student
Portfolio, 2010) as an illustration to future employers of their practical skills for interaction design.
This assignment followed on from a previous requirements elicitation and design assessment where students
had researched a problem, and had gained skills in using a variety of ethnographic, qualitative and
quantitative methods to collate data from which they derived personas (archetypes of users) and scenarios of
use, using the evidence and insights gained to define a set of requirements. While this assignment gave
them some key practical skills and experience of data gathering, due to the limitations in time and resources
they were able to only focus on a limited perspective.
For their next assignment we wanted them to gain design skills, and work with data that provided multi-
dimensional perspectives of the context to enable taking a more holistic view when considering possible
design solutions.
Linking the design brief to the social anxiety research study provided access to background data collated by
psychologists and provided a high level of ethnographic validity and reliability. From this data we were in a
position to provide robust examples of well-rounded personas, contexts of use and other evidence as a
starting point for this next assignment.
3. BACKGROUND TO THE APPLICATION CONTEXT

Earlier research has shown that 10% of university students experience severe social anxiety in relation to
learning tasks such as delivering presentations, working in groups and speaking in seminars (Topham, 2009).
Social anxiety may include severe shyness and embarrassment, physical symptoms such as blushing and
sweating, an inability to speak, negative thought patterns about personal ability and fear of being ridiculed. In
extreme cases, students may experience panic attacks. In both mild and severe forms, social anxiety can lead
to avoidance of the stressful situation and withdrawal from participation.
Experience shows that students with social anxiety can learn to cope with it over time through practicing
cognitive behavioural tasks, relaxation techniques and through accessing counselling support from the
institution. Unfortunately, however, the condition means that students are wary of seeking support or even
admitting they are experiencing problems.
As a result of these findings, the Department of Psychology and the Counselling service, in collaboration with
colleagues from Plymouth University had been contemplating the provision of an online diagnostic, self-help
and informational tool to provide additional support to students. As one student put it in the study: "I would
personally like to see more provision for anonymous advice and help, identifying myself to the university as
having problems is in my mind like admitting failure" (Topham, 2009).

As a result of collaboration and development with the interaction design teaching team in the Department of
Creative Technologies, it was decided to focus on a mobile application where there would be a lot of scope for
creativity in considering alternative design ideas, whilst still providing students with access to a grounded real
world context for the design and subject domain expertise. The application has been dubbed SASHA (Social
Anxiety Self Help and Appraisal).
4. DEVELOPMENT OF THE SUPPORTING MATERIALS

A website (Supporting Website, 2010) was created which provided clear and structured access to the
information in a similar format that we had asked the students to produce. It also summarised the
requirements for the SASHA design task. It had the added benefit of serving as a clear exemplar of an
efficient way of presenting requirements data to a design team.
The high level requirements were that the application was:-
Essential:
1. A self-appraisal for arriving at (and storing) the user's social anxiety profile in terms of where and how
they are affected;
2. A facility for recording the intensity of their social anxiety profile and changes within it;
3. An opportunity to learn self-help procedures matched to their social anxiety profile;
4. The ability to share their feelings anonymously with others and to access comments shared by others;
5. Pathways to further information and support if required
Desirable:
1. Sensitive, customisable audio/visual reminders and notifications;
2. Event tracking with reminders tailored to the closeness of the event;
3. SMS integration or a full SMS interaction route;
4. Video/audio diary capabilities.
Students were advised that their prototypes did not need to fulfil all of these criteria, but that they should
address at least two of the essential requirements. This was an individual assignment, however group
discussions and prototyping activities in class gave the opportunity for collaborative peer learning.
Their designs needed to illustrate consideration of interaction design issues, which included ease of the
communication and interaction, suitability of the interface to support different modes of communication,
efficient mechanisms for switching between modes and smoothness of the transition; as well as social issues,
such the ability to share experiences and content, regard for social protocols especially anonymity and
privacy, facilities for providing awareness mechanisms and limiting scope for negative social phenomena to
occur.
There were two components to the assignment. The first component comprised a report on the design project,
which included their final prototypes illustrating how the different features of their application would be used,
and a reflective commentary discussing their design process and the impact of their approach on the final
prototype. The students were encouraged to maintain a design journal as part of their design process and use
excerpts to underpin their discussion. The second component was a presentation and viva where they were
required to demonstrate how they had interpreted and addressed the requirements of the SASHA project and
employed principles and guidelines for mobile interaction design.
In addition to the website, we also provided opportunities for the students to quiz the “client” for the project,
the psychologist researching social anxiety, who was able to answer queries based on his experience as the
domain expert.
The students also conducted competitor analysis of other similar apps with a view to identifying poor, as well
as useful, usability and interaction design features. The analysis was based on both co-operative and
heuristic evaluations, drawing on specific design guidelines for mobile applications.
For high-fidelity prototyping students were shown how to use MockApp (http://mockapp.com/), Photoshop
mobile design templates (http://www.teehanlax.com/blog/2009/06/18/iphone-gui-psd-30/) and
Genuitec’s Mobione Studio (http://www.genuitec.com/mobile/) which also works as an emulator.
5. OUTCOMES
The overall work produced had a high element of creativity with the students identifying some innovative
solutions in interpreting and responding to the requirements.
The students followed an iterative user-centred design process which included an analysis phase where they
had to explore the design context in more depth, a conceptual design phase where they focused on particular
scenarios to identify ways in which the requirements could be interpreted (Figure 1) and considered the
holistic nature of the interaction using techniques such as storyboarding (Figure 2), followed by iterative lo-
fidelity paper prototyping (Figure 3), leading to hi-fidelity prototyping (Figure 4).
Figure 1: Example of using annotation to analyse a scenario

Scenario-based design (Rosson et al. 2002) helped students to analyse the requirements from a more user-
centric perspective, developing an empathy with the potential user and an appreciation for the emotional
elements of the system. In response to the following statement in the given scenario: “But he knows that, later
tonight he’ll spend hours going over how badly he did, what he could have said, what people thought of it”, a
student added their own annotated comment (Figure 1) “… this is how SA [Social Anxiety] keeps
perpetuating. Needs to spend this time reviewing and congratulating himself on his progress. System – or the
advice the system points to – should make it clear that SA takes time to conquer, and not to lose heart when
there are setbacks or progress isn’ t moving fast enough.”

Figure 2: Example of Storyboarding
Realising scenarios via the use of storyboarding is a useful technique for encouraging engagement with the
interaction process in context (Holtzblatt, et al., 2005). Storyboarding helped students to explore different
contexts of use, which in turn helped them to develop clearer ideas regarding the pragmatics of use in
different situations and identify corresponding supporting features.

Figure 3: Paper Prototyping
The use of paper prototyping (Davies, in Sharp et al. 2007) proved to be an effective method for generating a
wide range of ideas quickly and enabled students to get early feedback on concepts. However, some students
find the process of generating rough sketches quite hard; they either get caught-up in trying to make their
sketches overly neat and tidy, thus defeating the objective of the process being time efficient, or they get
locked into an initial idea which they find very hard move away from, or they find it difficult to overcome the
feeling that drawing is “childish” and prefer to start making hi-fi prototypes. Conducting collaborative exercises
in class, with students who were more comfortable with paper prototyping and were able to demonstrate its
potential to the less convinced students, helped to overcome these barriers to some extent.
Students typically addressed requirements for monitoring anxiety status, a self-assessment questionnaire,
semi-anonymous social profiling and sharing of advice and support with other users (Figure 4). The best
quality solutions combined an appreciation of the problem domain with awareness of emerging norms for
touch screen interaction, economy of display usage and social knowledge sharing. Many of the delivered
prototypes were richly annotated, showing links between the design decisions and the contextual evidence.
They also illustrated interactivity through the use of branching or animation within the presentation slides.
We invited our client for the project, Phil Topham, from the Department of Psychology, to the presentations
and vivas and he was able to provide a useful feedback from his perspective in relation to the proposed ideas
and solutions. The range of ideas and interpretation of the brief by the students impressed him. The students’
work also provided him with an insight into the potential of the technology and enabled him, and us, to think
more broadly about the scope of using this in different situations; this would have been difficult to do without
these tangible exemplars.
Figure 4: Examples of Hi-fidelity Prototypes
Students were able to gain practical experience of using these design methods. Feedback on a student
portfolio incorporating this assignment work, from an international design consultancy, based in Bristol,
provides further acknowledgement of the relevance of the project. Here is a quote from their Managing
Director “…it's great to see a really good and well rounded portfolio coming out of the Web Design course with
a really practical set of skills that can be applied to the workplace.”
From a research perspective, another key outcome of this assessment was that on the basis of this
collaboration and conceptual design activity started by students, we have been able to successfully apply for
funding for a formal research study to design, develop and formally evaluate a social anxiety app. We hope to
use components of some of the design ideas generated by the students, and following contractual IPR
agreements, will be able to offer the students a proportional remuneration for their contribution as appropriate,
together with formal acknowledgement of their work. This student collaboration has proved to be an efficient
model for piloting concepts that can then be taken further in a research project and one that works in an
environment of competing demands on the academic’s time.
6. CONCLUSION
This paper presents an overview of a real-world interaction design assignment where the focus was on
encouraging students to explore the conceptual design space to find creative solutions. By providing them
with background information extracted from a real research study with access to a domain expert, we were
able to assist with enabling a deeper analysis of the context, which is often difficult to do within the resource
limitations of an academic course.
Also by engaging the students in a real case-study we were able to explore the potential for further developing
the work as a cross-disciplinary collaborative research project. Integrating research and teaching in this way
creates an exciting environment where students and teachers become co-learners.
We hope to involve the next batch of students in a follow-up study where we can explore implementation
issues and evaluation studies with real-end users.
7. REFERENCES

Davies, M., Paper Prototyping as a core tool in the design of mobile phone user experiences, Case Study
included in Sharp, H., Rogers, H., Preece, J., (2007) Interaction Design: Beyond Human-Computer
Interaction, 2nd Ed. Wiley.
Example Student Portfolio (2010), http://ansteychris.wordpress.com/
Holtzblatt, K., Wendell, J.B., Elsevier.Wood, S. (2005) Storyboarding, In Rapid Contextual Design, A How-to
Guide to Key Techniques for User-Centred Design, pp229-243, edited by Holtzblatt, K., Wendell, J.B.,
Elsevier.
Rosson, MB., Carroll, JM.(2002), Interaction Design, In Usability Engineering, Scenario based Development
of Human-Computer Interaction. pp159-195, edited by Rosson, MB., Carroll, JM. Elsevier.
Sas, C. (2006) Learning Approaches for Teaching Interaction Design, In HCI Educators Workshop, 23-24
March 2006, Limerick, Ireland.
Supporting Website for the SASHA Assignment (2010),

http://www.cems.uwe.ac.uk/~pmatthew/IxD/SASHA/index.html
Topham, P. (2009) Feeling stupid: A survey of university students’ experience of social anxiety in learning
situations. Project Report. University of The West of England Bristol, Bristol UK. (Unpublished). Available at:
https://eprints.uwe.ac.uk/164/2/SA_FINAL_REPORT_FEB_2009.pdf
8. Acknowledgements
We would like to thank Phil Topham for providing his expertise on social anxiety and inspiration for the
assignment. We would also like to thank the Interaction Design students, Class of 2009/10 for their work
towards the coursework assignment, in particular Chris Anstey, Jenna Lopez, Karis Cooper, Kieron Gurner,
Sean Payne and Alex Older whose work is featured in this paper.

Embedding Inquiry based learning into Programming
via Paired Assessment
Sonya A. Coleman Eric Nichols

School of Computing and Intelligent Systems, School of Computing and Intelligent Systems,
University of Ulster, University of Ulster,
Northland Road, Londonderry, Northland Road, Londonderry,
BT48 7JL, N. Ireland BT48 7JL, N. Ireland
sa.coleman@ulster.ac.uk nichols-e@email.ulster.ac.uk
ABSTRACT
Changes within our approach to teaching can make some students feel uncomfortable. To overcome this,
inquiry based learning, which is strongly supported by research in the areas of intellectual development and
approaches to learning (Prince, 2007), can be used. Inquiry based approaches should be introduced in
combination with existing teaching styles in order to address the needs of all students. Pair programming
enhances the communication among peers and encourages students to ask questions of each other and be
more ambitious in their computer programming practicals. The students subsequently gain confidence from
one another to try different approaches to solving programming problems; this enhances deeper learning.
Additionally, working in pairs provides some students with the courage to ask questions of the teacher while
with their pair, which they may not do alone. This paper presents a case study on using pair programming to
encourage inquiry based learning within programming modules, to improve attendance and practical
assessment results.
Keywords
Pair-programming; assessment
1. INTRODUCTION
The concept of pair-programming has recently been introduced in U.S. universities and companies. By
definition, pair programming is a technique in which two programmers work together on one task at one PC.
One programmer does the coding (the driver) while the other programmer reviews each line of code as it is
typed in (the observer); the two programmers frequently switch roles. The observer should also be coming up
with ideas for code improvements allowing the driver to focus all of their attention on the tactical aspects of
completing the current task (Williams). Within industry, pair programming improves many aspects of
development including design quality, reduced defects, enhanced technical skills and improved team
communications (Cockburn 2001). These are skills that we also require in our students. In 2000 (in Missouri
State University) students were asked their views of pair programming, 70% of them favoured it (Sanders,
2003). Much research has been carried out and published on the benefits of pair programming within both
industry and academia. For example, within an industrial setting, Arisholm et al., (2007) carried out a
controlled experiment in pair programming using a combination of junior, intermediate and senior Java
consultants within Europe. The results suggested that the most benefit was gained by the junior developers in
that they could complete tasks in less time and more successfully than normal with the support of a partner.
The research in (Dyba, 2007) indicated that, given the correct pairing, pair programming could be very
successful. Hence in this case study the students are permitted to select their own partners with the aim of
finding the right pairs. Many projects have been undertaken in pair programming in academia, but primarily in
the U.S. rather than the U.K. The work of Braught et al., (2008) determined that students are more likely to
successfully complete their course using pair programming and are also strengthened as individual
programmers. In (McDowell, 2006; Simon, 2008) this was also found to be true as students have more
confidence in their work and particularly in the case of females in computer science.
Much research has also been carried out and published on the benefits of both inquiry based learning and
working in small groups. Inquiry based learning is an approach to increasing retention through the use of a
teaching approach that allows student-constructed learning rather than teacher transmitted information (The
Sparks Foundation, 2010). An old adage states: "Tell me and I forget, show me and I remember, involve me
and I understand." This is the basic principle behind inquiry based learning: it is any task or activity that

involves the students to get them thinking for themselves, discussing problems together and ultimately
embedding deep learning. Therefore by introducing pair-programming, we are encouraging the students to
work together to discuss and solve problems with the aim that if they are involved in creating solutions then
they will have a better understanding of programming. It has many benefits including increasing student
involvement, encouraging collaboration among students and developing a deeper knowledge due to the
students thinking through the processes themselves (Prince 2007; Prince 2006). The key aspects of inquiry
based learning from the students point of view are that the students: are involved in the process of learning;
engage in an exploration process; raise questions, propose explanations, and use observations; plan and
carry out learning activities; communicate with staff and peers (Prince 2007). Ideally in any computer science
or engineering programme we aim to provide our students with an appropriate level of theory, combined with
practical applications that enable the students to investigate the theory for themselves, develop it and
understand it. Within tertiary education we all aim to have our modules as interactive as possible. However,
we need to be careful in getting the balance correct: it is a well known fact that different students learn in
different ways; some learning from doing, others from reading, some even learn better with background music.
Therefore significant changes within our approach to teaching could make some students feel uncomfortable,
and hence in this paper we have introduced inquiry based combination with existing teaching styles in order to
address the needs of all students.
In this paper we present initial findings on the use of pair-programming as a means of embedding enquiry
based learning activities into an algorithmic programming module. We find that the students enjoy working in
pairs and it provides an overall improvement in the module assessment marks.
2. METHODOLOGY
Pair programming as a means of inquiry based learning was introduced into the Algorithmic Programming II
module in semester 2 of the academic year 2009-10 (AY0910). There is a wide range of first year students
who take this module including BSc Computer Science, BEng electronics and Computer Systems and BEng
Games Development. With respect to carrying out practicals and practical assessments, the students were
put into pairs.
2.1 Pairing
In total there were 58 students on this module. The students were allowed to choose their own partners, as
randomly selecting partners can lead to many problems during the course of the module. In week 1 the
students were ask to email the author with details of the partnership (one email per pair). There were initially
some problems with this. Some students selected their partner without asking them if they were willing to be
their partner and this resulted in some students being in two pairs. These issues were raised in the lecture
and students were asked to solve it among themselves. Some students were also reluctant to work in pairs,
but it was not optional, so they had to commit to working with someone. Of course, by week 3, a few students
had dropped out and so their partners had to be paired with someone else. However, none of these issues
were major, and were easily resolved prior to the first assessment taking place.
2.2 Assessment
In their pairs, the students completed the practicals and were assessed. However, assessments were still
conducted on an individual basis in the following manner:
a) both students had to be present at assessments in order to obtain their individual mark;
b) both students were asked individual questions about the practical, hence both students had to
demonstrate their own understanding.
By conducting the assessment in this way, no individual student received credit for their partner’s work.
The assessments were conducting during the timetables practical time. Approximately 5 to 10 minutes was
spent with any one pair and so they were able to return to their normal practical work. At the beginning of the
practical class, all students were requested to select which PC the practical would be running on and have it
running ready for the assessment to be conducted. The practical class would be commenced as normal and
then the assessment would begin. The assessor simply asked each pair to run their program, demonstrate
their output and then show their code.

Each student in the pair was also asked individual questions to gauge whether each student understood the
code or if one had essentially done the work. Each assessment was marked out of 10, with 2 marks given for
each of the following:
1. program running;
2. correct output;
3. good code structure;
4. good commenting;
5. understanding.
Obviously, the marks allocated for 1 – 4 were the same of each students, but an individual mark was then
allocated for understanding. This was adequate over the five assessments to ensure individual marks.
At the end of the module, the success of pair programming was assessed in two ways: a statistical analysis
was conducted to assess the overall improvement of coursework marks and attendance at the practical
sessions in contrast to the previous year; the students were given a questionnaire in order to collect and
analyze their views on pair programming. As this project was carried out in second semester, the first year
students had already completed one programming module in first semester without using pair programming,
so they were able to compare and contrast the advantages and disadvantages of both learning styles.
It should be noted that practical assessment was only one component of the coursework for this module.
There were two class tests which each student took independently and these were combined with the paired
assessments marks to create a individual assessment mark for each student.
In addition, after the exam component of the module was completed, additional statistical analysis was carried
out to ascertain if the overall exam performance has improved in comparison to the previous year, thus
providing evidence that pair-programming leads to deeper learning and understanding. However, this will also
have to be cross correlated with other module marks to determine if an overall improvement is unique only to
the Algorithmic Programming module, or if indeed the student cohort is academically superior to the previous
year.
3. STUDENT VIEWS
Question % response of ‘yes’
Do you enjoy working in pairs? 76.2
Would you have preferred to work on your own? 28.6
Would you have preferred to work in larger groups? 28.6
Do you feel working in pairs has helped your understanding 57.1
of programming?
Do you feel you have learnt from your programming partner? 57.1
Do you feel your programming partner has learnt from you? 42.9
Did you and your partner work together on programming 47.6
practicals outside the assigned time?
Table 1. Summary of student feedback
At the end of the module, the students were issued with a questionnaire in order to obtain their views and
opinions on pair programming. They were asked to compare and contrast the pair programming technique
used in this project with the solo programming approach used in their first semester programming module.
This provided us with the student’s overall view of pair programming. In total, 21 students completed the
questionnaire out of a possible 58. The questions and the percentage of students that answered ‘yes’ to each
question is summarised in Table 1.
As illustrated in Table 1, 76.2% of the student cohort enjoyed working in pairs. The general feedback
suggests that knowledge transfer within the partnerships did occur. The evidence also indicates that the
students prefer working in pairs rather than larger groups which could potentially have impact on many other
modules that incorporate group work as means of assessment. It is also interesting to note that 47.6% of the
student cohort worked on their practicals together outside of the timetabled practicals; perhaps the use of pair
programming helps to strengthen bonds and friendships amongst first year students.

4. EVALUATION
4.1 Quantitative Evaluation

The quantitative results of embedding pair programming into the Algorithmic Programming II module are
presented. In order to conduct statistical analysis thoroughly, we firstly compared the attendance with that of
the Algorithmic Programming II module in AY0809 to ascertain if attendance had improved. In addition, to
ensure that any improvement in attendance was not dependent on the specific cohort, we analysis the
attendance of a Mathematics module in semester one of AY0910 to obtain expected attendance profiles.
Details of how the analysis was conducted are in Appendix A.
An overall aim of embedding pair programming as a means of enquiry based learning in the programming
module was to improve practical attendance, enhance deep learning and thus improve practical assessment
marks with the ultimate goal of improving first year retention. With respect to practical attendance, we found
that there was an overall increase of 2.5% in practical attendance in comparison to the AY0809. Additionally,
when we cross-correlated the second semester data with first semester data for the same cohort we found
that the attendance was significantly higher than expected: practical attendance was 27.8% better than
expected. We investigated whether working in pairs during the practical was having any impact on the
attendance at lectures and found lecture attendance was 10.35% better than expected. This implies that
working in pairs encouraged students to attend in second semester, however there is still room for significant
improvement in the overall attendance.
We subsequently analysed the overall practical assessments for the module to determine if pair programming
improved the practical assessment results. Analysis showed the practical assessment marks had increased
by 31.2% in comparison to last year’s practical assessment marks and were 26.2% higher than the
expectation. Cross-correlation with other modules has also been conducted which highlights that attendance
and performance in the Algorithmic Programming module is slightly better than the Mathematics module.
However, it is not yet clear whether this is strictly due to the pair-programming or the students’ module
preference and hence next year we will try to gauge this via the student questionnaire by including questions
which will advice us about which modules students like and dislike.
In addition to the analysis conducted, we analysed the attendance and performance of the students we
deemed to have engaged in the module, i.e., those that submitted all of their coursework and completed the
exam. If we look only at these students and compare their performance in Algorithmic Programming II with
that of other modules which the same student cohort has completed, we find a significant improvement in both
attendance and assessment as demonstrated in Table 1.
% improvement
Practical attendance – all students +20.6%

Practical attendance – females +21.8%
Lecture attendance – all students +19.9%
Lecture attendance – females +15.7%
Assessment marks – all students +53.5%
Assessment marks – females +61.0%
Table 1 – Summary of cross correlation with the mathematics module
4.2 Qualitative Evaluation

From the questionnaire feedback and general discussion with the students, it was found that they enjoyed the
pair programming aspect of the module. Not only did they conduct their practicals in pairs but this also
extended to going to lectures together and working on tutorial problems together; some students even started
to travel together. The fact that they had permission to work in paired seemed to generate constructive
conversion between the students during practicals, and indeed tutorials, where they were talking to one
another to actively work through problem, bouncing ideas of one another and coming up with the optimal
solution. This is one of the clear goals of inquiry based learning in that students work together and discuss
problems and are therefore learning from each other in a deep and meaningful manner.

In was also found that paired programming suits the female members of the class very well. Traditionally
females felt out-numbered by males in computer science and related disciplines. Working in pairs (and they
all worked with another female) gave them the confidence to work through problems together and also to ask
questions of the academics together; this is vital in the challenge of retaining female students within these
domains.
The use of pair programming enabled practicals to be assessed in an easy manner particularly with large
module sizes and as the module size increases, it is possible to increase from pairs to small groups. It should
be noted however, that although the pair programming did encourage students to attend practical and lecture
and the module coursework marks were notably better than previous years, this was not reflected in the
examination marks. The examination marks were similar to previous years with no significant improvement
noted.
5. DISCUSSION
The idea behind this project was to introduce a practice in the Algorithmic Programming II module that would
encourage the students to engage and participate in the module and ultimately improve first year retention.
Now we need to consider: did we achieve this? Did we go some way towards achieving this?
Within the U.K. and Ireland, the issue of student retention is prominent. However, throughout the academic
year we can often determine which students are likely to fail first year based on their attendance and amount
of engagement in the degree program. This is particularly prevalent in computer science related disciplines as
students often learn Information and Communication Technologies ICT in schools and think this is computer
science. Therefore when entering the degree program many discover it is not what they first thought.
The main reason for introducing pair programming into the programming module was to get students to
engage with each other and the module, improving attendance, performance and overall retention. The
attendance at practicals improved in comparison to the attendance in AY08/09. As illustrated in Table 1, the
practical assessment marks increased on average by almost 54% compared with the expectation. Table 1
also illustrated improved performance among the females in the class which is also an expectation of pair
programming.
The next major aim of this project was to enhance learning by encouraging students to attend their practicals
and practical assessments and therefore develop a better understanding of programming than in previous
years. The improvement in learning would have been determined by improved class test and examination
marks. Unfortunately this was not the case. The average class test mark dropped by 2% and the overall
examination marks were similar to previous years. However, this was primarily due to approximately 8
students who never engaged in the module and did not sit the exam. This is still a relevant point as the aim of
the project was to improve engagement and attendance. Pair programming cannot benefit those who do not
take part. However the students who participated in pair programming clearly benefitted.
6. CONCLUSION
We have presented an approach to embedding inquiry based learning into algorithmic programming modules
via the use of pair programming. With respect to pair programming students work together in pairs and are
also assessed in pairs in their practical assessments. We have presented initial findings that illustrate that the
use of pair programming encouraged students to attend lectures and practicals and also, overall, the module
assessment marks were significantly higher than expected based on the cohorts experience performance and
the assessment marks from previous years. However, it is also evident that the use of pair programming did
not improve the examination marks in any significant way.
APPENDIX A
In order to obtain the expected results, the total possible attendance from the first semester's module was
obtained by multiplying the number of students by the number of weeks. Next the attendance every week of
the semester was summed and divided by the total possible attendance to obtain the attendance percentage.
This was carried out for the current and previous academic years. The comparison of these first semester
attendance records was then used against the previous year's Algorithmic Programming II module attendance
to achieve the expected attendance for the current Algorithmic Programming II module. The same method

was used to obtain expectations of practical assessment results. The benefit of using this method is that a
difference is quantified and applied to the same students in different years for attendance, assessments and
class tests. This way an amount of improvement or decline in the students without pairing is found, leaving the
difference in the analysis as the result of pairing; not ability.
7. ACKNOWLEDGMENTS
The authors would like to thank the Higher Education Academy Information and Computer Science Subject
Centre for funding this project.
8. REFERENCES
1. Arisholm, E., Gallis, H., Tore, D., Sjoberg, D.I.K., “Evaluating Pair Programming with Respect to System
Complexity and Programmer Expertise” IEEE Trans on Software Engineering, Vol. 33, No. 2, Feb 2007.
2. Braught, G., Eby, L.M., Wahls, T., “The Effects of Pair-Programming on Individual Programming Skill”
SIGCSE, March 2008.
3. Cockburn A., Williams, L., “The Costs and Benefits of Pair Programming” The XP Series, Extreme
Programming Examined, pp. 223-243, 2001
4. Dyba, T., Arisholm, E., Sjoberg, D.I.K., Hannay, J.E., Shull, F., “Are Two Heads Better than One? On the
Effectiveness of Pair Programming” IEEE Software, Vol. 24, No. 6, pp12-15, Dec 2007.
5. McDowell, C., Werner, L., Bullock, H. E., Fernald, J., “Pair Programming Improves Student Retention,
Confidence and Program Quality” Communications of the ACM, Vol. 49, No. 8, August 2006.
6. Prince, M., Felder, R.W., “The many faces of inductive teaching and learning” Journal of College Science
Teaching, Vol. 36, No. 5, pp14-20, March/April 2007
7. Prince, M., Felder, R.W., “Inductive teaching and learning methods: Definitions, comparisons, and
research bases”. Journal of Engineering Education 95 (2): 123–38, 2006
8. Rice, P., “Facilitating learning in small groups” http://www.heacademy.ac.uk/
9. Sanders, D., “Student Perceptions of the Suitability of Extreme and Pair Programming” Computer Science
Education, 2003
10. Simon, B., Hanks, B., “First-Year Students’ Impressions of Pair Programming in CS1” ACM Journal on
Educational Resources in Computing, Vol. 4, No. 7, Jan 2008.
th
11. The Sparks Foundation (2010). Why promote inquiry based science programming? Retrieved August 6
2010. http://scienceexplorers.com/wp-content/uploads/promoting-inquiry-based-learning.pdf
12. Williams, L., "Integrating Pair Programming into a Software Development Process" (PDF). 14th
Conference on Software Engineering Education and Training, 2001

EMERGENT REQUIREMENTS FOR SUPPORTING INTRODUCTORY
PROGRAMMING
NATALIE J COULL Ishbel M M Duncan

School of Computing and Engineering School of Computer Science,
Systems, University of Abertay Dundee, University of St Andrews
Bell St., Dundee, DD1 1HG North Haugh, St Andrews, KY16 9SX
(+44 1382 30 8938) (+44 1334 46 3270)
n.coull@abertay.ac.uk Ishbel@cs.st-andrews.ac.uk
_____________________________________________________________
ABSTRACT
The problems associated with learning and teaching first year University Computer Science (CS1)
programming classes are summarized showing that various support tools and techniques have been
developed and evaluated. From this review of applicable support the paper derives ten requirements that a
support tool should have in order to improve CS1 student success rate with respect to learning and
understanding.
Keywords
Introductory programming, learning, design, human factors, theory.
1. INTRODUCTION
Students learning to program are often initially frustrated by not only having to learn code syntax, but by
having deeper issues of understanding than is evident from any compilation failure. Students require to
engage with programming on both a cognitive and a conceptual level and, as help is not always immediately
at hand in a busy laboratory or in private development, an automated tool supporting those concepts, as well
as fundamental syntax and semantics, would be a useful addition to their educational armoury. The work
described here is the first part of a doctoral research project to determine what was necessary in a student
support tool for learning to program, and, secondly, to build and evaluate such a tool. The aim of the initial
research, therefore, was to review and then synthesize current research and practice in the field and to
determine fundamental support requirements that any learning system should have.
The paper outlines the current state of the art in learning to program research and practice followed by a
discussion of the applicable learner models. These give rise to the emergent requirements for a support tool
and the conceptual architecture for such a tool is then outlined. The paper concludes with a short summary of
the second part of the research project in which the requirements were substantiated within a prototype tool.
2. RELATED RESEARCH
The specific task of learning to program attracts particular challenges, and it is well recognized as a time-
consuming and frustrating process for the majority of novices (Johnson 1990; Edwards 2003; Kelleher and
Pausch 2005; Bennedsen and Caspersen 2007; Eckerdal 2009). Progress has been made in terms of
identifying the underlying difficulties faced by novices by investigations of possible underlying influences
(Simon, Fincher et al. 2006) and, separately, direct comparison with expert programmers (Mead, Gray et al.
2006; Kay 2007). Some older studies, for example, Soloway and Ehrlich (1984) explored the difference
between novice programmers and experts. These older studies lay the foundation of our current
understanding of the programming process. The Soloway and Ehrich paper states that the process of
programming requires at least two different skill sets: program comprehension (i.e. the ability to understand
each program part) and, secondly, plan recognition (i.e. the ability to understand the interactions among each
program part). Program comprehension is dependent on an understanding of the form and meaning of each
symbol and each line, and this skill is necessarily evident in experts. This study also demonstrates that
understanding arises as a result of recognition of the interaction among program lines in the form of a

systematic plan. This plan recognition is evidently present in experts but not in novices. Similarly, Mayer
(1981) considers the differences between novices and experts and presents a useful analogy to programming
in terms of playing chess. Mayer discusses a study where subjects were presented with chessboard
configurations and then asked to reconstruct them from memory. Although chess masters performed much
better than less experienced players when the chessboards came from real games, there was little difference
when the chessboards contained randomly placed pieces. This demonstrates that the chess masters do not
have better memories than the less experienced players, but an understanding of the patterns that can exist
among pieces on the board. Mayer compares this to a study by Shneiderman (1980) where expert
programmers were able to recall more lines of code than novices in a meaningful program but performed no
better than novices when the program contained random lines. Mayer suggests that it could be useful to
teach novices to program using “chunks” or “schemas” to help them learn the patterns evident in
programming.
More recently, Kay (2007) suggests using errors to determine and then ultimately predict erroneous actions of
beginners, intermediates and then advanced programmers. He also states, along with Ebrahimi and
Schweikert (2006) that mistakes came from incorrect plans or mental models and a previous experiment
(Lazonder and Van Der Meij 1995) demonstrates that novices require context specific information when an
error has occurred. Ebrahimi (1994) undertakes a review of existing work on the difficulties faced by novice
programmers and clearly states that the two challenges noted by the studies above impact significantly on the
learning process for novices. It is clear that knowledge of the syntax and underlying semantics is required to
develop programs. However, Ebrahimi states unambiguously that program plans are also a fundamental
requirement to successful program development. Ebrahimi also shows, through observational study, that the
concepts of “language constructs” and “plan composition” are central to programmer development and
intimately linked. Ebrahimi concludes that programmers cannot form plans without language and cannot use
language without plans and that, consequently, these must be taught concurrently.
The literature described above demonstrates that there are two challenges in learning to program. The first,
here termed program formulation is the syntax and semantics of the individual program parts that are
fundamental to the formulation of any program. The second, here termed problem formulation is therefore the
challenge of identifying and structuring the necessary program components to formulate a programmed
solution to a specific problem.
Both Fuller et al. (2007) and Rountree et al. (2005) state that code comprehension and programming are two
separate skills and need to be developed. Lister et al. (2004) investigated students’ ability to trace programs.
All these skills are considered essential for developing a student along the path towards experience. Malan
and Halland (2004) identified pitfalls to avoid when teaching programming, specifically identifying the problem
of teaching being too abstract and confusing the novice student. Edwards states that students need “explicit,
continually reinforced practice in hypothesizing about the behaviour of their programs….[and] need frequent,
useful and immediate feedback” (2003). Students may believe that once their program compiles then it works
and must be correct. This suggests that their mental model is perhaps not accurate or that they have failed to
understand the logic of their code. Murphy et al. (2008) states that novices behave unproductively when
debugging their programs implying confused mental models. Norris et al. (2008) notes that better performing
students spend less time on a task, using fewer compilations and writing less code implying better mental
models of their code.
Lahtinen et al. (2005) suggest that studying alone and example programs were the best mechanism for
learning and Fetaji et al. (2007) states that learning to program in a classroom approach is not the most
appropriate strategy. Hassinen and Mayru (2006) support a learning-by-doing theory and Shaffer (2003)
further suggests limiting the number of concepts introduced in a class. Further, Pillay (2003) states that one
on one tutoring is the best mechanism for helping novice programmers.
3. LEARNER MODELS
In order to improve the learning process that underpins the development of expertise in programming, a
number of models of student learning have been devised (Mayer 1981; Mayer 1997; McGill and Volet 1997).
These models are typically formulated in terms of stages in progressing from a novice to an expert
programmer and can be used to identify and describe the composite aspects of the activity of learning to
program. In turn, these activities, and the accordant skill base, can be used to determine the direction that we
expect students will follow and provide guidance and structure on how we need to help students begin this
process of learning.

Lund (2002) recognizes the utility of the work by Mayer (1997) in particular and has proposed a unified
framework of programming expertise to underpin the development of a teaching support tool, which clearly
defines the categories of knowledge required to program. These categories are identified, as per Mayer, as
Syntactic, Semantic, Schematic and Strategic. These levels can form the foundations of the learning support
that should be offered to students and can be mapped explicitly on to the two challenges presented above,
program formulation and problem formulation. The existing description of each challenge is further refined by
the use of Mayer’s learning model;
Program Formulation
Program Formulation encapsulates both the Syntactic and Semantic knowledge levels of Mayer (1997).
According to Lund (2002), Syntactic knowledge is specifically concerned with the syntax, or form, of a
programming language and the ability to apply that knowledge. Correct syntactic expression of constructs
within a particular programming language allows development of Semantic knowledge, i.e. the operation of
those individual constructs, which will enable them to develop a mental model of program execution or “what
goes on inside the computer as a result of a line of code” (Lund 2002). This mental model, combined with
knowledge of the underlying syntax, represents the skill set needed for program formulation.
Problem Formulation
Lund also defines Schematic knowledge as the ability to recognize patterns in code developed for previous
problems, also known as programming plans, and apply these plans to form a solution to the current problem.
As noted earlier, novice programmers have difficulty recognizing these patterns and, even when the patterns
are made explicit in one context, they are unable to transport these patterns to a given problem, i.e. another
context. The final layer in the model, Strategic, may be defined as the techniques employed to create and
monitor plans, so following a software development process (Lund 2002). Therefore, knowledge at the
syntactic and semantic levels maps directly on to program formulation and knowledge at the schematic and
strategic levels maps directly on to problem formulation.
Simon et al. (2006), state that students with pre-existing strengths in strategic or algorithmic articulation are
invariably successful programmers. One could interpret this as these students progress quickly through the
levels because they are already thinking at the problem formulation level and can therefore cope with lower
level issues such as syntax errors. Kordacki (2007) implies the ability of learners to use representation
systems to express their problem-solving strategies will make for better learners, suggesting the use of both
program and problem formulation. Soloway and Ehrlich (1984) previously claimed that a major source of code
problems came from plan composition failures, i.e. program formulation. A similar claim was also made by
Winslow (1996) when he stated that “..novice programmers know the syntax and the semantics of individual
statements, but they do not know how to combine these features into valid programs.”
4. EMERGENT REQUIREMENTS
From reviews of existing tools (Kelleher and Pausch 2005; Coull 2008) it is evident that a support tool may be
used to facilitate the learning process of novices who are learning to program. Further, from this review, a
series of requirements emerge. Generally, there is diversity in methods and approaches used to teach the
complex, interleaved concepts associated with programming. Therefore, a support tool that is sufficiently
flexible to accommodate many different instructional and delivery designs is of the most value. Ideally, the
support tool should be decoupled from any specific development requirement and further operate as an add-
on to any chosen programming environment. No support tool that aids program formulation with respect to
syntax should promote dependence on that tool to such an extent that the student is not able to progress
beyond the remit of that tool. To remove this dependence, and to benefit from support that takes account of
the taught context, it is necessary to present the student with both the standard compiler messages and any
supplementary error messages enhanced with context-relevant support. Again at the program formulation
level, and in recognition of the frequent semantic errors which students make, a support tool should detect
these common mistakes and provide relevant warnings to the novice. To further reduce the risk of
dependency the level of support should be reduced over time. In addition to these steps, the use of a support
tool should be voluntary as this affords students the opportunity to direct their own learning and allows those
students with existing experience to opt out of extended support provision and develop programs more
quickly.
To support a student more fully in the development of a solution to a given problem, i.e. problem formulation,
a support tool needs to have knowledge of both the key constructs and the relationship among those
constructs necessary for a working solution. A support tool should be able to offer assistance to the student in
identifying which constructs are needed to solve a particular problem and make sure that their solution meets

the requirements of the exercise. This in itself raises two issues. First, it is important that a support tool does
not direct a student towards a single model solution, but allows the student to develop a correct solution of
their own design, i.e. the support tool must recognize that variant solutions exist and these should be
supported. Second, the tool must provide support in stages, guiding the student through the development
process, rather than directing the student towards a complete solution from the outset or only operating with a
near solution. Finally, to reinforce concepts already taught to the students, a support tool should be able to
direct novices toward relevant, context sensitive (i.e. the general taught context such as terminology and
bespoke teaching packages) and context aware (i.e. the specific teaching activity currently undertaken)
teaching materials. The feedback provided, including links to teaching materials, should provide an effective
platform upon which to base student-tutor dialogue.
In summary, Figure 1 below lists the requirements of a support tool:
1 Present both standard compiler and enhanced support concurrently

2 Link to teaching resources as a means of information delivery and student-tutor dialogue
3 Identify and advise on commonly observed semantic errors
4 Embody knowledge of key constructs needed to solve a given problem
5 Embody knowledge of the relationships between the constructs needed to solve a problem
6 The knowledge should be disseminated to students in successive stages
7 Where appropriate, ensure that this knowledge accommodates variant solution forms
8 Provide support for different problems
9 All forms of support may be progressively reduced over the teaching period
10 Use of the tool must be voluntary on the part of the student
Figure 1 Core Requirements of an Effective Support Tool
5. A CONCEPTUAL ARCHITECTURE FOR LEARNING SUPPORT TOOLS

After identifying all of the requirements identified above, the next phase of the research was to develop the
architecture for support tool development. The architecture identifies the components needed in a system to
develop a technology-based tool that supports novices learning to program. Figure 2 illustrates those
components and the interrelationships among them. Central to the architecture is a software implementation
that draws on established resources and integrates the different aspects of support provided into a single
interface. These aspects are outlined here to provide an overall context and are then explored more fully in
subsequent sections.
The architecture comprises five main dimensions: Syntactic, Semantic, Schematic and Strategic structure
support together with interaction, i.e. invocation and feedback, with the support tool itself. To provide
Syntactic Support for program formulation it is necessary to supplement the (original) error messages arising
from the compilation process with extended text, where this text is sensitive to the taught context. To provide
Semantic Support for program formulation it is necessary to perform checks on the program code for common
(logic) errors and provide warnings to students, which indicate these logic errors. Knowledge of the problem
domain (a specific practical exercise) may be used to inform the development of strategic and schematic
checks for solution formulation, i.e. checks for key constructs and the relations among those key constructs
respectively. Through this mix of syntax, semantic, schematic and strategic checks, the support tool may
provide feedback to the student. This support must be integrated into the current development environment to
provide a seamless interface that is available whenever students elect to use it.

Figure 2 Generic Architecture
Program Formulation Support

Supporting the student in writing syntactically correct code is imperative for program formulation. As
discussed earlier, difficulties with syntax can affect student learning of both the language and the underlying
concepts. Many universities use industry-standard programming languages as a tool for teaching introductory
programming. The compiler error messages produced by these programming languages are designed for
expert programmers, and so their successful interpretation may depend on a degree of knowledge not
possessed by a novice. Rather than replace these expert level error messages, these messages should be
extended with content more suitable for novices (Requirement 1). Since the student should always be
presented with the original error message, they should soon recognize with the aid of the extended messages
why these errors have occurred and be able to resolve compiler-only errors in due course. These extensions
should use dialogue that the student is familiar with and, where possible, relate to the content of the actual
module (Requirement 2).
Similarly, Semantic Support is also very important for program formulation. Although problems at this level do
not occur with the same frequency as syntax errors, their presence is less obvious to the novice, and so they
are harder to detect, since they are manifest only at run-time. As there is no compiler support for these errors
students may spend a long time trying to identify and then rectify the problem. Indeed, based on observations
(Jadud 2006) students often make structural changes to their program in an attempt to correct its behaviour
rather than identifying a (more trivial) semantic error. Some other support tools, for example BlueJ (Johnson
2006), recognize the value of providing support for semantic problems and have incorporated a series of
semantic checks into the software support. To address Requirement 3 a support tool should perform checks
on syntactically valid code for commonly observed semantic errors. Again, the dialogue used in feedback
should be something that the student is familiar with and, where possible, relate to the content of the taught
material (Requirement 2).

Problem Formulation Support
To successfully formulate any program a novice must have sufficient knowledge of the syntactic and semantic
aspects of the language. However, the formulation of a solution to a given programming problem requires
identification of the required structure of the program, i.e. the interrelationships among the component parts of
the program. Many students need assistance with identifying such a program schematic (Lahtinen, Ala-Mukta
et al. 2005). A support tool that has knowledge of the appropriate constructs (Requirement 4) and these
interrelationships (Requirement 5) required for a specific programming exercise can be used to fully guide the
student towards creating a solution. Rather than present the student with a list of all the necessary constructs
and interrelationships, students should be guided in a stepwise manner through a solution development
(Requirement 6). This guide may then exploit the schematic patterns inherent in introductory exercises to
promote a systematic approach to software development. Perhaps the most fundamental and yet
conceptually challenging level of learning occurs at the strategic level. Here, a support tool should check that
the basic components required to solve the problem are in place (Requirement 4). Problems in identifying this
basic set of components are driven by a failure to recognize the (often implied) functional requirements stated
in a question and how those requirements are translated into the program development process. A support
tool should make clear, again in a stepwise manner (Requirement 6), that translation by making transparent
the links between program requirements and the components needed to meet those requirements. Further,
both here and in the Schematic Support offered, the support tool must recognize that there may be more than
one possible solution for a given problem and provide support for a range of solutions appropriate to the
problem posed (Requirement 7). Finally the tool should provide support for different types of problems
(Requirement 8), ideally the range of problems covered in their course.
Feedback
To enable students to become more confident in their abilities, and develop a skill set independent of any
additional support, it is necessary to reduce the amount of feedback provided as teaching progresses
(Requirement 9). The progressive reduction in support over the taught period will ultimately eliminate
dependence on a support tool. To ensure that the feedback is relevant and contains terminology that the
student is familiar with, the feedback should make a direct relation to core and/or supplementary teaching
material where possible (Requirement 2). To direct their own learning students must have the opportunity to
work with just the basic toolset, for example the compiler messages, and so invocation of any software
support must be at the individual student’s discretion (Requirement 10).
6. EXPERIMENTAL EVIDENCE
To evaluate the derived requirements and the generic architecture, a prototype implementation, SNOOPIE,
was built and linked to specific programming courses. The remit and feedback of the tool support was initially
informed by an extended series of laboratory observations. These demonstrated the multi-faceted nature of
problems that students encounter whilst they are learning to program and how these problems can be
mapped onto the different levels of program and problem formulation. Detecting erroneous problem
formulation is not easy to identify and analysis was done through interview, questionnaires and experimental
evidence. Erroneous program formulation can be detected through compilation errors and the number of
attempts and time taken to achieve a solution. The support tool was fully evaluated and demonstrated to have
a significant impact on the learning process for novice programmers. From a series of experiments with
multiple student cohorts over two years in two universities each using a different series of laboratory practicals
and lectures, five hypotheses were evaluated: that program formulation support improved short-term student
performance, that problem formulation support improved short-term student performance, that problem
formulation support reduced the time taken to achieve an ideal solution, that a combination of problem and
program formulation improved long-term student performance and that the system was perceived as having a
positive impact on the student learning process. Hypotheses 1 to 3 were tested using laboratory experiments,
hypothesis 4 was analysed using student performance at the module level combined with interviews with staff
and students and finally, hypothesis 5 was tested via experiments, questionnaires and interviews with
students. The results demonstrated that, using the SNOOPIE tool:
1. Students receiving program formulation support were able to solve errors faster than those without
support, i.e. program formulation support improves short-term performance
2. Students receiving problem formulation support were more able to produce a solution that met the
criteria than those without support, i.e. problem formulation support improves short-term performance
3. Students receiving problem formulation support took longer to produce a solution than those without
support. It was concluded that students were not being “spoon-fed” solutions and took time to think in

detail about a problem and consequently problem formulation support does not reduce the time taken
to achieve an ideal solution.
4. Student confidence was seen to improve through the use of the support tool and although the results
were less significant than those in earlier hypotheses, student long-term performance with both
program and problem formulation support was better than those with only program formulation
support.
5. Students perceived the support tool as adding value and having a positive impact on their learning
experience.
7. SUMMARY
Learning to program is recognized as a complex task that novices find challenging. Existing literature
demonstrates that learning to program is difficult because of the need to learn the rules and operation of the
language (program formulation), and the concurrent need to interpret problems and recognize the required
components for that problem (problem formulation). There exist many endeavours to support the novice in
this activity, including software tools that aim to provide a more supportive environment than that provided by
standard software facilities. This paper presents a new methodology for developing learning support tools that
addresses the dual task of program and problem formulation. A review of existing teaching tools that support
the novice programmer lead to a set of requirements for a support tool that encompasses the processes of
both program and problem formulation. The set of requirements is encapsulated in a generic architecture for
software tool development. A prototype support system was then built from the architecture and a series of
hypotheses was developed to determine system validity. By using a combination of evaluation methods
including anecdotal evidence, interviews, questionnaires and experiments, the results demonstrated how a
series of automated analyses at different stages in the student's development of a solution improved both
student confidence and performance.
8. REFERENCES
Bennedsen, J. and M. E. Caspersen (2007). "Failure Rates in Introductory Programming." Inroads (SIGSCE
Bulletin) 39(2): 33-36.
Coull, N. J. (2008). Support for Novices in an Object Oriented Programming Integrated Environment. School
of Computer Science. St Andrews, Scotland, KY16 9SX, University of St Andrews. PhD.
Ebrahimi, A. (1994). "Novice programmer errors: language constructs and plan composition." International
Journal of Human-Computer Studies 41: 457-480.
Ebrahimi, A. and C. Schweikert (2006). "Empirical Study of Novice Programming with Plans and Objects."
Inroads (SIGSCE Bulletin) 38(4): 52-54.
Eckerdal, A. (2009). Novice programming students' learning of concepts and practice. Information
Technology. Upsalla, Sweden, University of Upsalla. Ph.D.
Edwards, S. H. (2003). Rethinking Computer Science Education from a Test-First Perspective. OOPSLA’03
148-155.
Fetaji, M., S. Loskovska, et al. (2007). Combining Virtual Learning Environment and Integrated Development
Environment to Enhance eLearning. International Conference on Information Technology Interfaces.
Croatia.
Fuller, U., C. G. Johnson, et al. (2007). "Developing a Computer Science-specific Learning Taxonomy."
SIGCSE Bulletin 39(4).
Hassinen, M. and H. Mayra (2006). Learning Programming by Programming: A Case Study. Baltic Sea '06
Koli Calling.
Jadud, M. C. (2006). Methods and Tools for Exploring Novice Compilation Behaviour. Proceedings of the
2006 International Workshop on Computing Education Research. Canterbury, UK, ACM: 73-84.
Johnson, C. (2006). Abstract interpretation of student programs as a strategy for courseware development. In
Methods, Materials and Tools for Programming Education, Tampere Polytechnic: 14-19.
Johnson, W. L. (1990). Understanding and debugging novice programs, Artificial intelligence and learning
environments. Scituate, MA, Bradford Company.
Kay, R. H. (2007). "The role of errors in learning computer software." Computer & Education 49: 441-459.
Kelleher, C. and R. Pausch (2005). "Lowering the Barriers to Programming: A Taxonomy of Programming
Environments and Languages for Novice Programmers." ACM Computing Surveys 37(2): 83-137.

Kordacki, M. (2007). Modeling and multiple representation systems in the design of a computer environment
for the learning of programming and C by beginners. World COnference on e-learning in corporate,
government, healthcare and higher education (e-learn 2007). T. Bastiens and S. Carliner. Quebec,
Canada, Chesapeacke, VA:AACE: 1634-1641.
Lahtinen, E., K. Ala-Mukta, et al. (2005). A Study of the Difficulties of Novice Programmers. ITiCSE'05 10th
Annual SIGCSE Conference on innovation and Technology in Computer Science Education.
Caparica, Portugal, ACM Press: 14-18.
Lazonder, A. W. and H. Van Der Meij (1995). "Error-information in tutorial documentation: supporting users’
errors to facilitate initial skill learning." International Journal of Computer Studies 42: 185-206.
Lister, R., E. S. Adams, et al. (2004). A Multi-national study of reading and tracing skills in novice
programmers. ITiCSE WGR04: Working group reports from ITiCSE on Innovation and technology in
Computer Science Education.
Lund, G. (2002). Aspects of the Program Development Process used by Learner Programmers School of
Computing. Dundee, Scotland, University of Abertay. Ph.D.
Malan, K. and K. Halland (2004). Examples that do Harm in Learning Programming. OOPLSLA’04.
Vancouver: 83-87.
Mayer, R. E. (1981). "The Psychology of How Novices Learn Computer Programming." ACM Computing
Surveys (CSUR) 13(1): 121-141.
Mayer, R. E. (1997). From Novice to Expert. Handbook of Human Computer Interaction 2nd.Edition. M.
Helander, T. K. Landauer and P. Prabhu, Elsevier-Science B.V.
McGill, T. J. and S. E. Volet (1997). "A Conceptual Framework for Analysing Students' Knowledge of
Programming." Journal of Research on Computers in Education 29(3): 276-297.
Mead, J., S. Gray, et al. (2006). A Cognitive Approach to Identifying Measureable Milestones for Programming
Skill Acquisition. ITiCSE 2006. Bologna: 182-193.
Murphy, L., G. Lewandowski, et al. (2008). Debugging: the good, the bad, and the quirky -- a qualitative
analysis of novices' strategies. 29th SIGCSE Technical Symposii, on Computer Science. Protland,
Oregon: 163-167.
Norris, C., F. Barry, et al. (2008). ClockIT: Collecting quantitative data on how beginning software developers
really work. 13th Conference on Innovation and Technology in Computer Science Education. Madrid,
Spain: 37-41.
Pillay, N. (2003). "Developing Intelligent Programming Tutors for Novice Programmers." Inroads (SIGCSE
Bulletin) 35(2): 78-82.
Rountree, J., N. Rountree, et al. (2005). Observations of Student Competency in a CS1 Course. Australasian
Computing Education Conference 2005, Conferences in Research and Practice in Information
Technology. 42.
Shaffer, D., W. Doube, et al. (2003). Applying Cognitive Load Theory to Computer Science Education.
Workshop of the Psychology of Programming Interest Group.
Simon, S. Fincher, et al. (2006). Predictors of Success in a First Programming Course. Eighth Australian
Computing Education Conference (ACE2006).
Soloway, E. and K. Ehrlich (1984). "Empirical Studies of Programming Knowledge." IEEE Trans. on Soft. Eng
SE-10(5): 595-609.
Winslow, L. E. (1996). "Programming Pedagogy – a Psychological Overview." SIGCSE Bulletin 28: 17-22.

PYTHON FOR TEACHING INTRODUCTORY PROGRAMMING: A
QUANTITATIVE EVALUATION
Ambikesh Jayal Stasha Lauria Allan Tucker

Department of Computing, SCISM, Stephen Swift
University of Gloucestershire, UK Brunel University, UK SCISM,
ajayal@glos.ac.uk Stasha.Lauria@brunel.ac.uk Brunel University, UK
Allan.Tucker@brunel.ac.uk
Stephen.Swift@brunel.ac.uk
ABSTRACT
This paper compares two different approaches of teaching introductory programming by quantitatively
analysing the student assessments in a real classroom. The first approach is to emphasise the principles of
object-oriented programming and design using Java from the very beginning. The second approach is to first
teach the basic programming concepts (loops, branch, and use of libraries) using Python and then move on to
oriented programming using Java. Each approach was adopted for one academic year (2008-09 and 2009-
10) with first year undergraduate students. Quantitative analysis of the student assessments from the first
semester of each year was then carried out. The results of this analysis are presented in this paper. These
results suggest that the later approach leads to enhanced learning of introductory programming concepts by
students.
Keywords
Teaching programming, object oriented approach, procedural approach, java, python.
1. INTRODUCTION
Teaching programming is a complex task. The task is even more challenging for introductory modules. Failure
rates are not marginal (Bennedsen & Caspersen, 2007). One of the common problems shared by Computer
Science Departments is the lack of basic programming skills reported by module leaders of courses following
first programming courses and how to equip students with better programming skills after the introductory
courses. First programming courses typically emphasise the principles of object-oriented programming and
design from the very beginning. An alternative approach is based on starting with a more traditional
procedural approach first. The evidence from past research seems to suggest that Object Oriented
Programming is generally more complex than procedural (Robins et al., 2003). McCane (2009) has
demonstrated the effectiveness of Python to teach introductory programming by using qualitative analysis.
In this paper it is therefore suggested to introduce Python (Python Software Foundation, 2010) for the basic
(procedural) aspects of programming and then introduce Java (Java, 2010) to focus on object-oriented
aspects. The use of Python is expected to both reduce the overhead attached to Java syntax and allow
immediate feedback while the students practice with basic instructions (due to the interpreted nature of
Python). In other words, at the beginning of the module, novice student are able to devote particular attention
to procedural concepts, flow of control, flow of data, etc.
The aim of this paper is to evaluate the success of using Python to introduce basic concepts (loops, branch,
and use of libraries). Such evaluation is based on the analysis of student assessments. To measure the
success of the proposed method, quantitative analysis of the assessments outcome are presented and
discussed. Ehlert (2009) highlights the importance to develop an experimental setting for a fair comparison
and the importance to control the variables carefully. The experimental setting used in this paper controls and
defines the variables carefully and measures them using clear indicators.

2. LITERATURE REVIEW
Various approaches to teach programming have been summarised in (Lemos, 1979) and (Engel et al., 2001).
But there is an ongoing debate in the teaching community over the best approach to teach introductory
programming (Lister et al., 2006; Pears et al., 2007). We have found four experimental studies that compare
object oriented approach with the traditional procedural approach. The first study by Decker (2003) has found
no difference in student performance between the two approaches. The second study by Reges (2006) has
found significant gains in student satisfaction and enrolment after replacing the object oriented programming
first curriculum with a procedural approach. The third study by Vilner (2007) has found no significant gains in
student performance between the two approaches. The parameters used in this research to compare the two
teaching approaches are the pass rate of students and grades in questions related to recursion, efficiency of
algorithms and designing of classes. The fourth study by Ehlert (2009) has found no significant gains by
changing the object-oriented programming first approach with object oriented programming later approach.
According to Ehlert (2009), different pegagogical dimensions complicate the analysis of different approaches
to teach programming. Bruce (2005) argues that there is a need for more experimental studies to examine the
different approaches to teach programming.
3. METHODOLOGY
Within this paper Python assessment submissions are compared with Java ones. The Python and Java
assessments are very similar and therefore comparisons should give a reliable measure. The aim of the
comparison is to quantitatively measure a student’s ability to master basic programming concepts when
Python is used instead of object-oriented Java. Both assessments consisted of students having to implement
a program (See below Section 4 for further details).
The following three comparisons have been carried out: grades, programs with bugs and frequency of
keywords. The grades of the submitted program are a measure of success of the student ability to implement
programs. Bugs are interpreted as a measure of their overall understanding of programming. Frequency
values of keywords are interpreted as a measure of the familiarity with basic concepts of programming. The
four keywords used in the frequency measure are “if”, “for”, “while” and “import” (in combination with the
use of “random” class). They are indicators of student’s use of conditional, loops and import (the random
library) statements. Occurrences of these keywords in commented sections of the code have been discarded.
No distinction has been made between correct and incorrect use of the statements in the frequency
measurement.
Two different academic years have been used to collect data. During the 2008-09 academic year, the
approach to emphasise the principles of object-oriented programming and design using Java from the very
beginning is followed. During the 2009-10 academic year, the approach to first teach the basic programming
concepts (loops, branch, and use of libraries) using Python and then move on to oriented programming using
Java is followed. In both cases, students were assessed after approximately ten weeks of teaching. Java
assessments are from 2008-09 academic year, whereas Python assessments are from the 2009-10 academic
year. Also all students in a cohort have (some) similar background since they have to fulfil similar
requirements to be accepted. Some may have a better proficiency than other but this is true in equal measure
for both cohorts. The research is based on the assumption that students cohort for the academic year 2008-
09 and 2009-10 are of similar academic level. This assumption is supported by the fact that the admission
criteria for both the years were the same. No student was in both cohorts. Table 1 summarises the student’s
cohort.
Academic Year Language Assessment point Number of students

2008-09 Java After ten weeks of teaching 157
2009-10 Python After ten weeks of teaching 185
Table 1: Summary of student’s cohort

4. MODULE OUTLINE
Both 2008-09 and 2009-10 modules during the first ten weeks emphasize mastery of basic skills. In particular,
lectures and lab tutorials/exercises have focused on loops, conditionals, arrays/lists and use of
libraries/packages (in particular the ones to generate random numbers). BlueJ (BlueJ, 2010) was used in the
2008-09 module as a tool to practice with Java, whereas in 2009-10 the standard Python tools have been,
used (i.e. both the graphical user interface and the command line applications). Approximately, the same
amount of time has been dedicated to each topic during both years.
Students are assessed for their ability to implement programs. Criteria for the assessments include:
generalizability, complexity, coverage of loops, conditionals, etc. Moreover, the ability to produce a program
that could compile and run without errors has been considered a requirement for a pass. However, for both
Java and Python data, small typos (a missing semicolon, etc.) have not been considered bugs (even if the
submitted program did not run correctly). That is, they have not been graded as a fail grade, although they
have been graded as just above a pass. To assess the style of the submitted program the source code was
inspected.
Students have an hour to complete the task and they can use the same environment as the one they used
during their lab tutorials (BlueJ, Python, etc.). That is, during the test they can implement, run and debug their
programs. They have access to both the standard Java/Python documentation and to the module resources
(lecture notes, lab examples, etc.). Tasks are kept as similar to one another as possible in both assessments.
5. RESULTS AND DISCUSSION

Tables 2 and 3 summarise the results of the quantitative analysis.
Columns 3 to 6 in Table 2 show the number of occurrences of programs where respectively at least once a
conditional, a for loop, a while loop and the import of the random package has occurred. The second
column (Total Students) indicates the total number of assessments and the last column indicates the
occurrences of a program that could not run without performing some debugging first.
Total Students If % for % While % random % bugs %

Python 185 67 36 81 44 27 15 97 52 51 28
Java 157 36 23 1 1 8 5 42 27 90 57
Table 2: Key Indicators. Occurrences and percentages for each key indicator. The Total column
indicates the total number of assessments.
Total Students A % B % C % D % E % D(40) %

Python 185 6 3 15 8 22 12 121 65 21 11 36 19
Java 157 6 3 10 6 22 14 42 27 78 50 19 12
Table 3: Grades. Occurrences and percentages for each key indicator. The Total column indicates the
total number of assessments.
Table 3 shows the grade distribution. The use of loops, conditional, etc. to achieve the given task was part of
the marking criteria. For example, a grade A requires the correct use of conditional and the correct use of
loops and the correct use of libraries in the program implemented. The last column (D(40)) shows the number
of occurrences of programs that did not run without debugging but where bugs have been nevertheless
considered of minor entity(and therefore graded as just a pass).
The results presented in Table 2 show that there is evidence of a significantly increased use of all the four
markers when Python is used as programming language. Moreover, the occurrence of bugs is lower with
Python than with Java. The results presented in Table 3 show that grades distribution shows a higher
percentage of programs able to meet the minimum requirements for a pass when Python has been used.

In order to statistically validate our results, Two sample t-test was conducted on the four markers (for “if”,
“for”, “while” and “random”) shown in Table 2. R (The R Foundation, 2010) was used to carry out this
statistical test. The confidence interval was taken as 0.95. The p-value when variance was assumed to be
equal was 0.043 and when the variance was assumed to be unequal was 0.048. As the p-value is less than
0.05, the difference between the samples is statistically significant. So the number of key indicators used by
students statistically differs when Python was introduced as compared to the Java only approach. This implies
that the Python first approach had a positive effect on the grades.
These results imply that our approach to introduce Python for the basic (procedural) aspects of programming
and then switch to Java to focus on object-oriented aspects enhanced students understanding of the
introductory programming.
6. CONCLUSIONS
This paper presents experimental results from real classroom comparing two different approaches to teach
introductory programming. The first approach is to emphasise the principles of object-oriented programming
and design using Java from the very beginning. The second approach is to first teach the basic programming
concepts (loops, branch, and use of libraries) using Python and then move on to oriented programming using
Java.
The first teaching approach using Java has been followed for the academic year 2008-09 and the second
teaching approach using Python has been followed for the academic year 2009-10. The student assessments
towards the end of first semester in each academic year have been used to carry out the experiments
described in this paper. Three indicators namely grades, programs with bugs and frequency of keywords have
been used for comparison. The grades of the submitted program are a measure of success of the student
ability to implement programs. Bugs are interpreted as a measure of their overall understanding of
programming. Frequency values of keywords are interpreted as a measure of the familiarity with basic
concepts of programming. The four keywords used in the frequency measure are “if”, “for”, “while” and
“import” (in combination with the use of “random” class).
The experimental results discussed in this paper show a measured positive (increase) in all the three
indicators in favour of the second approach which is to first teach the basic programming concepts (loops,
branch, and use of libraries) using Python and then move on to oriented programming using Java. The use of
Python could facilitate the mastering of basic concepts such as loops, branch, and use of libraries for novice
students. A possible explanation could be that with Python students can focus on the crucial basic issues
without being distracted by the overheads. Moreover, the complexity of the programs used during their
practice could be tailored in a more accurate way to their level of proficiency.
One limitation of this research is that it considers only one case study. Such case studies require considerable
amount of time as they have to fit in with the academic calendar and hence we could only consider one case
study. We hope that similar case studies will be replicated by other institutions as well. The results from our
case study are positive and statistically significant in favour of our proposed approach which is to first teach
the basic programming concepts (loops, branch, and use of libraries) using Python and then move on to
oriented programming using Java. We hope that our research will encourage debate over different
approaches to teaching introductory programming and in turn will lead to adoption of better teaching
methodologies.
Also this research focuses only on the quantitative analysis of the student assessments and does not elicit the
views of the students about their learning experience. So an interesting direction for future work would be to
collect and analyse the qualitative data from the students about their perspective.
This research takes into account the performance of students up until the first semester. We are analysing the
performance of students over the whole academic year and will be publishing it at a later stage.

7. REFERENCES
Bennedsen, J., Caspersen, M.E. (2007), Failure rates in introductory programming. ACM SIGCSE Bulletin,
39(2), pp. 32–36.
BlueJ, (2010), BlueJ - Teaching Java - Learning Java, http://www.bluej.org/ (01 Dec 2010)
Bruce, K.B. (2005), Controversy on how to teach CS 1: a discussion on the SIGCSE-members mailing list,
ACM SIGCSE Bulletin, 37(2), pp. 111-117
Decker, A. (2003), A tale of two paradigms, Journal of Computing Sciences in Colleges, 19(2), pp. 238-246
Engel, G., Roberts, E. (2001), Computing Curricula 2001 Computer Science: final report, IEEE Computer
Society and Association for Computing Machinery, 28, pp. 2004
Ehlert, A., Schulte, C. (2009), Empirical comparison of objects-first and objects-later, ACM Fifth international
workshop on Computing education research workshop, pp. 15-26
Java, (2010), Oracle Java Technologies, http://www.oracle.com/us/technologies/java/index.html (01 Dec

2010)
Lemos, R.S. (1979), Teaching programming languages: A survey of approaches, ACM SIGCSE Bulletin,
11(1), pp. 174-181
Lister, R., Berglund, A., Clear, T., Bergin, J., Garvin- Doxas, K., Hanks, B., Hitchner, L., Luxton-Reilly,
A.,Sanders, K., Schulte, C., and Whalley, J. L. (2006), Research perspectives on the objects-early debate.
ACM SIGCSE Bulletin, 38(4), pp. 146-165
McCane, B. (2009), Introductory programming with python, The Python Papers Monograph, 1, pp. 1-18.
Pears, A., Seidman, S., Malmi, L., Mannila, L., Adams, E.,Bennedsen, J., Devlin, M., and Paterson, J. (2007),
A survey of literature on the teaching of introductory programming, ACM SIGCSE Bulletin, 39(4), pp. 204-223
Python Software Foundation, (2010), Python Programming Language, http://www.python.org/ (01 Dec 2010)
Robins, A., Rountree, J., Rountree, N. (2003), Learning and Teaching Programming: A Review and
Discussion. Computer Science Education, 13(2), pp. 137-172
Reges, S. (2006), Back to basics in CS 1 and CS 2, ACM SIGCSE Bulletin, 38(1), pp. 293-297
The R Foundation, (2010), The R Project for Statistical Computing, http://www.r-project.org/ (01 Dec 2010)
Vilner, T., Zur, E., Gal-Ezer, J. (2007), Fundamental concepts of CS1: procedural vs. object oriented
paradigm-a case study, ACM SIGCSE Bulletin, 39(3), pp. 171-175

STUDYING FIRST YEAR FORENSIC COMPUTING: MANAGING THE
STUDENT EXPERIENCE
David W Salt Harjinder Singh Lallie Philip Lawson
University of Derby University of Derby University of Derby
School of Computing School of Computing School of Computing
Derby Derby Derby
d.w.salt@derby.ac.uk h.s.lallie@derby.ac.uk p.lawson@derby.ac.uk
ABSTRACT
The experiences of students studying Computer Forensics at the University of Derby, together with the views
of the teaching and technical support staff, strongly suggests a growing mismatch between expectations and
reality, an underdeveloped ability to problem solve and communicate findings and unrealistic expectations of
the job market.
In improving the conceptualisation of how our courses can better equip students for employment and how
their understanding of some of the intellectual and advocacy aspects of investigating cases and presenting
evidence can be improved we make suggestions on what works and what needs to be improved. Emphasis is
put upon investigative skills rather than investigative tools (the reverse of student expectations), marshalling
and presentation of facts and the growing importance of triage approaches.
We conclude with asset of recommendations on how digital forensics courses can be improved in terms of
delivery, managing student expectations and aspirations, and increased engagement with industry and
commerce.
Keywords
digital forensics, information security, expectations, employment
1. INTRODUCTION
The University of Derby has been running Digital Forensics and Information Security courses at both
undergraduate and postgraduate levels for more than 4 years. Digital Forensics has become increasingly
popular as a destination course for many computing students. Perhaps as a result thereof, the number of UK
institutions offering this as a bachelor’s degree subject has increased dramatically, at the time of writing there
were 75 undergraduate courses listed on UCAS.
An important underlying assumption is made within this study that digital forensics is a multi-disciplinary field
which encompasses areas of computing, law and science. This view is consistent with that presented by
Kessler and Schirling (2006) but somewhat contradictory to that presented by Gottschalk, Liu, Dathan,
Fitzgerald and Stein (2005) who suggest that it is in fact a subfield of forensic science. A fuller exploration of
these views is outside of the remit of this article but is nevertheless an important area for further research as
the set of key professional and technical skills involved in either area of study can differ quite remarkably.
The course at Derby includes three modules related to criminology and IT law (delivered by colleagues from
the School of Law), two modules based around forensic science (delivered by academics from the School of
Sciences) and has a significant emphasis on Information Security. It is not uncommon for digital forensic
courses to offer modules related to Information Security, this means that students undertaking such courses
are well placed to pursue opportunities in the Information Security Industry. A combination of the subject
areas described here is likely to lead to the emergence of a number of pedagogic issues some of which have
previously been identified by Yasinsac et al (2003).
The problem of students accepting and recognising the justification for certain elements of the curriculum to
exist within their chosen degree is one that has been faced by many institutions and courses. For instance,
students on the undergraduate Computer Networks course at Derby have in the past questioned the
existence of even the most introductory programming modules. This does not seem to have been a problem
for the digital forensics course in Derby. Students have not generally questioned the existence of law and
even forensic science within the digital forensics curriculum. The induction process (the point at which
curricular justification is presented) has in this respect served the course well.

The curriculum at Derby broadly follows the curriculums outlined by Gottschalk et al., (2005) and Yasinsac et
al (2003) the latter of whom outlines four employment/further study pathways for students on such a program:
digital forensics technician, policy maker, professional and researcher.
Student expectations on such courses can be diverse, there is however little or nothing written about
managing these expectations. The observations contained herein have been informed through the study of
two cohorts of year one undergraduate students. The first cohort consisted of 23 students, the second cohort
of 42 students. The study seeks to explore and investigate what are deemed to be ‘isolated’ views and
perceptions amongst the student cohorts and there is no suggestion that the perceptions explored herein are
representative of the entire cohorts. In particular we focus on:
• the perceptions and skill levels of students who come onto the course
• the manner in which we deconstruct and manage those perceptions and skills
• the complexities of teaching a diverse range of skills.
2. OBSERVED PROBLEMS
Some of the problems relating to student expectations may be linked to what the term computer forensics
conveys and means to students. Other relevant terms to describe the sub-discipline are digital forensics (the
term preferred by the authors), network forensics and forensic computing. Some of these terms have been
explored by Yasinsac et al (2003) and are not the subject of discussion here.
2.1 Unrealistic Career plans and aspirations.

Increased storage capacities and the continued threat of terror attacks are likely to motivate increased
employment opportunities in the area of Digital Forensics. The allocation of an additional 650 million pounds
to Cyber-Security initiatives announced in the Strategic Defence Review may also stimulate the jobs market
(The Stationery Office, 2010).There are however a number of factors which make cohort-wide career
aspirations in the digital forensics industry unrealistic:
• Increased University graduate competition. An increasing number of Universities are offering courses in this
area and there is likely to be a corresponding increase in competition for employment in this sector. There is
probably more opportunity in the Information Security sector. However, students generally do not recognise this.
• Career progression and pay. Graduates entering the digital forensics market are likely to be paid less than those
entering the Information Security sector. A student who completed his master’s degree in Forensic Computing and
Security in 2010 recently reported that the only suitable job he was able to find was as a “forensic technician” at a
starting salary of £14,000 per year. He is currently widening his job search beyond the UK. It is not clear that this is
a typical experience partly because we are yet to have students graduate from our undergraduate course. Our
postgraduate students have reported that the job market is difficult to negotiate. It is however clear that the job
market currently shows no signs of being able to absorb forensics specialists in the numbers graduating from UK
degree courses. This situation is likely to have a massive effect on the employability of our students and should
impact upon the way in which we design and market our courses. Some further discussion of the jobs market can be
found in section four.
• Unrealistic Understanding of Job Roles. Whilst courses such as the one offered at Derby interface well with
employers and endeavour to provide students with a realistic understanding of the digital forensics industry,
students do nevertheless have an unrealistic understanding of what is actually involved in working in the sector. For
instance, students have often not thought through realistically in terms of the potential emotional impact of the
material that they will be handling on a daily basis.
Students may well have developed an interest in digital forensics based on portrayals in the media (e.g.
C.S.I.). Not only is the reality of the job less glamorous but students are often surprised at how painstaking
and time consuming real investigative processes are. Searches which appear to yield results instantaneously
in media portrayals can take hours or days in real cases. This isn’t helped by experiences on digital forensics
courses wherein the digital forensic images that students have to investigate are often deliberately small so as
to ensure that tutorial outcomes are achievable, if unaccompanied by appropriate disclaimers, this can
contribute to enforcing unrealistic views.
Students coming onto the course generally expect and aspire to careers within a digital forensics technician
role. This manifested itself more clearly at Derby when the majority of the first year two cohort seemed

generally to restrict their placement applications to opportunities in the digital forensics industry without
seeking opportunities more widely in information security or computing in general.
One way to offset this view is to adopt a career centric approach to marketing and presenting the course such
that students are introduced to career pathways and post-study professional opportunities at the earliest given
opportunity. In addition to this, it may be useful to introduce students to an ontological model such as that
proposed by Brinson, Robinson and Rodgers (2006) wherein the study of digital forensics has been classified
into a hierarchical model which describes curricular areas within legal, academic, military and private-sector
professions. However such a model is intentionally digital forensics specific and does not incorporate related
disciplines which the course themes support, it could therefore be modified to reflect specific course themes.
2.2 Fuzzy approaches to problem solving.

Digital forensics students are required to be open-minded about potential answers, they need to understand
that there are sometimes inconclusive answers to problems. Digital forensic practitioners are not always
seeking the end solution (i.e. to prove innocence or guilt), they are often seeking evidence which when
brought together and presented logically by a legal practitioner may present a propositional conclusion. This is
evidenced by the reports submitted by first year students when they often use phrases such as “this proves
the accused was guilty” and “I have no doubt he did it.” In the first place their primary purpose is to investigate
and not to offer unsolicited opinions on guilt or innocence. In the second place as Smith and Brace (2003)
point out the adversarial and ritualistic aspects of the courts do not necessarily respect the findings of
software engineers.
The potential size of digital images and the time taken to search them is a well-known bottleneck in digital
forensic investigations. There has been a move towards using triage systems (Parsonage, 2009) to decide
what should be investigated, in what order and priority and what not to investigate. Students will not, at first,
be familiar with triage and they can be overwhelmed by the sheer volume of data that can be searched.
Students need to cultivate a particular mind-set in order to navigate through all the data. They have an
observed tendency to try and search everything when new to the discipline and they have difficulty in devising
strategies to narrow down the search space based upon both theories of how criminals may try to cover their
tracks and efficiency of search algorithms.
The Digital Forensics sub-discipline therefore requires a particular ‘fuzzy’ approach to problem solving
wherein the solution to the problem (as far as the digital forensic practitioner is concerned) falls within certain
boundaries (guilt or innocence) but is not a clear one i.e. a propositional solution. The approach can very
rarely, if ever, be to search everything and therefore a methodological approach to building a triage strategy is
highly challenging. Suggestions of how to do this and how to weight different options are still under discussion
(Rogers et al, 2006; Parsonage, 2009) in law enforcement agencies so it is hardly surprising that students
struggle to find a suitable framework in which to operate.
2.3 Intricate hardware and superior soft skill requirements

Preconceptions such as those described earlier may result in an unrealistic view of the technical nature of the
subject area. There are specific architectural issues that digital forensics students need to become more
familiar with in comparison with students on other computing (particularly information systems related)
courses. Digital forensics students need to develop quite possibly the most intimate level of understanding in
terms of hardware and architectural concepts. They need to develop, for example, an intricate understanding
of hard disk and file system structures as well as the operating systems and other software related concepts.
At Derby practical laboratory sessions are actively supported by computing technicians as well as lecturers.
Technicians, who often find themselves supporting students outside of scheduled classroom hours, have
commented that some students do not even have the basic understanding of operating systems or what are
the differences between the Mac, Linux and Windows operating systems. They also commented on the need
for students to understand the heuristic nature of an investigation which often requires them to think about
quite simple questions such as what software is available to hackers and criminals, how is it used and where
is it commonly to be found on computer systems. Too often students were seen to be conducting searches
with software tools such as Encase before they considered these fundamental questions which would deliver
answers and search strategies much more efficiently than unstructured searches.
Even when students showed improvements in formulating investigative strategies they quite often missed
some of the fundamental things. Some completely misunderstood the importance of time zones. For instance,
the fact that New York was 5 hours behind London was missed leading to a misinterpretation of evidence and
leaving it open to challenge in court.
This intricate understanding further broadens destination career pathways and students with such a skill set
may indeed progress to hardware design or systems administration related roles.
Conversely students must also possess excellent interpersonal skills – in particular the ability to write and
present their work. The experience at Derby has shown two particular professional skills related areas
wherein some students can struggle:
• Students have had particular problems with developing the ability to debate and present strong conclusive legal
arguments. Some seem fearful of the need to stand up as a witness and defend their findings.
• Some students find the necessity to develop unambiguous and clear reports which defend their arguments, quite
challenging.
3. THE DERBY EXPERIENCE
We have attempted to deal with some of the issues described above by: greater industrial involvement,
assessments designed to focus on industrial issues and a deliberate and focussed deconstruction of
preconceptions. It is probably too early to present conclusive arguments and responses to some of the
problems described herein, in fact there may not be conclusive approaches to resolving these. The responses
can therefore be best described as ‘work in progress’.
3.1 Deconstruction of industry views and biases
For any academic discipline that has been glamorised somewhat by media portrayal, it is important that a
process of deconstruction takes place early in the delivery of the course.
Following the experience of the first cohort, the course team undertook to make a deliberate attempt to
deconstruct these views. This was done primarily through industrial engagement and industry focussed
assessment, however the team also considered it important to address these preconceptions through the
induction process at the outset of the cohort experience. There is some focus during the induction process
aimed at not only developing a realistic understanding of the industry but also alerting students to
opportunities available in the areas of Information Security and also in computing generally.
3.2 Industrial Engagement
At least one guest lecture is organised to be delivered by a director or practitioner from within the forensics
industry, this is an approach that other academic practitioners within this discipline have also adopted (Troell,
Pan and Stackpole 2004). The focus of the talk is to emphasise the nature of work that a forensic technician
may be involved in. The talks thus far have emphasised that:
• There is a growing demand for digital forensic practitioners, however there is also a disproportionate increase in
courses focussing on digital forensics.
• Digital forensic practitioners may be required to conduct investigations involving contraband material which
includes indecent images of children, bestiality and violence. Whilst larger organisations may be better prepared to
managing resources so that certain staff are not exposed to this, smaller organisations have less flexibility in this
respect. There is also a duty of care on the part of the employers to provide appropriate counselling and access to
psychological services for staff exposed to extreme materials. Arguably University courses in digital forensics
should also incorporate some techniques on maintaining mental well-being when dealing with such evidence.
• The digital forensics industry is new, it is undergoing growth and developing in its maturity. Career structures and
progression pathways are not as clearly defined as they may be in other industries.
3.3 Industry focussed professional skills assessment
A common problem in computing related courses is that of teaching communication and professional skills
(CPS). This issue has been addressed at Derby through the tighter integration of CPS within the curriculum.
For instance digital forensics is used as the vehicle for all CPS.
The CPS assessment in this case involves the defence of a digital forensics related business proposal under
heavy and strict scrutiny by course team members and industrial invitees. The presentation portion of this
assessment places the students under heavy pressure.
Amongst other tasks, the students must:

• Cost the proposal in precise terms
• Analyse competition
• Clearly define remit and scope of the proposal
This exercise gives students a more realistic view of what is expected within the forensics industry.
Students are also required to present evidence gathered in the course of a simulated digital forensic
investigation in a mock court (we have a full size courtroom at Derby University) and to be questioned by
members of the law teaching staff to defend and explain their findings. This gives students first hand
experience of what it means to be an expert witness.
4. THE JOBS MARKET
As noted earlier there is are questions about the level and type of job that will be available to our graduates.
One of the problems we face is tracking and quantifying the kinds of jobs that might be suitable. Very few
resources exist which quantify the opportunities. An examination of jobs advertised on ITJobsWatch (2010)
shows 1100 Information Security jobs were advertised in the three months to 11thh November 2010 with an
average salary of £52,500. Jobs in this sector ranked 230 compared to 198 in the same period in 2009 and
256 in 2008. By comparison average wages for Computer Forensics specialists were lower and there were
only 31 posts advertised in the three month period to November 11th. 30th. Job ranking within IT was 784
compared to 625 in the same period last year. The number of jobs for Forensic Analysts in IT in the same
period was only 3. Further interrogation of the statistics available on this one site failed to yield any other
obvious digital forensic employment. It is unlikely that our graduates are starting on the salaries quoted on
this site yet it is unclear where else they are being employed. Irrespective of the question of how much lower
entry level salaries must be, it is abundantly clear that advertised jobs in digital forensics are few whereas the
demand for information security professionals is much higher.
A search for “digital forensics”, “digital analyst” and related terms turns up a number of individual
advertisements but they are limited in number compared to other IT sector opportunities. An example from
www.jobsite.co.uk yields an advert for a “computer forensic analyst” with a salary range of £25-£30K, the
skills requirements include case management and forensic analysis of computer-based equipment and mobile
phones, securing and preserving digital evidence, EnCase, familiarity with ACPO guidelines and
“unimpeachable integrity.” However, this is a standalone job with no evidence of multiple job opportunities.
Similarly, adverts can be found for “mobile phone examiner” and “cell site analyst”, for example, but they are
few and far between.
Much more work needs to be done to understand the demands of the marketplace and we are currently
undertaking a survey of companies working in the forensics area to determine how they recruited existing staff
and what their likely demands are for the future. There is an uncomfortable sense that we may be producing
too many graduates with the wrong mix of skills to match employment demands.
5. CONCLUSIONS
Universities develop and promote courses based on student demand and the proliferation of digital forensics
courses and application figures bear testament to the fact that demand is still high. In contrast demand for
employers for digital forensic practitioners is demonstrably low at present although the current political climate
may indicate demand is likely to rise. The authors believe that we have an ethical responsibility to ensure that
student expectations are as closely aligned with the reality of the job market as we can achieve. Current
employment trends indicate therefore that our courses should be equipping students with broad skills as
information security professionals rather than just as digital forensics experts.
Assuming our initial assumptions about the job market are correct and that our observations of student
capabilities are representative we make the following recommendations:
• Digital Forensics courses should ensure their curricula include information security skills and course titles
should reflect this
• The recruitment and induction processes are vital in managing student expectations
• Career planning should start in year one

• Generic problem solving skills need to be integrated into courses
• Communication skills should be central to digital forensics with students receiving adequate coaching and
practice as expert witnesses
• Links with industry and commerce are even more important in this field than other aspects of computing.
Current practitioners should be invited to speak to students about both technical subjects and career planning
• The ability to classify and triage search problems and presentation strategies are important skills to cover
• Strategies for ensuring psychological wellbeing of forensic practitioners should not be optional
Universities need to appraise their activities honestly and within an ethical framework. In introducing digital
forensics courses we need to rapidly assess the needs of our students and to ensure that our practices and
their expectations align with the reality of the employment market. At present there is still a gap which needs
to be addressed.
6. REFERENCES
Brinson A., Robinson A.,Rodgers M., (2006), “A cyber forensics ontology: Creating a new approach to
studying cyber forensics”, Digital Investigation, Vol 3s, S37-S43
Carlin A., Curl S., and Manson D., (2005). “To catch a thief: Computer forensics in the classroom”. In
Proceedings of the 22nd Annual Information Systems Educators Conference (Columbus, OH, Oct.),
Association of Information Technology Professionals, Chicago, IL.
Gottschalk L., Jigang, L., Dathan B., Fitzgerald S., and Stein M., (2005), “Computer Forensics Programs in
Higher Education: A Preliminary Study”, SIGCSE'05, February 23-27, 2005
HM Government, Securing Britain in an Age of Uncertainty: The Strategic Defence and Security Review,
2010, The Stationery Office.
ITJobsWatch. 2010. Information Security Jobs. [Online] (Updated 30th June 2010) Available at:
http://www.itjobswatch.co.uk/jobs/uk/information%20security.do [Accessed 11th November 2010]
Kessler G.C., and Schirling M.E. (2006). “The Design of an Undergraduate Degree Program in Computer &
Digital Forensics”. Journal of Digital Forensics, Security and Law, 1(3), 37-50.
McGuire T., Murff K., (2006), “Issues in the Development of a Digital Forensics Curriculum”, Consortium for
Computing Sciences in Colleges
Parsonage, H. 2009 Computer Forensics Case Assessment and Triage. [Online] (Updated November 2009)
Available at
http://computerforensics.parsonage.co.uk/triage/ComputerForensicsCaseAssessmentAndTriageDiscussionPa
per.pdf [Accessed 11th November 2010]
Rogers M., et al(2006) “Computer Forensics Field Triage Process Model.” Proceedings of Conference on
Digital Forensics, Security and Law pp27 -40
Smith F., Bace R., 2003. A guide to Forensic Testimony, Boston Addison Wesley
Troell L., Pan Y., Stackpole B., (2004), “Forensic Course Development – One Year Later”, SIGITE’04,
October 28–30, 2004
Yasinsac A, Erbacher R.F. et.al, “Computer Forensics Education”, IEEE Security and Privacy, vol 1, no. 4, pp.
15-23, July 2003

Vol 10 Iss 1 Feb 2011

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 10 Iss 1 Feb 2011

Uploaded by

Copyright:

Available Formats

ITALICS Volume 10 (1) February 2011

Editorial – Stephen Hagan, University of Ulster

Ontology-Based E-Learning Personalisation for Disabled Students in

Virtual Learning Environments for Mobile Learning

Generalised Diagramming Tools with Automatic Marking

Teaching Media and Information Literacy to Postgraduate Researchers

Bob’s Project Guidelines: Writing a Dissertation for a BSc in Computer Science

Building a Software Development Environment to Enrich Learning and Enhance

A Real-World Mobile Interaction Design Task

Embedding Inquiry based learning into Programming via Paired Assessment

Emergent Requirements for Supporting Introductory Programming

Python for Teaching Introductory Programming: A Quantitative Evaluation

Studying First Year Forensic Computing: Managing the Student Experience

EDITORIAL BY STEPHEN HAGAN, University of Ulster & ANNE MORRIS, Loughborough

Julius T. Nganji Mike Brayshaw Brian Tompsett

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 1

2. THE NEED FOR DISABILITY INCLUSION IN HIGHER EDUCATION

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 2

3. THE SEMANTIC WEB

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 3

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 4

Figure 2: Class Disability of ADOOLES ontology

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 5

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 6

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 7

4.2 System Functionality

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 8

5. CONCLUSION AND FUTURE WORK

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 9

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 11

Laura Crane Phillip Benachour Paul Coulton

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 12

2. FLEXIBLE LEARNING AND MOBILE LEARNING

Figure 1 – Characteristics of Mobile & Flexible Learning

3. VIRTUAL LEARNING ENVIRONMENTS

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 13

Fig 2. MyPlace: Educational Social Web Based Software

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 14

Fig. 3. Chloropleth demonstrating nomadic access and popular campus routes

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 15

Campus Mean of Coverage Quality

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 16

Fig. 4. Student’s responses towards VLE & online social networks

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 17

6. DESCRIPTION OF THE STUDY

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 18

Fig.7: SWOT analysis constructed by student focus group

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 19

Communication Market Report, OfCom. August 2008

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 20

Small, G., & Vorgan, G. (2008). iBrain. New York: HarperCollins.

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 21

Dr Pete Thomas Dr Kevin Waugh Dr Neil Smith

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 22

2. GENERALISATION OF DIAGRAM TYPES

Copy Book Guest

Figure 1 Different types of diagram

Figure 2 An ERD showing spatial relationships

Drawing Tool Diagram Mark Scheme

Figure 3 Drawing tool and marking engine

ITALICS Volume 10 Issue 1 February 2011 - ISSN: 1473-7507 24

Figure 4 The ERD Revision Tool