Submitted for the Degree of

BSc(Hons) in Multimedia Technology

Project Title: A Multimedia Learning Tool to Assist in the Learning of British Sign Language

Author: Ewan McIntyre Supervisor: Iain Stewart

Except where explicitly stated all work in this report, including the appendices, is my own. Signed: Date


British Sign Language is a growing language, which was recently recognised as an official minority language by the British Government. There is currently, however, a shortage of teachers and of sophisticated and accessible tools which will either assist in the teaching of the language, or act as a teaching resource to interested parties. This project aimed to research the tools currently available to assist students in the learning of British Sign Language, and to research multimedia theory, specifi cally in the area of learning. The findings of the research were then examined to ascertain whether modern multimedia techniques could be used to produce an application which improves on what is currently available. The main conclusions which were drawn were that it was indeed possible to create such an application, which used multimedia learning theory to maximise the potential benefits to students using it. A prototype application was developed, along with recommendations for any future development of the prototype into a fully func tional application.


Acknowledgements This project would not have been possible without the support of my family and friends, especially those who assisted me with the testing phase of the project. My father, Graham McIntyre deserves a lot of thanks for allowing himself to be videoed for the project. I would also like to thank my project supervisor Iain Stewart, who provided me with valuable advice and assistance throughout the development of the project. I owe thanks to the other staff at Caledonian Univ ersity who gave me assistance when requested. I would finally like to thank my Mum, Morag McIntyre, for the encouragement you gave me to complete my degree, and for everything else.

Ewan McIntyre



1 2

Introduction ................................ ................................ ................................ .............................. 2 Literature Review ................................ ................................ ................................ ...................... 4 2.1 2.2 2.3 2.4 Multimedia Learning ................................ ................................ ................................ .......... 5 Structure of BSL ................................ ................................ ................................ ................. 9 Critical Review of Existing Products ................................ ................................ .................. 11 Research Conclusions................................ ................................ ................................ ....... 26

3 4

Development Methods ................................ ................................ ................................ ............ 28 Design & Implementation ................................ ................................ ................................ ........ 31 4.1 4.2 4.3 4.4 4.5 Interface Design ................................ ................................ ................................ ............... 31 Functionality ................................ ................................ ................................ .................... 37 Installation................................ ................................ ................................ ....................... 40 Updates ................................ ................................ ................................ ........................... 42 Video format and compression ................................ ................................ ........................ 44


Testing ................................ ................................ ................................ ................................ .... 48 5.1 Testing Strategy ................................ ................................ ................................ ............... 48 Testing Results ................................ ................................ ................................ ........ 51

5.1.1 6

Conclusions ................................ ................................ ................................ ............................. 53 6.1 6.2 6.3 Discussion ................................ ................................ ................................ ........................ 53 Future Work ................................ ................................ ................................ .................... 55 Conclusion ................................ ................................ ................................ ....................... 58


References ................................ ................................ ................................ .............................. 59 7.1 Web References................................ ................................ ................................ ............... 61

1 Introduction

Sign language is a visual language, which uses signs to represent words or phrases. There are many different forms of sign language used around the world. British Sign Language (BSL) is the form of sign language which is used in the United Kingdom , and was formally recognised as an official minority language by the British government in 2003. It is difficult to locate an accurate number of current signers in the UK, but the number is increasing. Sign language is typically used to communicate by users with hearing problems, but there has been an increasing wish for different groups of people, including friends and relatives of the deaf, interpreters, and many other groups of people to learn and use BSL and other sign languages. In the UK this means there is an increasing demand for tutors of BSL which is a problem given that there is a lack of teachers available for this purpose in the UK at this time (National Deaf Children s Society, 2009) . This suggests that there is an increasing need for simple and widely accessible methods to learn BSL. There are issues with the teaching and learning of BSL, regional differences can affect the learning process and can contribute to a lack of accepted standards. There is also the feeling that technology has possibly not been embraced to its full potential in this field. These are all reasons that make this a valid and worthwhile project to investigate and undertake. The author of this project also has a strong family connection with the learning of BSL; a close family member is a qualified user of British Sign Language and is employed as a sighted guide for DeafBl ind individuals. With these different factors in mind, this project will investigate how BSL is currently being taught and learned, and if modern multimedia learning techniques could be used to make the process easier and more effective. The advances made in multimedia technology in recent years offer a real opportunity to harness these new possibilities and use them in the field of Education. This is a constantly evolving field, new methods and this project hopes to utilise these new tools to make the process of learning increasingly user friendly and efficient. To achieve this, the following research question was defined; Will it be possible to produce an interactive dictionary to assist in the learning of sign language, using modern multimedia techniq ues? 2

As a visual language, certain multimedia techniques stand out as being particularly suitable for the teaching of BSL. The use of video seems to be an obvious way to demonstrate and educate the use of signs. The ability of modern multimedia systems to effectively link information, and to personalise a users learning experience would also seem to be a useful feature, which can be exploited to develop a comprehensive learning strategy. The aims of the project are, therefore; y To research the multimedia material currently available for learning BSL (and other forms of sign language) y y To research multimedia learning techniques To apply the conclusions reached by this research and design a learning application that improves on the material currently available y To produce a visual, interactive and personalized dictionary to assist in the learning of BSL


2 Literature Review

The aim of the research carried out was to investigate the nature of sign language, and how it is currently being taught and learned. Once this had been established, further research into the techniques of multimedia learning, and how they could best be applied in this context was carried out. This approach effectively split the project into two distinct sections: Stage 1 would first research the principles of Multimedia Learning and how current technology could be best used to implement these principles. Secondly it would research the language of BSL itself; how it is structured and used by its speakers. Finally a critical review would be carried out on the material that is currently available which covers similar ground; similar learning applications for sign language, online courses. Once this research was completed, it should naturally lead to, and indeed justify the existence of Stage 2 of the project. This stage would gather the results of the research done in Stage 1, and examine them in detail , in order to understand and utilise the best practises discovered by the research. This in turn would lead to the specifications and design of the application that would be produced by the project.


2.1 Multimedia Learning

Learning a subject using a multimedia learning package is a relatively new concept. The power of modern computer processors and the advent of fast, reliabl e access to the internet have made it possible for a large number of subjects to be studied in this fashion, including language learning, and sign language learning. Mayer & Sims put forward their Multimed ia Theory (Mayer & Sims, 1994), and it was reconfigured by Mayer in 2003. There are six main points to this theory: 1. Multimedia effect 2. Contiguity effect 3. Coherence effect 4. Modality 5. Redundancy 6. Personalization The Multimedia effect states that a student will learn more from graphics and text than from text alone, (Mayer, 2003). This is based on Paivio s Dual Coding theory, (Paivio & Clark 1988).

Fig 2.1 (Paivio & Clark 1988) 5

Paivio s theory (illustrated in fig 1) suggests that the human brain has two cognitive subsystems. One specialises in receiving and interpreting verbal information, such as text or language. The other specialises in non verbal material, such as images. This principle lends itself to the creation of this project in a multimedia environment. The proposed application will have both text, still images and video to assist their learning. The contiguity effect (Mayer, 2003) has implications for the physical designs of the proposed application. The principle states that a learner will learn more from a multimedia image if the text and corresponding image are placed close together on screen. Careful thought will have to be given to the layout of the screen and the position of the material. Similarly, the temporal contiguity effect tells us that information will be better assimilated by the learner if text and images are introduced simultaneously. The coherence and redundancy effects tell us that extraneous material can distract the learner from the thrust of the material being taught, restraint must be shown if the learner is not to be overwhelmed by the amount of information onscreen at one time. The personalization effect informs us that a student will learn more efficiently if conversational language is used in place of formal language. This all contributes to the issue of screen design. Szabo & Kanuka (1999), state that a poor use of screen design principles can result in increased time taken for instruction and learning, and reduced motivation for the learner. The information presented should be chunked, or broken down into smaller segments so that the learner is not overwhelmed by large amounts of information being p resented in one go. The proposed application will need to have limited amounts of text to go with the visual components to create the most efficient learning environment. Another aspect of interface design is that of aesthetics. Aesthetic design is obvi ously a matter of personal choice for both the user and designer, but poor aesthetic design will hinder the learning process. Careful consideration needs to be given as to how


an interface looks, to avoid discouraging a user before interaction has commenc ed. (Jacko & Sears, 2009) Another aspect of any learning interface, whether multimedia learning or not, is how the learner will assimilate and retain the knowledge gained. Atkinson & Shiffrin (1971) detailed th e Multi Store Model of Memory, and has been r evised and updated since then, including by Baddely, Alan (2003).

Fig 2.2 Atkinson & Shiffrin (1971)

Fig 2.3 Baddeley (2003) This is not specifically relevant to the design of the proposed application, but it is helpful to be aware of how the human memor y takes in, stores and discards information. One of the key parts of this model, and one which is relevant to the application, is that of rehearsal. The model shows that for information to make the 7

progression from short-term to long-term memory, the user/learner must rehearse the information. This means to repeat the information, until it sticks in the mind. One aspect of the application which this is relevant to is the ability to update the dictionary, to place relevant notes in a particular sect ion. This will help the learner to rehearse the knowledge by interpreting it and noting their own perspectives on it. Even the basic functionality of the proposed application, the use of videos which can be replayed as many times as the learner requires, will facilitate the concept of rehearsal, and assist the learner in retaining the knowledge gained. The research done in this area has confirmed the idea that the proposed application will benefit from being created in a multimedia environment. Using i mages, text and videos in tandem will enhance the learner s experience, and will also increase the quality of their learning. The amount of text onscreen will need to be limited, or broken up into smaller pieces if a larger amount is needed. This will s top the learner from being overwhelmed by large, inaccessible pieces of text. Any closely related material, text, images or video, will need to be situated close together as this will increase the likelihood of the learner linking and retaining the knowle dge gained.


2.2 Structure of BSL

Sign language is very different from spoken language. There are, of course, many different forms of sign language. By necessity, this project will focus on British Sign Language (BSL), its FingerSpelling dictionary and th e FingerSpelling language of British DeafBlind. Having defined the sign languages being discussed, the first issue the differences between BSL and spoken English. This is not such an issue with FingerSpelling (of either kind), as this involves the spelli ng out of English words. Fingerspelling is not as large a part of BSL, it is mainly used to spell out names, and words which are unknown to the signer or simply do not have a sign (Mills, 1988). DeafBlind Fingerspelling is much more important, as it can be the only way a user has of communicating. Both kinds of FingerSpelling do not have issues with sentence structure, though, which makes their implementation in this project more straightforward. BSL however, has its own structure. According to Deafwor ks (1999), a sentence in English can be understood from the order of words and from the intonation and inflection used in speech. In BSL, the emphasis is more on the intonation, and the body language or facial expression used in conjunction with the sign. The speaker is advised to try to omit words such as to , and , of from their signing. This places the emphasis firmly on the nouns and verbs in a sentence. Mills (1988) uses placement as an example of how structure can be totally different in the two languages. The example used is that of a ball which is under a shelf. In English, the sentence would simply be The ball is under the shelf . The BSL example given is of one hand making the sign for ball, while the other hand makes the sign for shel f above this. This is a good example of the complexity of BSL, and the difficulties evident in constructing a sentence builder as this project originally proposed. Miles (1988) devotes a large section to the facial expression and body language which are used to communicate BSL, in conjunction with the basic hand signs. This is of interest, as it reinforces the need for a visual, interactive method to teach or learn BSL. The sheer variety of facial expressions and body language which can alter the 9

meaning of a sign would be very hard to demonstrate in a learning environment without exact demonstrations. Miles (1999) advises that any learner would be best served learning personally from an instructor. This again stresses the importance of the videos, which are the main component of the proposed application, and will hopefully serve as a substitute for a teacher, while imparting a similar learning experience. Another problem in the creation of a sentence builder is that there are signs in BSL which encompass several words in English (Spence & Woll, 1999). One example shown is that of the English phrase I ask you , which is represented as a single sign in BSL. Other complexities are evident in the order of verbs and nouns in a BSL sentence. Spence & Woll (1999) make a distinction between effective and affective verbs, and how this affects sentence structure. Effective verbs cause something to exist, while affective verbs affect something that already exists. If the signer is using an effective verb (in conjunction with a noun), the verb will come first, as the noun does not exist until the verb is used. In the case of an affective verb, the noun will come first. This can conflict with other conventions of BSL that the topic of a sentence (which will often be the main noun), will come first (Spence & Woll 1999). The issue of BSL sentence structure is particularly relevant in the construction of the proposed application. The initial plan was to incorporate a sentence builder, where the user would type in a sentence, and the application would then generate the sentence in BSL, which would then be visually displayed using videos of each separate word. Further research into this possibility has highlighted the fact that this approach is not really feasible, given the time constraints, and the author s technical skills. The refined proposal is that the dictionary in the application will be much more vocabulary centred. The user will select from a list of words, or can search the database, and the relevant video will then be displayed. It would be helpful for the student to have an idea of how a sentence could be constructed using the word that has been demonstrated; a link to sample sentences using each word would help to instruct and reinforce the different structure of sentences in BSL. 10

2.3 Critical Review of Existing Products

This section will look at similar products which are currently available on the web , and also on CD/DVD. The aim is to assess the material which is currently available, and whether this can be improved upon. This exercise should also help to define the possible structure of the proposed application, and give guidance on features which are or are not useful and efficient. The first site looked at can be found at, and is also available on a CD. The main part of the site is a dictionary of specific word groups. As can be seen from the screenshot below, these are arranged into different sections, so the user can easily learn related groups of words together. These groups include days of the week, weather, time, directions and others. There is also a more straightforward dictionary section, with larger lists of words. The actual demonstration of the word is via an animated gif. This is basically a series of images which will be displayed in sequential order. This saves on file size compared to a video of the same scene. There are , however, no user controls, so the scene has to be watched through from start to finish each time. The overall effect is less satisfactory than using a video, the display is jerky and imprecise, with a lot of nuance being lost.


Fig 2.4 The previous screenshot (Fig 2.4) shows the format of the site. On the left is the home page, with the selection of word groupings. The top right of the screenshot is what appears once the weather section is entered, and the bottom right is the animated gif detailing the word cold .

Fig 2.5 -

The second screenshot shows the dictionary section of the site. This section functions in the same way as the word group section, although the words are grouped alphabetically. The selection of the words seems rather random, although this is probably because the majority of the words are from the group sections, and make more sense in that context. 12

This site is similar to what has been proposed. Hopefully the animated gifs would be replaced by videos, and a larger, more comprehensive dictionary would be employed. The splitting of the words up into related sections works well here, and makes it easier for the user to find what they are looking for. The site has a page which mentions sentence structure, and gives one example. More can be found if the product is accessed through the CD, rather than the website. This touches on what the project is aiming for, and demonstrates the possible difficulties in creating an application which would translate English phrases into BSL phrases.


The next site looked at is . This site is less professionally made, and looks like it was conceived as a personal project by the two creators. It has not been updated since 2007, but still serves as an example of a similar product, although it is largely uncompleted. This site also splits the words which are displayed up into sections, colours, countries and animals are the only three which are completed. In each section there is a video, which displ ay a number of different words being signed, with captions beneath them.

Fig 2.6 - The videos are hosted on YouTube. This solves any potential problem of hosting the videos, but does mean that video quality will not be high, and that control of the videos is lost. The videos run through from start to finish, so if the user wants a specific word from a section, they will have to sit through the whole video. The quality of the video is fairly high, and the user gets a much better sense of signing the word from this, as opposed to the animated gif approach used by other sites. This reinforces the intention for the proposed project to use video for this purpose. The main flaw in this site use of video is that the videos are not broken up, accessing


each individual word would make this a better experience than the way it is currently.

15 This is a much larger site in scope, and was created partly by Cath Smith, who has published a number of books on the subject. There are a number of interesting sections which are relevant, but this will focus on the interactive sections, which comprise a BSL Fingerspelling translator and keyboard translator, and the same two options for DeafBlind. The Fingerspelling translator allows the user to type in a word or a phrase and the Fingerspelling translation will be displayed in order. Similarly, the keyboard translator allows the user to press a key, and the translation of that letter will be displayed.

Fig 2.7 - The user has the option to speed up or slow down the display, and there is a limited selection of quick phrases that can be chosen. The site uses simple diagrams for this translator, which works quite well for the translation of Fingerspelling, as body language and facial movements are not an issue. BSL Fingerspelling is different from Deafblind Fingerspelling, and this is reflected here with different translators. This is similar to the proposed Fingerspelling section of the project, although videos are the intended mode of delivery.


This site does not attempt to offer any BSL translators, it focuses on Fingerspelling. This illustrates the difficulties involved in creating a product which does this satisfactorily. The site does what it intends well, although the use of videos would enhance what has been done here considerably. This is a course to learn BSL, rather than an online resource. It costs £15, which gives the user access to the material for 1 year. Because of this, this report can only comment on the promotional material available. Again, the words are split into sections, such as family, time and colours. Video clips are then used to demonstrate the words.

Fig 2.8 - Rather than having a dictionary to access, it appears that this course is more of a linear instructional course, which the user would complete in weekly lessons, sitting tests and finally gaining a certificate if they pass.

17 This site offers the closest example of what is being proposed in this project. There is a video section for BSL, as well as videos for BSL Fingerspelling. On the site, t here are a small number of BSL videos. This resource is also available on CD, which costs £40, and seems to have a comprehensive database of videos for 1700 words. This also includes regional variations for signs, example sentences, and the word spelt ou t in Fingerspelling

Fig 2.9 - One thing this product does not have is anything concerned with Deafblind Fingerspelling, which is one of the propo sed components of the project. It is well made, and the video quality is fairly high. It has a Fingerspelling machine , which is similar to the Fingerspelling translator from listed above. It uses


videos rather than still images, and it shows how much more effective thi s approach is. Again, in this example, the words are split up into sections, such as colours and family. This seems to be the accepted approach for the teaching and learning of any language, not just sign language. It seems unlikely there would be a more intuitive approach to take.

19 This is another online resource, which is also available on CD, funded by the Scottish Executive. Once again, the words available are split up into sections. This product uses animation to demonstrate the signs. This works fairly well, although video adds a realism that just cannot be achieved by animation, no matter how professionally done it is. The interface design is something of a problem with this application. The central display is fine; it tells the user which section they are in, and which sign is currently being demonstrated. The buttons on either side of this have no markings however, this does not give the user much feedback on what they are doing.

Fig 2.10 - Another problem with this application is that of loading times and efficiency. This was tested on an up to 8 MB connection , but there were very slow loading times


for each individual animation, with no feedback to the user as to why the video was not displaying.

21 This is a resource for learning American Sign Language (ASL). Obviously this is a different language from the one the proposed application would be using, but the principles involved in learning the two languages are the same. This is an online learnin g course, which has to be paid for to gain full access. The first word from each letter in the alphabet is available as a preview. The amount of words available is larger than any of the products previously looked at, and the accompanying videos are of high quality.

Fig 2.11 - The Fingerspelling section of the site has a similar layout, and uses still images to demonstrate each letter. One feature of this site, that was not seen o n any previously, are the hints beneath the word, which gives the user hints as to the movement of the hands. This is a useful feature, which adds to the information being presented to the user.

22 This is another American site, which offers a range of dictionaries, the main dictionary, which has a large number of words in it, a religious signs dictionary, a conversational dictionary and an ASL for babies dictionary. The main dictionary is unusual, in that it does not have sections for the different words ; they are presented as a straight dictionary. The conversational section offers different themes for the words, like the applications that have been viewed previously. The babies section is a collection of simple words that children will learn before more complex language. The religious section is very comprehensive; with many words which are not commonly used signed, abomination and antichrist are two examples.

Fig 2.12 - This screenshot shows the phrase section of the site. It has a selection of different subjects, and within each it has a dropdown list of related phrases. The example above is the citizenship phrasebook. This is perhaps a partial solution to the problem of sentence structure in BSL which causes problems in translating a sentence from English. Possibly directing the user to related phrases when a word translation is requested is a feasible way of demonstrating sentence structure, without having a program that directly translates a sentence, as this would be problematic to create.


Below is the main dictionary section of the site. The desired letter is clicked on, and a list of available words appears. The video s are of high quality, and are fully user controlled via the panel on the right.

Fig 2.13 - ASL pro also offers the world s first Video Dictionary . This feature works well, although it is still in the testing phase. The site has a basic main page, from which each letter of the alphabet can be selected, then the words. Each video is then downloaded to the phone. This could possibly cause problems on phones with small memory capacity (which is the majority of currently used phones), although the file sizes are small, they would build up if a lot of videos were downloaded.

Fig 2.14 -


This is an interesting and useful addition to the ASL pro site. This is a possible direction the proposed project could take, and further research will have to be done about the feasibility of this feature being incorporated into the project.


2.4 Research Conclusions

The research done here shows that the proposed application is a viable product. The applications looked at in the review of existing products are a representative, but by no means exhaustive example of the similar products which are available . The proposed application is similar in technique and scope to some of the sites and applications viewed. The aim of this project is primarily to produce an application which will assist in the learning of BSL. As the research has discovered, there are already a number of similar products currently in the marketplace. The challenge is to produce an application that will improve on any facets of the application which cover material already presented. The introduction of a personalization feature, which will allow the user to create and save their own notes, as well as build up a dictionary of words which have been previously learned will help to make this product be a useful addition to what is available. One point taken from this research is that of the structure of the site. Every site looked at split the words available into specific section, for example travel, health, colours, countries. This seems the most effective and intuitive way to proceed when designing the interface for the applicati on. The design of the user interface, both functional and aesthetic, will also play an important part in how successful the application will be. Attention will need to be paid to the points uncovered in the research into multimedia learning techniques, a s to how best to create this application. The research carried out has intimated that the proposal as originally envisaged requires some modifications. The idea to have a sentence building tool, where the user would enter a sentence, and the program would then construct the sentence in BSL and display the appropriate videos in order, was overly ambitious. The complexity of structuring BSL means that this feature would not be feasible in a project of this scope. The other proposed features are achievable, and should improve on the similar products which are currently available.


The product which will be developed is intended to have the following features: Preliminary Specification: A BSL dictionary; where each word will have a video displaying the correct sign along with a definition. Each user will have a separate user identity, which will allow them to personalise the dictionary by updating the notes. Each user will have a personal vocabulary where words that have previously been learned will be accessible separately. A quiz section will also test the user on the words they have learn ed. This feature would reinforce the learning that has already been achieved, and maximise the possibility of the user retaining the knowledge they have learned. A finger spelling guide, where a video would be produced for each letter, and the user should be able to input words and the videos will display in sequence to illustrate the correct pronunciation. It should be noted at this point that the timescales and sc ope envisaged for this project have been severely compromised at this stage in the development, due to unforeseen circumstances in the project developer s personal life. The developer has decided to continue with the project at this time, and hopes that t he quality and scale of the project will not be compromised too much.


3 Development Methods

The research that has been carried out up to this point was to indicate the best direction for this project to take to best answer the initial research question. The proposed application will be CD/DVD based. There were a number of factors which influenced this decision; as the application is intended to make use of a large number of video clips, the size of the application is obviously a concern, as is bandwidth if the videos were to be streamed by users. Distributing the application in this manner eliminates these concerns, and ensures that users who do not have access to the internet will be able to use the application without restrictions. This does not rule out the possibility of the application being modified and hosted online in the future. At this stage it was felt that the application would benefit from being distributed on a physical format. The proposed application will use video to demonstrate each sign in the dictionary, and will allow the user to log -in, and access a personalised dictionary and notes. To facilitate this, a back -end to the software application will need to be developed. Typically, in a web application, such storage and access methods would require a data storage structure hosted on a server. As CD/DVD is to be the intended distribution format, a connection to the internet cannot be guaranteed for the user. To this end, ADOBE Air seems to be the best solution currently available. Other solutions were considered ; Microsoft s Silverlight is a similar framework for building applications. At the time of research however, it became apparent that the database functionality was not quite at the level of ADOBE s AIR system, crucially, accessing a database on the local, host machine was a function that was still in alpha testing at this stage; there was a possibility that this would no longer be the case when Silverlight 4 was released (this was intended for April 2010), but this would be too late to fully implement for the purposes of this project. Another option that was considered was that of using the Delphi framework. This is a Windows based application development tool, which has the benefit in this case of the project creator being fa irly experienced in using it. Unfortunately, this framework has no cross platform support, and was discounted for this reason, as this would restrict the scope of potential users. The project 28

creator had similar experience of creating applications in ADO BE Flash, which is one of the frameworks which can be used to create applications that run with AIR. Aside from this, there were other apparent benefits from choosing this course; AIR has built in SQLite database functionality, which allows the applicatio n to access a database, and this also allows the application to access a database stored locally on the host machine, without the need to contact a host server. This means that an internet connection is not required, and all the information can be stored, accessed and updated as required. AIR is also a cross platform system runtime, and it supports native desktop functions, such as using the clipboard for copy and paste. The application itself will be developed in Adobe Flash CS4. FLEX is another platform which can be used to create AIR applications, but it was felt that the creator s inexperience with this platform would create an unnecessary obstacle to the project, and it was felt that using Flash offered the best chance of creating an application that would successfully fulfil the requirements of the original research question. The database structure itself will be a crucial factor in whether the application itself will be successful. Close attention will need to be paid to the normalization of the data, and how it is stored and accessed. The large number of videos, and the associated data, mean that it will be very easy to cause unnecessary complexities in the application. The first stage of the design process will be to ascertain that the desire d functionality of the application is possible. This will require a very early prototype to be built, to test the viewing of videos on demand, accessing them from a database, allowing the user to search the application for the desired video, and being abl e to update information on the database. Once this has been achieved, the user interface will be designed. This will encompass the actual look of the interface, colour scheme, button design and placement, using HCI principles to ascerta in the most effective methods. Once a prototype has been developed, a series of cognitive walkthroughs will be performed, testing the application design against a set of heuristics. This will entail a number of scenarios being created, which will involve the user walking through the 29

application in different ways, trying to access different information. The navigation structure and interface will then be reconsidered based on the findings of this technique. This should minimise the amount of mouse clicks required to perform tasks, and ensure that users can intuitively navigate their way around the application. At the early stages of development, storyboards and navigation maps will be produced, and meeting s will be arranged with potential interested parties to assess the material, and give feedback on its viability, with suggestions on possible improvements and alterations. Once a prototype has been developed, a questionnaire will be distributed to evaluate the prototype, and to ask more general questions about what the test users would like and expect to see in an application of this type. It is anticipated these will be people who have learned, or are currently learning BSL, as these are the obvious target users. A selection of these same users will be asked to evaluate the application after the initial development stage. This will comprise the target users, and expert users will also be recruited for evaluation purposes.


4 Design & Implementation

4.1 Interface Design

In designing the initial prototype of the user interface, various factors had to be taken into consideration. HCI and Multimedia Learning principles, which recommend the best ways to increase the usability of an interface, will need to be looked at and applied to the design of the interface. Placement of the various elements on screen is very important. The main information, in this case the video demonstrating the sign and its accompanying notes will occupy the centre of the screen, as this is the material that the user s attention will be drawn to. One problem with this approach is that before a video is selected, the space that the video will occupy will be blank, which leaves a large piece of unused space on the screen. However, th is space cannot be used to display any important information, as this will not be displayed while a video is being viewed. It is possible a placeholder image or the application logo would be the best solution to place in this space. Consistency is anothe r important principle of HCI. It is essential that the user knows where they are at all times, and what their possible options are. To adhere to these guidelines, there will be consistent placement of buttons throughout the interface. The buttons which will appear on most pages are the exit button and the log out button. These will be placed in the corners of the screen, to utilise Fitt s Law (Crossman & Goodeve, 1983), which states that the four corners of the screen are the most quickly and easily acc essible points of the screen. If these buttons are placed in these positions consistently, the user should have a clear idea of how to escape from their current location, no matter where they are in the application. The navigation structure is another important stage in the design process. A well designed navigation structure makes the user s path through the application a simple and intuitive task. To this end, the design of this application aims to be a fairly simple affair, with a minimum of different screens. The amount of material 31

( © ¨¢©¦ ¡© §¡ ££¢    ¡©  §¦ ¢¡© ¨' ¥ §£¦ ¢¥ ¤ ££¢  §¦ ¡ ¡© ¨¢ ¥ £ §¡   ¡© ¨'     ¨ § §  © ¢   ¥  §¦ ¥¨§ §¨   ¢ ¡©  ©¨  ¡©¢ ££¢    ¡© ¨¢©§ ¢£¦¦§ ¡© ¨¢ ©¨ ¨'  §¨¢© ¢¥ ¥ ¢£§¨  ¦ §   ¡© ©§©¢£¢ § ££¢  ¡ ¢¡   §£¦ ¨¢  © ¥  §¦ §¨  § ¤ ££¢   ¡ & © ¦ ¡©  ¦ ¡© ¨¢ £¤§ ¢¡ § ¥¨§ £¤¢ §  ¢ ©§¡  ¨¢§©¨ ££¢  ¨¢©§ ¢£¦¦§ % ¡© ©§¡  £© § ©¤§ ¨ §© ¤ © §¡ ¨¢ ¢ ¥ ©¨ ¦£ ¥ ¨¢©§ ¢£¦¦§ ¡©  §© ¢¡© © ¥  ¥¢ ¨ ¥¨§ ¥ ©    ¨¢© # ¡ §  ¡© © ¨¢©£ ©¨  ¢¥   ¤¨ §  §© ¨¢¨¨§£¦ ¡© ¨¢" $¨¢§©¨ ¨¢©§ ¢£¦¦§ ¡© ¥£¡ £§¢ ©§ ©§¡  ¢ ¨¢ ¥ ¡©  §© ¢¡© ©§ ¨¢© # ©¨ ¨¢© ¦ ¡  ¢ ¨¢ " ¦ ©©!
y e es se es e ese c ee e e e c c c se e e e s eve e c c c es se es e s ss s ee y e e ye e e ec s ss ss e se s e s e e e e s s sys e s ve e e es e s c e s ee c ev e se e e c se ce s e e ce e se e c ve ss e sc e se e c y e e ee c e e s e s e e e s e e e e se e e e jec s e 

§   ¡© ¢¡ ¡© ¨ ¨¢¥¨ ¦ ¥ ££§ ¢§¨ ¥ ¥ §£¦ ¢¥ ¤ ££¢  ¥¨§  §¦ ¢¡© §¢ ¥ § ¤ ££¢  £§¢ ©§ ©¨§ £  ¡©  ££ ¨  § ©¨¢ ¨¢§ ¡© © ¦ ¨ ¡© ¥¨§   ¢© © ¢ § § ¡© ¢  © ¢   ¨¢ £  ¡©¢   ¡©  ¡  ¨  ¨¢£ § §¡ © ¢ ¨ §© ¤ ££¢  ©§¡© ¡ §¦¦§ £§¢©¢¨¢ ¡   ¡  §© ©£ ¢¢¥ § ¢¡© § ¨¢©§ ¢£¦¦§ ¡© ¨¢ ¥ §£¦ ¢¥ ¤ ££¢  ¡ ¢¡ 
e s e s c ce sc ee ye e ses e s c y ye c y e e ey e e ev e e e e s s c e se e ve ec es ces e ccesse v sc ee s e e se c esses ee s es s e se e eve e e e e

Se c

pi Y d h ga ` cd Y fe ` d c cba` Y WXRI UR I WH P VRSU TSRQ P I H @ 0 12 F605 3 2 C 4 9) 39) 3 )CE 73 4 CC7D 2CB 0 12 CC) 06 G)2)3 897 9)EF6 ) 127D 6 37 12 ) 3 )CE 73 C2 073 4 0 127 CC7D 2CB 0 1A @987 3 07 3 12 065 )4)2)3 12 1 0)
s s ye ye e s c s se ec e c ev e se e se ec e s e es e s s cc cs e ev n t s ee y e es e es e e s e e





View a word (on entry a full list of words will be displayed, these will be



filtered by the topic choice Once Se ec



topics will be displayed and be F 41 e y c e


View their personalised vocabulary/dictionary, which will display relevant, updateable notes


The words will have multiple videos, with close ups and body l anguage focused on

y y

Download new material Log out, or close the application

One of the problems encountered at this early stage, is how to display the topics and words that the user is able to select and view. The initial idea is to have these displayed in dropdown menus. An obvious problem is that with the large number of words which the final application would hold, these dropdown menus would become large and unwieldy. A partial solution to this is to have the list of words available filtered by the user choosing topics of words, and possibly sub topics. This would still mean that the initial list of words would be large, but the user would have the option to easily make the list more manageable. Another tool to assist the user at this point will be t he option to order the list either alphabetically or by topic, without filtering the word list down. After obtaining some initial feedback from target users, some issues were raised with the proposed navigation structure, specifically with how the user is presented with the words, and how the dictionary is visually structured and grouped. The initially proposed format of having multiple drop -down menus filtering the visual access method to the dictionary is possibly not the most efficient or intuitive of methods. Problems are possible with an overload of information provided, especially in the menu that displays the available words. There are filters in place, which would reduce the amount of options presented. It is felt that alternative methods of filtering and presenting the information should be considered at this stage of the development. HCI principles are also a concern in this regard , as the research done into Multimedia Learning showed (Mayer & Sims, 1994); care has to be taken not to overload the user with information that is redundant according to their current location within the application. Attention will be paid as to how online repositories of large amounts of information structure and present it, such as encyclopaedias and dictionaries among others. It is possible that an optimal solution 33

will not present itself in the timeframe available for this project, but it is important to optimise the application as far as is possible, and to plan for future upgrades and maintenance of the application. After a review of these kinds of resources, several possible approaches seem feasible. As the original research in the literature review showed, information like this is best presented in chunks. The dictionary will be available to search as a whole, but will function better in smaller chunks. Based on this, and subsequent research, the dictionary will be split up into topics for the user to interact with. In addition to this, if the user wishes to view the dictionary without the use of topics, the words will be split up by letter and alphabetical order. This is obviously too much information to present and divide on a single page, as was the initial intention, so further screens will have to be created to hold this information. This corresponds to the research conducted previously, namely that information should be split up, or chunked, so that the user does not suffer from an overload of information. To make the navigation more intuitive, and that less information is presented at one time , a tabbed structure is felt to be the most efficient and easily recognisable metaphor to use. Flash, the development tool being used to create the application, does not have a tab structure component, so one will have to be created, or at least an approximation of one. A tab for the complete dictionary, with drop down menus filtering both the letter desired, and then the further divisions within that letter seems to be the best way to present this section. It is felt that a further dropdown box to present the actual words after filtration is not the best method of displaying this. A list box, with selectable words and a scroll bar for the overflow would eliminate the problem of very large and unwieldy drop down lists, which makes the whole process more intuitive and efficient for the user. A further tab for the topic lists would also be available. This would follow a similar structure to the previous tab, which will ensure consistency is maintained. In this instance, dropdown menus would be employed, with topics, and then sub topics filtering the list of words, which will again be displayed in a list box. The personal vocabulary would again follow a similar structure, with the words divided by letter and sub letter. The phrasebook section has slightly different issues. When developing this section, it was found that 34

although the phrases can be split up into topics, further division into sub topics is not as straight forward; phrases are not as easily treated in this way as single words are. The decision was taken at this stage to limit the filtering process to general topics for the phrasebook section. This is a possibility for work at a future stage of development; it was felt that with the available timescale for the project, that this was an unnecessary, time consuming complication.

Fig 4.2, 4.3 These two screenshots show the layout of the application. The tabbed structure can be seen, with the application split up into six different sections. The video is centrally positioned, with the relevant information (the notes and the definition) positioned closely, as the research done has shown that this is the most efficient 35

method of presenting multimedia information to maximise user retention of the information, according to the contiguity effect (Mayer, 2003).


4.2 Functionality

The next stage of the development is that of the database structure. The platform chosen for the development of the application has had an impact on the database design. Previously it had been assumed that the database would have to physically store the multimedia files that would be accessed by the application. After the decision was taken to distribute the application on CD/DVD, and use Adobe AIR to develop it, this simplified the situation. As AIR allows the application to access a database located on the local host machine, this means that only the path to where each multimedia file is located needs to be stored in the database. All the relevant data fields will have to go through normalisation, which should reduce the data to separate tables, and produce the most efficient database structure. The main problem was how to resolve the multiple many to many relationships that the application presented. After preliminary work on this, the following database structure has been devised: Usertable: userID, username, fname, sname, password Wordtable: wordID, wordname, wordlocation, definition, notes, topicID, letter, subletter UserNotes: noteid, userid, wordid, usernotes Topictable: topicID, topicname Vocabtable: vocabID, userID, wordID These tables should function to allow the user to access the information they wish to access quickly and efficiently. Further tables will need to be created, as well as additional fields in these tables, for a phrasebook and other parts of the application. The exact functionality of the application as a whole is another complicated issue , which has obvious implications for the way both the interface is designed, and the structure of the back end; how the data is stored and accessed. One of the original


ideas was to have updateable notes which would be viewed in the users own personal vocabulary section. It was felt after receiving feedback that the complete application should be log in based, with each word having personalised notes displayed, whether the word is in the u sers vocabulary or not. This approach actually simplifies the database structure. A definitive solution had yet to be devised for the problem of creating a table for the user notes. This decision meant that a very simple notes table, with fields called wordname, wordnote and userid could be created. The wordname and userid fields were implemented as compound keys. The decision was taken to dispense with a numeric id key for this table. This was due to the method used to insert information into the tab le. Ideally, this table would have had a numeric ID key, which would automatically increment when a new record was added to the table. It was decided after research into the most efficient methods, that updating the notes table would require a REPLACE co mmand. Basically, in mySQL, this command checks the table for the required record, using the primary key, and if it does not exist, inserts the data as a new record. In this case, having an ID field meant that the update function would not work as desire d, as it was automatically incrementing, a new row would be added every time the user tried to update the notes. With the change to the table structure, the user will either insert or update their notes on a specific word. The original database structure had undergone several changes from that detailed previously in this report. With the application being completely login based, it becomes imperative that each user is completely separate from the others, so that their notes can be stored separately. Each user has a unique user name, and a unique numerical ID number, which automatically increments as users are added to the application. This will ensure the integrity of the data stored. The revised data structure can be seen in the Entity Relationship Diagram (ERD) at Fig 4.4.


Fig 4.4

Another change that was decided on at this stage of the development was that of the default notes. The original intention was to have default notes for each word in the dictionary, which could be updated by each user f or personal reference. The decision was taken to not supply default notes; it was felt that, as a definition was supplied, it was not immediately obvious what information could be supplied in the default notes. This function is to be left for the user to implement as they see fit.


4.3 Installation

At this stage in the development, testing was done with publishing the application files. Up to this point in the development, testing of the application was done through the Adobe Flash interface. To test the interface properly, the application has to be published as an Adobe AIR application. This involves creating an installation package, which AIR does automatically. The author needs to point AIR to all the files and folders which will be required to make t he application run correctly; in this case the database and the video files , and these are then bundled together in the installation package. The application also needs a digital certificate to be included, in this instance created by the author to reassure users that the application is safe to install on their machine. In this early testing phase, the application runs smoothly on the three different machines it was installed on. The main issue with the installation process that was not fully appreciated when this platform was chosen is that the user machine will need to have Adobe AIR installed. This is not a huge problem, it is a relatively small file and easy installation process, but it is obviously a slight barrier tha t was not anticipated. Another possible problem is that the user may not have an up to date version of Flash Player installed on their machine, or indeed have the player installed at all. Testing will have to done on a machine which does not have the pla yer installed. One solution is to include Flash Player in the installation package. This will also be part of the testing, as simply including this as an executable file may not be enough to associate the application with the player. Instructions will b e included to counter these problems, but it is possible that a more elegant and intuitive solution could be found. A solution was indeed found to this problem, and is discussed in the testing strategy section (5.1) of this report. A further problem was found with PCs that are running Windows Vista and 7 as their operating system. The application runs fine on Windows XP, but there are permission issues with the newer editions of Windows. The application needs write permissions to insert data into the dat abase, and Vista and 7 do not allow this to happen. One solution is to run the application as an Administrator, but again it is


possible that a further workaround can be found, without having to instruct the user to do this.


4.4 Updates

The next stage in the development that was embarked on was the update feature. This feature will allow the user to download updates from the internet, and these will then be integrated into the application. This is not a core element of the application; one of the major de cisions made was to host the application on CD/DVD rather that it being a web based application. This of course means the user does not need internet access to use the application. It was decided for the purposes of this prototype, that updates could be offered online as an option. In a completed and distributed application, updates would be distributed using both mediums. The development issues discussed here focus on the online hosting of the update material. The updates would comprise new videos, an d a text file which would insert the details for each new item in the appropriate tables in the database. The structure of the updates is more difficult than was envisaged at the planning stage. Should the updates be blanket updates, inserting new topics and words? Or should there be a range of topics available to the user, who can pick and choose what to download. At this stage, the most important factor is to ensure that the facility works. A number of download methods were considered and tested. F lash does offer an htmlLoader component, which was successfully implemented in testing. Unfortunately, at this stage, the component does not seem to offer a way to actually download files, rather than simply view them. If a solution to this problem canno t be found, an alternative method will have to be found, one which is possibly not as intuitive for the user. The solution that presented itself was to open the users default browser from within the application, and give it the necessary URL where the download material is located. A satisfactory method of then automatically downloading the material was not apparent. The solution found and implemented at this stage was simply to have detailed instructions for the user to follow to download and install the material. This is not an ideal solution, as it relies heavily on the user having some knowledge of file paths to place the material correctly, but at this stage a better solution could not be found. This will be discussed further in the future development stage of this report. AIR does provide an application Directory 42

command, which allows the application to direct the user to the correct folder to place the material. After this has been done, the user will enter the name of the update package they have downloaded, and the application will read in the text files contained in the downloaded package which contain the required SQL statements to enter the material into the database. Again this could possibly be improved on, with the application automatically reading the files, but it may not be possible for this version of the application. Another issue with the downloaded material is the compression of the files which are downloaded. It was felt that to avoid the user having to download many files, it woul d be sensible to compress the videos and text files for each package into a single archive file. It was felt that using 7-zip to compress the files into the archived ZIP file was the best choice, as it is an open source software technology, and as such is freely available.


4.5 Video format and compression

As the application is being designed using Adobe Flash and AIR, this has implications for the format that the videos which will be accessed by the application can be. As initial testing was being carried out, a problem was discovered with the initial form at in which the videos were stored and accessed. The videos were being imported into Flash, and separate SWF movies were being created with them. These movies were being called by the application and displayed as separate movies in an external Flash player at the appropriate time. The problem that was encountered with this approach was while using this method in conjunction with the download facility offered with the application. Initially, the files were being compressed and archived using Winrar, which allows the user to download one file and decompress it on their machine, which should save the complication of downloading numerous files. The problem arose when the SWF files were decompressed and placed in the folder for the application to access. The SWF files had become corrupted at some point during the compression / decompression process. Fortunately in this instance Flash and AIR offer an alternative approach. Rather than loading external SWF files, Flash CS4 has a built in video player, which l oads the video files within the application without the need to call, or indeed create, external SWF files. This eliminates the corruption problem, and actually simplifies the process of loading the external videos into the application. Once this alternative method had been established successfully, the next issue was of video formats and compression. The native Flash video player can use FLV/F4V and MP4 video files. Using Adobe Media Encoder, there are many variations of file size and quality which can be achieved using different levels and types of compression. In this instance file size is not such an important issue, as the application is not online at this time, and download or streaming speed is not a factor. However, it seems prudent to keep the file sizes reasonably small, to avoid filling up the user s hard drive with large video files. Conversely, video quality is very important in this context, the ability of the user to clearly see and understand what is being displayed is crucial to the su ccess of the application. To achieve the best balance between file size and video quality a series of tests were set up. Using a high 44

quality source video, Adobe Media Encoder was used to compress this into a variety of formats. Table 4.1 shows the results of this testing. The compression was tested for both the fully zoomed out video, which will encompass the full body shots of signs, and zoomed in video, which will be used for the alternate videos, and the FingerSpelling videos.


Table 4.1
MP4/H.264 File Size (original 10Mb) PAL DV HQ IPOD LARGE IPOD SMALL DID 1.56 1.14 NOT Good Good possibly too small YAHOO VIDEO 4.2 Very Good, large file YOUTUBE SD 3GPP 352x288 15fps 5.36 556kb Very good. large Poor, only 15 fps. same same same PLAY 1.54 1.13 Good Good Video quality Zoom size Zoom Quality

FLV/F4V F4V Same FLV Same

File size 2.24 2.34 2.24 726kb

Quality Good Good OK Good, video is the correct size

Zoom size 2.23 Same Same Same

Zoom quality Same

F4V Web Large F4V Web Medium & small F4V Web Modem


Good quality, but very small video size


Web small/


Good quality, good size


medium FLV web large 2.35

Very good, video is too large.


After these tests were carried out, it became apparent that FLV/F4V video was the best format to use. Neither offered any real advantage, either in file size or in video quality. The decision was taken to use FLV, as these files can run on Flash Player 8, and F4V files run only on Flash Player 9 and newer. This is not strictly relevant to the application, as Flash Player 10 will be included in the install package. The sensible


option would seem to use the videos that are the most accessible to the user however. Using this format, the next stage is the type of compression that would be used. The best results came from the Web compression, sizes Medium and Small. The file sizes for these two formats were 1.01Mb and 744Kb. The Web Medium format exports to around the desired size for display on screen (around 320 x 240 pixels), and is not a huge increase in file size, so this is the format that was decided on for the video demonstrations in the application.


5 Testing

5.1 Testing Strategy

Testing is an important stage in the development of any product. Any basic development lifecycle includes a testing or evaluation stage. If the product does not function the way it is intended to, the chances are that it will never be used, and the intended audience will never be reached. The testing that will be done on this application will focus on the functionality of the various properties of the application. There are various different methods of testing this type of application ; these are some of the options available: y y y y y Heuristic evaluation Cognitive walkthrough User Evaluation Observational testing Think-aloud

A heuristic evaluation involves a user making two or more passes through a system, and evaluates it against a set of heuristics, or recommendations. A cognitive walkthrough involves a user (or users) being set a series of tasks, and the steps that must be taken to achieve these tasks are analysed for usability, efficiency and other factors. User Evaluation refers to users giving feedback through questionnaires or interviews about their experiences with the system or application. Observational testing involves observing, either directly or through a medium such as video recording, test subjects using an application, while Think -aloud evaluation requires the test users to vocalise their thought as they navigate a system or application. This is not an exhaustive list of testing methods, but it was a reasonable number to assess before deciding on a strategy. Each method has positive and negative aspects, but it was decided that using aspect s of multiple methods would gain the best results, and hopefully provide worthwhile 48

feedback. To this end, a cognitive walkthrough was devised, which would instruct the test users to perform a series of tasks within the application. These tasks would be designed with a set of commonly used heuristics in mind; namely: y Visibility of system status: is the user aware of where they are located within the application, and what is occurring at a given time? y User control and freedom: The user should have clearly posted escape routes if they enter a section of the application erroneously y Consistency: The same words, buttons and other factors of the application should be consistent throughout the application y Recognition rather than Recall: The user s memory load s hould be minimised by clear instructions as to how to proceed y Recovery from Errors: Error messages should be accessible and plainly displayed

Match between System and Real World: The System should be accessible and recognisable to the user, and be structur ed in a logical fashion

The walkthrough devised does not explicitly evaluate these heuristics, but with the use of a questionnaire, appropriate questions will be asked which will produce a picture of whether the application succeeds when evaluated against these principles. This devised testing strategy incorporates ideas from a number of evaluation methods, and will hopefully analyse the application successfully. One issue which arose while devising a test strategy was that of the distribution of material to the test subjects. It had been assumed that the application would be relatively simple to distribute and run, but this proved not to be the case. The Adobe AIR framework was required on the user s machine to actually run the application, a factor which was missed in the initial research in to the various possible platforms. It had been assumed that an up to date Flash player would be sufficient, but this was not so. The AIR installer file was also a problem; despite being relatively small in the development stage, as only two videos were included to keep the size down, problems were still encountered. The easiest way to distribute the file was assumed to be by email. Unfortunately some of the test subjects had file size limits imposed 49

by their email providers, and the AIR installer file exceeded some of these limits. After consultation with Adobe s own forums and documentation, a relatively efficient workaround was found to this distribution problem. A facility called a Seamless Badge Installer was available, which allowed the user to download the AIR installer file from a website. The Badge Installer also checked whether the user had the Adobe AIR framework installed on their machine, and downloaded and installed this automatically if this was not the case. The only potential problem with this is that Flash Player 10 is required in the user s browser to allow this method to work, but as the Badge Installer is hosted on an HTML web page, this can be accounted for by including a message and link to download if this is the case. This method of distribution is far more efficient than the originally envisaged method, and allowed the application to be tested by a larger and more relevant body of users than would have been possible previously. These users will include expert users , whether experienced in learning or teaching BSL, or experienced in using and evaluating multimedia applications. These will be balanced by users who have no expertise in any of these areas, and users who have some experien ce in one of the areas.


5.1.1 Testing Results

The testing was carried out with a number of users, who each raised salient points about the interface and any problems, and possible solutions. Not all of these were acted upon, but there were several which have helped the application become a more rounded and complete experience. The completed questionnaires can be found on the CD which accompanies this report. One of the main points raised in the testing phase of the project was that there was no help facilit y, where the user could look for instruction as to how the application works, and how to perform specific tasks. This is a valid criticism of the application design, and illustrates the benefits of this kind of testing; the project developer has spent a l ot of time looking at, using and designing the application, and many things that seem readily apparent to someone who is so close to the product are not as evident to a user who is unfamiliar with the application. At this stage of the development a help s ection was added, which should allow the user to use the application as the developer intended. It was also decided to direct any newly registered users to the help section before they reach the main dictionary, as this will give the user context specific instructions in how to use the application before they actually start to use it. Other points were raised, mainly about the aesthetic design, and some HCI issues, like the appearance of the Notes section of the page being very similar to the appearance of the video player; these have been addressed at this stage to take advantage of the advice and criticism given. Issues were also raised with the quality of the videos provided in the tested application, but these were not at the quality level of the videos which will be in the finished application. It was also pointed out that, even though the application is a prototype, the lack of a variety of videos meant that the application did not put across the full scope of how it would function. To alleviate these concerns, a second set of prototype videos were created. These encompassed a full topic worth of videos (10 videos), and one phrase. Also included was the full DeafBlind FingerSpelling language. The decision was also taken to remove the duplication of videos for every word included in the prototype; it was intimated by the testing process that this was the main fault in this section of the 51

application. These were replaced by simple videos that stated the name of the word the video referred to, and the words test video . It is hoped that this approach will give any future test users a more comprehensive look at the potential of the application. One element of the application that has been changed as a result of testing is that of screen size. Test subjects were unhappy with the default size of the window that the application runs in, and there was also a bug discovered that upon user logou t, the application fully maximized, eliminating the default red close button. It was decided to take out the default full screen facility. This actually serves to make the application more user friendly, as the user then has the option of enlarging the screen from a smaller window, a choice that was not as readily apparent when the application ran in a full size window on start up. Another issue that was highlighted on more than one occasion was that of the update method; this was anticipated as being a p roblem before testing, and the test subjects were informed of this in the questionnaire they were give. It was still a useful exercise to have the difficulties in the chosen approach highlighted however.


6 Conclusions
6.1 Discussion

This project looked at whether it would be possible to create a multimedia application that would assist the user in learning sign language, specifically British Sign Language (BSL). The obvious answer to the question posed would be Yes , as there are several existing products which claim to do just that. However, as the early research was carried out, it became apparent that focusing on the modern aspect of multimedia would be the most effective strategy for this project. One of the key aspects of the early research was to carry out a critical review of those existing products, which performed a similar task. The research found that there were already several applications available, the challenge that was posed was to create an application that was truly earned the title multimedia, and that improved on the material already available. The more accomplished, and more impressively designed of these existing applications invariably were the ones that cost money to access fully. The application that was created for this projec t had the advantage of being designed to be a freely available resource. The research carried out into the field of multimedia learning informed the design of the application, so that it could best assist anyone using it. Using the various aspects of May er & Sims multimedia theory, the application was designed to be accessible, and to maximise the chances of the user retaining any knowledge acquired from using the product. The research done in this area was invaluable in answering the original question, it helped to show what would be expected from a multimedia learning interface, and how to maximise the inherent potential contained within the concept. British Sign Language, indeed any sign language, is not as straightforward as envisaged when this project was undertaken. The different structure of sign language to that of written or spoken English was a large, unanticipated hurdle. This resulted in the proposed application being altered, as the original plans had included a sentence builder. This had to be left out of the prototype that was built, as it was not feasible to construct a tool of this complexity in the timescale available. 53

After the testing and evaluation stage, where the completed prototype application was subjected to a cognitive walkthrough by a mix of expert, target and inexperienced users, feedback was received which further reinforced the notion that the initial question could now be answered positively. The users mainly agreed that the application was indeed a useful tool, which wo uld assist users in learning sign language. The main reservations that were given as a result of this testing were about the site s aesthetic design, and some possible HCI issues, which have been addressed. One issue that was highlighted was the quality of the videos contained in the application. The feedback received was concerning the videos which were included in the prototype distributed for testing purposes. Comprehensive testing was carried out on the different types of compression which were avail able for the appropriate file formats. The videos which would be included in later versions of the application would be encoded using the knowledge gained from this testing.


6.2 Future Work

Obviously, as a prototype, there have been options which were con sidered during the development of this project, which would not have been possible to implement in the timeframe allotted and with the resources available. Some of these were omitted due to their complexity, and the time it would have taken to implement them successfully. Others were not implemented due to the project developer s relative inexperience with the platforms being used and the skills needed to make them function successfully. These are all ideas or facilities that could be taken and implemented in a version 2 of the application. One such idea which was considered not be viable at this stage of development was that of regional variations. The initial research done showed that although British Sign Language is recognised as a language, like any other language there are regional dialects and variations. A sign may mean one thing in the south of England, but have a different meaning in Manchester, and a different meaning again in Glasgow. Ideally the application would recognise this, and when a w ord is selected the appropriate video would be displayed, relevant to the already established location (or desired location) of the user, who would be offered the other videos as an option. The download system that the application offers is not ideal, in a version 2 there are a few ways this process could be improved. The current method relies heavily on the user having knowledge of file paths, and being able to follow instructions in placing the downloaded material in the correct folder. Obviously this places a lot of trust in the user, and there is plenty of scope for errors. The optimum way to implement the download facility would be to have the process completely automated, so that all the user would have do is to click a button and the desired material will be downloaded and ready to use without any other user action. Another approach which was briefly considered was that of blanket updates; that the user would be able to click an update button and any new material that had been created would be do wnloaded and placed within the application. This process 55

was eschewed in favour of more specifically targeted updates, the idea that the user could select a new topic and download that, rather than have a surplus of new material that could be irrelevant to the user at that time. In future versions however, offering the user the choice of targeted material or the blanket update would be preferable. An initial idea was to incorporate a Deafblind word generator. This would allow the user to enter a word, and the relevant videos would show in the correct order to display to the user how to sign the word in Deafblind (and BSL) FingerSpelling. This was not considered a priority when developing the application, as the initial research showed that this feature had already been implemented by similar products. It would be a useful addition to the application in future versions, but it was felt that the time and effort would be better spent on more innovative features at this stage of the development. The phrasebook section of the application contains complete phrases, and an initial idea was to link the individual words in the dictionary to any relevant phrases that were included in the phrasebook. This was not implemented due to the complexity of sorting each word into its context within a phrase. This is an idea that could be implemented in any future version of the application. One of the initial ideas considered in relation to this application is that of a sentence builder. The idea was that the user would en ter a sentence in plain English, and the application would then construct the sentence in BSL and display the appropriate videos in order. The research done into the structure of BSL quickly intimated that this would be a task beyond the developer at this stage of the applications development. BSL has a very different structure to that of spoken languages, and developing algorithms to account for this and display the sentences in the correct order would certainly have been too ambitious a task to take on at the time. However, this would be a very valuable addition to the application if it could be implemented successfully, and should certainly be considered when developing any future versions of the application.


One smaller aspect of the design that was unable to be implemented was the further filtering of the alphabetical lists in the dictionary section. The idea was to have a drop down list with the letters of the alphabet to filter the selectable words which appeared in the list box. This was impleme nted successfully. Then the list could be further filtered by a second dropdown list, which would have each letter divided into smaller chunks: for example B would be broken into BAA BOM, BON BRA, BRE BEA, BEB BIG, BIN

BUT and so on. The best w ay to implement this seemed to

be to dynamically divide up each letter depending on the number of words which were relevant, but this caused real problems during development, and was subsequently shelved to concentrate on other aspects of the application. This is an idea which could be resurrected in any future development. A facility contained within the application to test the user, dynamically based on the signs they have placed in their personal vocabulary or on topics they have previously learned would be a valuable addition to the application. These would likely take the form of interactive quizzes, but the exact format of these has not been developed as yet.


6.3 Conclusion

The project can, then, be deemed to be a relative success. The research question can be answered positively, and an application was built which corroborates this answer. As was stated earlier in this report, the scale of the project was compromised by unforeseen circumstances in the project developer s personal life. This forced the project to be put on hiatus for a period of 8 weeks, which did have an impact, especially on the research phase of the project. Despite this, it is felt that the project has achieved all of the aims that were set out at the start, and can be viewed as a success. The initial aims were stated as: y To research the multimedia material currently available for learning BSL (and other forms of sign language) y y To research multimedia learning techniques To apply the conclusions reached by this research and des ign a learning application that improves on the material currently available y To produce a visual, interactive dictionary to assist in the learning of BSL

All of these initial aims have been achieved. The material available online was critically reviewed. Useful information about the structure of such applications was gained, as well as a number of pointers as to how not to proceed. This research, in tandem with the research done into how multimedia techniques can be best employed to create an efficient and effective learning environment, culminated in the creation of a learning tool that, in its prototype stage is a viable product. If this product was to be developed further, it is felt that this would be a valuable addition to the field of learning sign language, and would improve on the material which is currently available, which was another of the initial aims of the project.


7 References

Atkinson, R. C. & Shiffrin, R. M. (1971), The Control Processes of Short Term Memory Scientific American, 225(2), 82-90 Baddeley, Alan. (2003), Working Memory: Looking Back and Looking Forwards, Nature Reviews Neuroscience, 4(10), 829-839 Baron, Naomi S. (1981), Speech, Writing and Sign (A Functional View of Linguistic Representation), Bloomington: Indiana University Press Bowden, Richard et al. (2004), A Linguistic Feature Vector for the Visual Interpretation of Sign Language. CVSSP, School of EPS, University of Surrey, Guildford, Surrey, UK , 1-12

Crossman, E. R. F. W. & Goodeve P. J. (1983) , Feedback control of hand -movement and Fitts' law. The Quarterly Journal of Experimental Psychology Section A , 35(2), 251 - 278 Deafworks . (1999), Improve your BSL Deafworks, Clerkenwell, London Jacko, Julie & Sears, Andrew. (2009), Human Computer Interaction . CRC Press, Florida USA Kadir, Timor et al. (2004), Minimal Training, Large Lexicon,Unconstrained Sign Language Recognition. 1-10, CVSSP, University of Surrey, Guildford, UK Kyle R.G. et al. (2000), Sign language: the study of deaf people and their language, Cambridge University Press Macsweeney, Mairead et al. (2002), Neural systems underlying British Sign Language and audio-visual English processing in native users. Brain, 125(7), 1-11 Magill, Richard & Hodgson, Anne, (2003), Start To Sign. RNID, London


Mayer, Richard E. (2003), The promise of multimedia learning: using the same instructional design methods across different media, Learning and Instruction , 13(2), 125-139 Mayer, Richard E. (2001), Multimedia Learning. Cambridge University Press, Cambridge Mayer, Richard E. (1995), The Cambridge Book of Multimedia Learning. Cambridge University Press, Cambridge Mayer, Richard E. & Sims, Valerie K. (1994), For Whom Is a Picture Worth a Thousand Words? Extensions of a Dual -Coding Theory of Multimedia Learning Journal of Educational Psychology , 86(3), 389-401 Mayer, Richard E. & Moreno, Roxana. (2005), A Cognitive Theory of Multimedia Learning: Implications for Design Principles Multimedia Learning, 1-9 Miles, Dorothy. (1988), British Sign Language London Paivio , Allan & Clark , James M. (1988), Bilingual Dual-Coding Theory and Semantic Repetition Effects on Recall. Journal of Experimental Psychology: Learning, Memory, and Cognition , 163-172 Schlesinger , I.M & Namir, Lila. (1978), Sign Language of the Deaf ( Psychological, Linguistic and Sociological Perspectives). Academic Press, London Siple ,Patricia. (1978), Understanding Language through Sign Language Research. Academic Press, London Smith, Cath. (2004), Signs Make Sense. Souvenir Press, London Smith, Cath. (2005), Let s Sign Dictionary. Co-Sign Communication, Stockton-on-Tees Sutton-Spence, Rachel & Woll, Bencie. (1999), The Linguistics of British Sign Language - An Introduction. Cambridge University Press, Cambridge A Beginners Guide, BBC books,


Sweller, John. (1994), Cognitive Load Theory, Learning Difficulty and Instructional Design. Learning & Instruction, 4(4), 295-312 Szabo, Michael & Kanuka, Heather. (1999), Effects of violating screen design principles of balance, unity, and focus on recall learning, st udy time, and completion rates. 1999, Journal of Educational Multimedia and Hypermedia, 8(1), 23-42. Charlottesville, VA: AACE

7.1 Web References

ASLPro. (2005). ASLPro Main Dictionary. Available: Last accessed 30 October 2009. (2010). A Guide to British Sign Language. Available: Last accessed 29 October 2009. British-Sign. (2008). Introducing British Sign Language - Online Course. Available: -british-signlanguage-online-course. Last accessed 21 October 2009. Deafsign. (2009). How To Learn BSL. Available: Last accessed 1 November 2009. Stories In The Air. (2006). Stories In The Air. Available:,_BSL_Dictionary_learn_online. html. Last accessed 29 October 2009. National Deaf Children s Society. (20 09). About NDCS. Available: l_project.html. Last accessed 29 January 2010.


Signing Online. (2008). Signing Online - Learning American Sign Language. Available: Last accessed 30 October 2009. Trickey, George & Lewis, Keziah. (2006). Art of vision - featuring signed videos in BSL. Available: Last accessed 25 October 2009.