Shorthand Writing on Stylus Keyboard

Shumin Zhai IBM Almaden Research Center 650 Harry Road, San Jose, CA, USA
zhai@us.ibm.com
ABSTRACT

Per-Ola Kristensson Department of Computer & Information Science Linköping University, S-581 83, Sweden
perkr@ida.liu.se
Graffiti in the Palm Pilot and Jot in Windows CE, have been used in commercial products. The fundamental weakness of handwriting as a text entry method, however, is its limited speed, typically around 15 wpm [6]. Such a speed is good enough for entering names and phone numbers on a PDA, but too limited for writing longer text. Virtual keyboards, tapped serially with a stylus, are also available in commercial products, typically in the familiar QWERTY layout. To improve movement efficiency, stylus keyboard layout can be optimized either by trial and error [15] or algorithmically [25]. Depending on the degree of optimization, the expert text entry speed with stylus keyboard can be more than 45 wpm on layouts such as ATOMIK (Fig. 1, see [26] also [15]). There are also weaknesses to stylus keyboarding. The simple tapping movement may feel tedious to repeat for prolonged use. Stylus keyboarding also requires intense visual attention, virtually at every key tap, which prevents the user from focusing attention on text output. Our current work began with two observations in stylus keyboard research. First, it has been noted that some words or fragments of words are connected in stylus keyboards. With these words, such as the in Fig 1, it is possible to stroke through the keys rather than tapping on them individually, hence introducing a form of more fluid movement closer to drawing [25]. Second, for a wellpracticed word, users tend to remember its pattern, the trajectory of stylus movement from key to key as an integrated chunk, rather than individual key taps [25]. These two observations led us to imagine letting the user directly write patterns as a basic mode of entering words on a stylus keyboard. Each pattern of a word is formed by the trajectory from the first to the last letter of the word on the keyboard. Fig. 2 shows a few examples of such patterns defined by the ATOMIK keyboard in Fig. 1.

We propose a method for computer-based speed writing, SHARK (shorthand aided rapid keyboarding), which augments stylus keyboarding with shorthand gesturing. SHARK defines a shorthand symbol for each word according to its movement pattern on an optimized stylus keyboard. The key principles for the SHARK design include high efficiency stemmed from layout optimization, duality of gesturing and stylus tapping, scale and location independent writing, Zipf’s law, and skill transfer from tapping to shorthand writing due to pattern consistency. We developed a SHARK system based on a classic handwriting recognition algorithm. A user study demonstrated the feasibility of the SHARK method.
Keywords

Text input, shorthand, gesture, stylus keyboard, virtual keyboard, pervasive, mobile, handheld devices, human memory, learning, skill acquisition.
INTRODUCTION

Text input - ranging from writing emails, filling forms, typing commands, taking notes, to authoring articles and coding programs - constitutes one of the most frequent computer user tasks. The QWERTY keyboard, for various reasons, has been accepted as the standard tool to accomplish this task for desktop computing (see [25] for a brief review). The emergence of handheld and other forms of pervasive or mobile computing devices, however, calls for alternative solutions. Consequently text input has been revived as a critical HCI research topic in recent years. There have been various methods proposed, developed or studied (See [5, 14] for surveys). The two classes of solutions that have attracted most attention are handwriting and stylus based virtual keyboarding. Handwriting is a rather “natural” and fluid mode of text entry, thanks to users’ prior experience from writing on paper. Various handwriting recognition systems, such as

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2003, April 5–10, 2003, Ft. Lauderdale, Florida, USA. Copyright 2003 ACM 1-58113-630-7/03/0004…$5.00.

b k d c a n i m f l e s x h t o v r u w z
Fig. 1 The ATOMIK Keyboard (adapted from [25] by permission)

visual attention as serially tapping all the letters. however. word frequency in a language tends to follow Zipf’s law 2 . the user can write their shorthand. the 100 most common individual words make up 46% of the entire British National Corpus (http://www. causing a general tendency that letters from A to Z run from the upper left corner to the lower right corner of the keyboard. Efficiency Traditional shorthand writing systems. However if we define it on the familiar QWERTY layout. In theory such form of shorthand can be defined on any keyboard layout. it maximizes. for example. with a highly skewed distribution (Fig. The benefit of using shorthand for a small set of common words is disproportionably large. this can be a form of shorthand writing. See Fig. This gives helps users look for letters that are not yet memorized. A shorthand system defined on a stylus keyboard. the time to perform such visually guided steering cannot be expected to be any shorter than tapping1. . because the commonly used consecutive keys are deliberately arranged on the opposite sides of QWERTY (See [24] for a long and [25] for a short review on this issue). the users could produce these patterns with much less visual attention and presumably greater ease and comfort. 3). Therefore. To precisely cross all letters defining a word would require just as much.the word have Fig. 2 Word patterns defined by ATOMIK Keyboard (a dot indicates the starting end) MOTIVATING PRINCIPLES Scale and location independency This means that in any given scale. is a power-law function f ~ 1/ia with the exponent a close to unity. Scale and location independency is the first principle in our current work. 3 Word frequency distribution in BNC Transition from tapping to gesturing Finally. for gesturing to be effective. Duality Is there any advantage to pattern gesturing over individual letter tapping? The answer depends on how the gesture is produced and recognized. The word the alone constitutes over 6% of the BNC. if not more. In comparison to hand writing based on alphabetic or logographic characters such as Chinese. For the less familiar. Furthermore. takes significant time and effort to master. ATOMIK (Alphabetically Tuned and Optimized Mobile Interface Keyboard) was optimized by a Metropolis algorithm in which the keyboard was treated as a "molecule" and each key as an "atom". which takes hundreds of hours of practice to be proficient [7]. The atomic interactions among all of the keys drove the movement efficiency . Furthermore. 2 and Table 1 for examples of shorthand symbols defined on ATOMIK (Fig 1). If so. Zipf’s law effect The second principle of the current work lies in efficiency. 2 Zipf’s law models the observation that frequency of occurrence of event f. without sacrificing the first two features. the letter connectivity of the most common words [25].org). the average word gesturing length on ATOMIK is also minimized. weighted by the statistical frequency of the corresponding pair of letters . This means that a relatively small set of shorthand gestures can cover a large percentage of text input. 1 Fourth.bnc. For familiar words whose patterns are well remembered. which may facilitate skill transfer between the two modes. With the exception of touch-typing on physical keyboards. with each letter constituting only one straight stroke and with the entire word as one shape. does not have to contain a complete or even a large set of words. Gesturing and tapping a word with SHARK share a common movement pattern. In other words. writing a word pattern defined by a stylus keyboard can be much more efficient. users are usually reluctant to invest time in learning a human computer interaction skill. As long as the user produces a pattern that matches the shape of the prototype of a word. Both modes of typing are conduced on the same input surface and the system distinguishes tapping from stroking and gives output accordingly. one can use stylus tapping.towards the minimum. For example. ATOMIK is also alphabetically tuned. We choose the ATOMIK layout for the current work. as a function of its rank i. such as Pitman’s. it would involve frequent left-right zigzag strokes. because one can use both tapping and shorthand gesturing. Frequency of English Words (%) 8 6 4 2 0 Rank 1 11 21 31 41 51 61 71 81 91 Fig. We call this combination method SHARK shorthand aided rapid (stylus) keyboarding. a user’s repertoire of shorthand symbols can be gradually expanded with practice. It is possible to use the law of steering [1] and Fitts’ law of pointing to quantitatively analyze the difference between tapping and tracing. the system should recognize and type the corresponding word for the user. patterns must be recognized at least partially independent of scale and location.defined by the summation of all movement times between every pair of keys.

two distinct modes of operation for novice and expert respectively. A letter is selected when it passes through the cursor. which reinforces the pattern and pushes the user back to expert mode. For stylus keyboard optimization. RELATED WORK Cirrin (Circular Input). For other input methods. A weakness of the T-Cube is that it works at alphabet level and hence could not be very fast. The idea of bridging novice and expert modes of use by common movement pattern in SHARK was inspired by Kurtenbach. We have outlined five motivations and rationales of using SHARK as a new approach to text entry. In what follows we first briefly point out previous work related to SHARK. Dasher demands user’s visual attention to dynamically react to the changing layout. 3 The SHARK system may count the frequency of each word used. The idea of optimizing gesture for speed is embodied in the Unistrokes alphabet designed by Goldberg and Richardson [9]. a SHARK shorthand pattern can be optionally displayed on the keyboard (by connecting the letters with dotted line. T-Cube defines an alphabet set by cascaded pie menus. Many The need of entering text on mobile devices has driven numerous inventions in text entry in recent years. sometimes crossing to another zone. For an expert whose action is fast. syntactical modeling. is probably the closest prior art to SHARK. While it is possible to develop “iconic gestures” for common words like the. is the nature of writing one letter at a time. If mastered. however. When a word is tapped enough times. In Unistrokes every letter is written with a single stroke but the more frequent ones are assigned with simpler strokes. visually guided tapping is easier. Cirrin explicitly attempts to operate on a word level – the pen lifts up at the end of each word. 11]. one can potentially write faster in the Unistrokes alphabet than in Roman alphabet. are not in Cirrin. We then switch to the human side by presenting an experiment that tested users’ ability to learn and recall SHARK gestures. marking menu “reveals” itself by displaying the menu layout. 11]. is at the expense of visual attention. Dasher dynamically arranges letters in multiple columns. and constitutes the fifth principle of SHARK.For a novice user. location and scale independency. A novice enters characters by following the visual guidance of menus. which is an important foundation of SHARK. A self-revealing menu approach has also been explored in text entry. The fundamental limitation of Unistrokes. Particularly innovative in marking menu is the use of delayed feedback. Instead of having pull-down menus and shortcut keys . The stylus trajectory determines which letter is selected. 14. and finally returning to the center zone. such gestures are rather complex due to the fact that the stylus has to return to the center after every letter. 25]. is another approach of continuous gesture input. In this short review we focus on continuous gesture-based text input. designed by Mankoff and Abowd [16]. Each character is entered by moving the stylus from the center of the radial layout to one of the eight outer zones. We will conclude the paper with a discussion of future work. pattern recognition techniques have been previously invented for “online” handwriting recognition. whose movement is hence minimized. If a shorthand gesture is forgotten. Quikwriting designed by Perlin [17] uses continuous stylus movement on a radial layout to enter letters. some of the key bases of SHARK such as open-loop pattern production rather than crossing individual keys. a marking menu uses the same directional gesture on a pie menu for both types of users. However. The user draws a word by moving the stylus through the letters. their system does not display visual guidance at all so the user’s actions become open loop marks. while an expert user could enter the individual characters by making menu gestures without visual display. We then describe the design and implementation of a SHARK system to demonstrate that SHARK is practically feasible based on current pattern recognition technology. combination with and transition from stylus tapping. for example) suggesting that the user switch to shorthand for that word. Buxton and colleagues’ work on marking menu [10. one can fall back to taping. We have developed a SHARK recognition system based on the classic elastic matching algorithm [20] which computes the minimum distance . Cirrin also attempts to optimize pen movement by arranging the most common letters closer to each other. Because the letter arrangement constantly changes. Dasher. Quikwriting is fundamentally a character entry method. Blackwell and MacKay [23]. Cirrin operates on letters laid out on a circle. with likely target letters closer to user’s cursor based on the proceeding context. statistical modeling and neural networks (see [21] or [4] for surveys of the common techniques). although the majority has not been properly researched with either theoretical or empirical human performance studies. see [5. In this sense. This gradual and smooth transition from novice to expert behavior can be very similar to the process of mastering marking menus [10. the user may switch to the more fluid “expert” mode of shorthand gesturing 3 . SHARK GESTURE RECOGNITION This section deals with SHARK gesture recognition. The same movement gesture affords smooth transition from novice to expert. readers are referred to [25] and also [15]. This minimization. however. by Ward. For a novice whose action is slow and needs visual guidance. When beyond certain threshold. in the T-Cube method designed by Venolia and Neiberg [22]. as was shown by the authors [22]. including template matching.

a right horizontal stroke followed by a stroke to the upperleft direction will always be can. One is the use of transient pie menus. for example. an empirical study is necessary. Further study is needed to find the best ambiguity resolution method for SHARK. on the other hand. Arabic numbers. However. if the user makes a right horizontal stroke closer to the upper left region of the keyboard where c-a-n are (Fig. In sum. there are words that share the same SHARK gesture. although experienced users may transfer common traces between some words. Trained stenographers learn a great number of shorthand symbols. filtering and normalization in scale. The algorithm and techniques used in our system can all be found in the previous literature [2. The second approach we designed maintains efficiency but sacrifices the location independence principle for the ambiguous words. the distance between the unknown shape and the prototypes are computed by elastic matching. Based on these operations. to determine which word the user intends to write. We use dozens of Roman letters. we focus on the most basic question . and so on. Some words that appear ambiguous offline are not in fact ambiguous. if further to the right. To completely address these questions requires extensive long-term research. An experienced user may simply remember the second stroke as part of the shorthand for that word. Fig. there are ample evidence that people can master a large number of symbols. The other is from a prototype – the ideal shape defined by the letter key positions of a word. punctuation marks. 700 separate hieroglyph symbols were used in ancient Egypt [8]. to. is also a horizontal stroke but it will never be confused with the word can. After preprocessing. Greek letters. Note that since the recognition is online (real time). and produce shorthand gestures defined on stylus keyboard at all? Can they learn a useful number of SHARK gestures in a relatively short period? There are reasons to expect that people can remember a large number of symbols. remember or discriminate (see [13] for a recent study on gesture similarity). 20. if closer to the lower middle region. although this is still less stringent than tapping. to understand people’s ability to learn SHARK. 4. it pops up a pie menu with all the candidates (usually only two or three) in a consistent order. We have designed two approaches to resolve ambiguities. the system gives the user can. As shown in Fig. SHARK gestures (See Table 1 for examples). One disadvantage to this approach is the loss of efficiency due to the added second stroke. One set of points is from the shape that a user produces (a unknown shape). because the candidates are presented in a consistent location of the pie menu and the selection of choice depends on direction only. It is beyond the space limit and the scope of this paper to explore the optimal recognition technology for SHARK. A USER STUDY Issues to be explored The SHARK method raises many human performance questions. For example. are formed in a novel approach that does not break a continuous shape down to elements. The system checks the start position. it is necessary to discuss a special aspect of SHARK recognition – ambiguity handling. Although in low percentage. we have written a SHARK notepad application with dual modes of input . the word can. 4. The shape of a SHARK gesture is not always unique and hence creates ambiguity. when the recognition system found more than one match to a sample. and to are completely identical if we do not consider scale and location. 1). Similarly left and down is always to and so on. At this early stage. and mathematic symbols. 21]. A literate Chinese person must learn two to five thousands of unique characters. such as -ing and -tion. For example. Resolving ambiguity by a transient pie menu . The same is true to do and no (see Fig.shorthand gesturing and stylus tapping. an. an. The word in.can users learn. even Chinese characters are made of common radicals. However.between two sets of points by dynamic programming. As shown in Fig. the user does not have to look at the menu. The weakness of the partial location dependency method is that it requires more visual attention when writing the words with ambiguity in order to make sure the location is closer to the intended word on the keyboard. or the geometric center of the sample. 1). With experience. A user inexperienced with this particular ambiguous word would look at the menu and make the second stroke in the direction of the candidate intended. for example. The word corresponding to the prototype that has the shortest distance to the sample is returned as the recognized word. particularly for some short words. the system takes the direction of the SHARK gestures from pen down to pen up into account. It is likely that in people’s memory Chinese characters are reduced to these radicals and the two dimensional relationships among the radicals. independent of location. Clearly. relative to the letters defining the multiple ambiguous prototypes. 4.

If correct. The user could write the SHARK gesture anywhere on the keyboard in any scale. the rehearsal interval will keep its current value. models and insights on how people learn and remember skills. Briefly. Go to 2. the ERI method suggests that trial repetitions for learning should be neither totally massed nor randomly distributed. Research has shown that active retrieval is a key to memory retention [19]. The user may go on to the next word or practice the current word a few more times with the choice of seeing the keyboard layout by clicking show keyboard. two male four female. 6). None of them had any prior experience with SHARK or the . between 20 and 30 years old. 6. We could also make the interval shrink if the learner repeatedly fails to recall a gesture. Rather. If the rehearse list is empty. 4. 2. Otherwise. Different from the original ERI method which schedules rehearsal at increasing but preprogrammed intervals. [18]) and human memory (e. The user may learn a SHARK gesture by displaying the ATOMIK keyboard layout The learning program maintained two lists of words: a word list containing the maximum number of words to be learned in the study and a rehearse list keeping all words being actively rehearsed at various intervals. The literature on skill acquisition (e. and practice the gesture as many times as wanted to Fig. The interval of rehearsal for a particular SHARK gesture expands only if the learner could recall the gesture correctly. The secondary goal of this user study is to design an effective method to help users learn SHARK gestures. 5. 3. This forced the user to actively retrieve SHARK movement patterns from memory. If the timer is below 30 sec present the word to the user. Otherwise pick a new word from the word list and present it to the user. Each word in the rehearse list had its own timer. 5). they should be optimized by systematically increasing the interval between repetitions.How well people can learn new skills partly depends on the methods they learn and practice the new skill with. although the rate of expansion and the unit of training in stylus keyboarding needed further investigation. else return it with an unaltered rehearsal interval. Learning method explore the tolerance of acceptable shapes for the target word. The word that matched the user’s SHARK gesture was then displayed in the last entry area (Fig. The algorithm that managed the ERI scheduling worked as follows: 1.g. [3]) presents a variety of theories. the interval stays the same. Pick the word from the rehearse list that has the earliest rehearse time according to the value left in its timer. A compelling method from that literature that may be relevant and useful in helping people to learn SHARK is practicing with expanding rehearsal interval (ERI) [12]. participated in a five-session experiment. the word would be rescheduled to appear at an interval twice the current value. The user could choose to learn (for a new word) or relearn (for a forgotten word) the SHARK gesture by displaying the ATOMIK keyboard. Experimental set up and procedure The experiment was carried out on a PC attached with a Wacom tablet model ET-0405-U as the pen input device. the participant was asked to write the SHARK shorthand for that word on the stylus keyboard area that did not show the layout (Fig. ERI has been acclaimed as an important result in human memory research and is also supported by recent thoughts in the field of skill acquisition and memory [19]. Screenshot of the experimental set-up: the user was asked to write a word (“inside”) by SHARK without visual reference Fig. pick a new word from the word list and initialize it with a rehearse interval of 30 sec and put it in the rehearse list. Six paid volunteers. before moving on. In each cycle of rehearsal of a particular word by our ERI method. return it to the rehearse list with a doubled rehearsal interval. If the participant could not recall the SHARK gesture of a practiced word or failed to write it correctly. counting down from its current rehearsal interval value.g. we decided to make ERI adaptive based on the learner’s performance. If the user writes the correct SHARK gesture of the target word. together with the gesture prototype drawn in dotted lines connecting the letters in the word (Fig. Recently it has been shown that such a method could be effective to learning stylus keyboarding [26]. but our experience shows that this is not necessary in practice and may lead to frustration. 6).

Note that all these traces ran across different days (sessions). Results and discussion local member within always follow without during bring although example question The results of the user study shows that all of the participants could learn to write correctly recognizable SHARK gesture for any word presented to them. Session 1 was 40 minutes practice with ERI only. Note also symbols listed in the table are SHARK prototypes. with rounded corners for example. In the experiment a total of 100 words were in the word list. Fig. Part one was a test session of words a participant had practiced in previous sessions. A solid dot signifies the start point of a SHARK symbol. Memory Rehea rsa l Inte rva l (minute s) 70 60 50 40 30 20 does ot her t hose was Due to space constraint these SHARK symbols can only be listed in small size. and were suggested to take short breaks anytime they liked to. whereas others found the frequency of practicing adequate. the last session. was a final test session only. Sample ERI traces . if practiced enough times (typically 7 to 15 ERI cycles). 7. The participants were given at most two chances to recall and write the SHARK gesture correctly. lasting from 6 to 20 minutes. These words and their corresponding SHARK shorthand prototypes are listed in Table 1. suggesting the expansion rate (factor of 2) too aggressive. and sometimes a full weekend between a test and the previous practice session. Some participants could keep up with the ERI expansion and correctly write the SHARK gesture every time (See other in Fig. We focused on the baseline of SHARK shorthand memory in this experiment and made sure no ambiguity existed in the list. Fig. The next 50 words were selected from the top 100 to top 300 most common words.Table 1. In test sessions. it is possible to make the expansion rate adaptive to individuals based on their performance. The static shapes do not reveal the dynamic stroke directions. which make them more distinct. Words in the study and their SHARK shorthand4 the and in inside have has had having he him his it its they them was their not for you your she her with on that this these those did does done doing are our from which will were said can whose went gone other another being seeing knew knowing about could think people after right because between before through place become such change point system group number however again world course company while problem against service never house down school report start country really provide ATOMIK keyboard. but it could certainly be further tuned. the ERI method appeared effective. and 4 consisted of two parts. 4 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Prac tic e No. 3. Participants were asked to take a 5minute break in the middle of the practice session. 7 for example). 50 of the 100 words used were selected from the top 100 most common words in the British National Corpus. Session 2. Actual user’s sample shapes always deviate from the prototypes. 7 shows the ERI traces of a few sample words by one participant. Part two was 40 minutes practice with ERI. Some participants felt that new words disappeared too quickly. In sum. There was a period of minimum one day (night). words were presented in a random order. Session 5. For example.

and innovate many aspects of it. High-level constraints. and 58. Most of them found the SHARK method exciting. Two participants mentioned “it was easier to remember complex shapes than simple ones”. it is difficult to find precedence of human capability in this regard. much can be done in making the recognition more sophisticated. 50 to 60 shorthand gestures learned in four sessions can already be very useful in text input. Given the prospect of creating a whole new system of efficient writing. 70 60 50 40 30 20 10 0 1 typing sometimes”. on average they correctly produced 48. The study showed no sign of slowing down in user’s ability to learn more SHARK shorthand symbols. accuracy. preferably integrated in the SHARK system and combined with the use of it. long term study of users’ learning and usage of SHARK. 8 and 9. which is probably more topological. particularly in the initial sessions (“It is fun!” “Ought to be a really effective way of writing once you get a hang of it” etc). 9). Participants occasionally drew mirror image of the correct SHARK shape. to replace keyboard typing sometimes”. In general. not conclusions. More words learned in each session During the study participants were encouraged to write down comments on SHARK and the learning method used each day. Because these symbols are constructed differently from other large character sets such as Chinese. 2/6 answered “Yes. these subjective data can only be taken as a reference. given the Zipf’s law effect. Total number of words correctly written in test sessions. How many such symbols can eventually be learned. 8. is unknown. if limited at all. when a physical keyboard is not available” and 4/6 “Yes. human longterm memory capacity is only limited by acquisition speed. and 0/6 “Yes. When asked if they would use it in a list of situations. +3 stimulating). the number of new words learned per session was rather constant. a great deal more research is needed in the future to understand. +3 definitely yes). Total number of words learned First attempt only First or second attempt 2 3 4 Testing session Fig. such as the context of proceeding words. participants rated 0. can also potentially be used to make SHARK recognition more tolerant to “sloppy” writing. +3 easy) to learn.6 on the scale of (–3 definitely no. The participants were usually displeased when they wrote a word correctly in their mind. participants were able to correctly write more words in each learning session. on average about 15 words more per session. from both human memory and motor control. averaged across participants Mean and Standard Deviation of the number of new words learned per session 18 16 14 12 10 8 6 4 2 0 1 2 3 Testing session 4 Fig. 1. Interestingly. contrary to our concern that long words will be difficult to handle with SHARK. which emphasizes the degree of proportion. It suggests that our recognizer based on a standard elastic matching algorithm. to replace keyboard typing all the time”.As shown in Fig. or slightly higher toward later sessions (Fig.83 (between 62 and 39) words in their first attempt. particularly in real use of text production. 0. User’s initial reaction is often the most critical to the adoption of an interface style. modeling and empirical measurement of SHARK gestures-based input in terms of speed. validate/invalidate.6 on (–3 difficult.67 (between 77 and 49) words if counting the second attempt when the first failed. and improvement of learning method. the usability of various ambiguity resolution methods. improve. We have only done the most basic work to show the feasibility of SHARK. 9. None of the participants found the method tedious throughout the study. change. with situation specific “work-around” and probably borrowing techniques from modern complex speech recognition systems.6 on the scale of (–3 frustrating.6 on (–3 difficult. In the final questionnaire. 0. Since there is no direct comparison to methods with similar amount of experience and the study was short and necessarily artificial. +3 satisfying). To the question “If such a method is made available for practical use. and capacity. Other compelling research issues include its expert speed limit.83 on (–3 dull. would you learn it?”. theoretical quantification. In terms of developing more advanced recognition algorithm that matches human perception of shapes more closely in the future. as well as machine recognition point of view. but the system was “unfair” and did not recognize it. In the final test. did not necessarily accurately reflect user’s cognitive model of the SHARK gestures. +3 easy) to use. It is particularly encouraging that the majority could imagine using SHARK even to “replace keyboard . 0/6 participant choose “not at all”. their answer averaged 1. although practice eventually overcame such types of error. Two participants attempted to memorize the ATOMIK layout and tried to guess the SHARK gesture based on their memory of the layout when asked to write a word the first time. Some of the anecdotal comments were also informative.

and J. Odaka.W. 16. Michiels.. Skill acquisition and human performance. 1995. Proc. T. A. Cooper. Bjork. 1993. and T. The 1st Annual Conference on Technological Advancements in Developing countries. It was evident that people could learn a useful number of SHARK shorthand symbols in a rather small amount of time. A user study indicated the feasibility of SHARK. Visual similarity of pen gestures. Zhai. 258-264. S. REFERENCES 1. Tech. H. Accot. Academic Press: London. An empirical evaluation of some articulatory and cognitive aspects of "marking menus". INTERCHI. Proc. Beyond Fitts' Law: Models for Trajectory-Based HCI Tasks. 18. IEEE Transactions on Pattern Analysis and Machine Intelligence.S. Hillsdale. S. 80-87. 12. p. Proc. 3. H.A. Zhang. Tappert. 17.147-198. New Jersey: Lawrence Erlbaum Associates Publishers. Kanji. H. C. Long. 13. 2002. ACM UIST. Proctor. 10. Columbia University. Note. User Learning and Performance with Marking Menus. p.M. Buxton. 4. J. 1-23. Nov 1978. Human-Computer Interaction. G. MacKenzie. R..E. 129-136. Paul Maglio. Soukoreff. p.A.S. ACM UIST. C. Human Memory . p. Mankoff. by practicing with the ERI paradigm. C. Blackwell. Newell. Human-Computer Interaction. The key rationale for designing SHARK rests on five principles: efficiency. p. S. Hunter. and R. Cognitive aspects of skilled typewriting. Proc. 12(8).214. 2(4): p.A data entry interface using continuous gesture and language models.K.229-270. and F. Zhai. I. 20. and A. 4th International Joint Conference on Pattern Recognition. Baddeley.com/inputManuscript. CHI2000. We have developed a SHARK recognizer based on an elastic matching recognition algorithm. 2002.216.. A historical study of typewriters and typing methods: from the position of planning Japanese parallels. D. Part of this work was conducted when Shumin Zhai was a guest professor at Linköping University. Maria Holmqvist and Pernilla Qvarfordt for their assistance. p. An overview of handwriting recognition. 24. Proc. 8(1): p. Masuda. and G. Perlin. 1994/2002: available at http://www. Dasher . Proc. Buxton. 25-31. 9. 2000. 1980. and transition from tapping to gesturing. 1982.. Thousand Oaks.C. 1992. Accot. duality of tapping and gesturing. Quikwriting: Continuous Stylus-based Text Entry. Kurtenbach. 23. 25. 1990. Proc. Hiragana.D. T. L. location and scale independency.. Landay. Suen. Springer-Verlag: New York. Text entry for mobile computing: Models and methods. 14. and W. 6.A.C. Participants could learn SHARK shorthand symbols.K. I. Movement Model. 175-202. Rowe. The design and evaluation of a high-performance soft keyboard.. Dutta. requires a great deal more research in the future. Zipf’s law effect. Hits Distribution and Learning in Virtual Keyboarding. W. CHI.. Revised Edition.A. 215 . B. Landauer. and M. p. 213 . MacKenzie. Proc... and R. Neiberg. W. Techniques and Technology (book manuscript).. We thank Santa Anna IT Research Institute AB and Sture Hägglund for support. 1999. Arakawa.A. Goldberg. in Practical Aspects of Memory. T-Cube: A Fast. R. 22. Proc. Zhai. Sue.X. Performance Optimization of Virtual Keyboards. 30-46. G. ACKNOWLEDGMENTS Gardiner. 17(2&3). Proc. natural writing is too slow and stylus keyboarding is tedious and visually demanding. and considered even to use it to replace physical typing sometimes. Moran. 756-771. Bjork.Y. The Psychology of Human-Computer Interaction. and A.S. 265 270.Theory and Practice. The State of the Art in On-Line Handwriting Recognition. Human Computer Interaction. Smith. p. Schmidt. 7. 1978: Aris & Phillips. 1998. p. To write a significant amount of text. 1998. Boston: Allyn and Bacon. J.. 2. J. A. and J. Richardson. and S. 19. Beigi. 207-217. We also thank Jingtao Wang. To understand all issues involved and the full potential SHARK. Sellen. at a average rate of 15 words per (45 to 50 minutes) session. A. 295-302. 1983. On-line recognition of handwritten characters – Alphanumerics. Ward. . p. Venolia. 15. Katakana. Human Input to Computer Systems: Theories. Cursive Script Recognition by Elastic Matching. p.. CHI. Yamada. and D. SelfDisclosing Pen-Based Alphabet. 21. Most participants found SHARK shorthand fun to use. ACM UIST. 1983.A. CA: Sage. K. 11. Cirrin: a word-level unistroke keyboard for pen input. and W. Speed writing is an old research topic with renewed interest due to the need of using mobile computing devices. K. D. 3(4): p.C. ed. 1993. IBM Journal of Research & Development. 26.P.SHARK. MacKay. 5. possibly even beyond mobile computing as a form of speed writing. CHI. 1993. 1994. pp.17-24. I. 1998: p. Abowd. Amsterdam. Psychological Science. D. We have proposed a new approach that combines the two at a shorthand level .CONCLUSIONS 8. Note.billbuxton. Wakahara.. 17(2&3). 625-632. A. CHI2002. A. Buxton. .html Card. 26(6): p. The conceptualizations of practice: common principles in three paradigms suggest new concepts for training. Egyptian Grammar: Being an Introduction to the Study of Hieroglyphs (3rd edition). and S. Optimum rehearsal patterns and name learning. Tech. Touching-typing with a stylus.W. p. CHI'97. 1978. 810 – 812. Proc. theory and practice. Kurtenbach. SHARK defines a shorthand symbol for each word according to its movement pattern on a stylus keyboard layout. Journal of Information Processing. Proc. and C. 1994. and R. Tappert.

Sign up to vote on this title
UsefulNot useful