Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Jun Rekimoto " Pick-and-drop: a direct manipulation technique for multiple computer environments" (ACM UIST 1997)

Jun Rekimoto " Pick-and-drop: a direct manipulation technique for multiple computer environments" (ACM UIST 1997)

Ratings: (0)|Views: 4 |Likes:
Published by Jun Rekimoto

More info:

Published by: Jun Rekimoto on Oct 20, 2012
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Pick-and-Drop: A [direct Manipulation Techniquefor Multiple Computer Environments
Jun Rekimoto
Sony Computer Science Laboratory Inc.3-1413 Higashigotanda, Shinagawa-ku,Tokyo 141 Japan+81-3-5447-4380rekimoto@csl.sony.co.jphttp://www.csl.sony.co.jp/person/rekimoto.html
This paper proposes a new field of user interfaces calledmulti-computer direct manipulation and presents a pen-based direct manipulation technique that can be usedfor data transfer between different computers as well
as within
same computer.
The proposed Pick-and-Drop allows a user to pick up an object on a display anddrop it on another display as if he/she were manipulat-ing a physical object. Even though the pen itself doesnot have storage capabilities, a combination of Pen-IDand the pen manager on the network provides the il-lusion that the pen can physically pick up and movea computer object. Based on this concept, we havebuilt several experimental applications using palm-sized,desk-top, and wall-sized pen computers. We also con-sidered the importance of physical artifacts in designinguser interfaces in a future computing environment.
direct manipulation, graphical user
terfaces, input devices, stylus interfaces, pen interfaces,drag-and-drop, multi-computer user interfaces, ubiqui-tous computing, computer augmented environments
In 8. ubiquitous computing (UbiComp) environment [18],we no longer use a single computer to perform tasks.Instead, many of our daily activities including discus-sion, documentation, and meetings will be supported bythe combination of many (and often different kinds of)computers. Combinations of computers will be quite dy-namic and heterogeneous; one may use a personal digitalassistant (PDA) as a remote commander for a wall-sizedcomputer in an presentation room, others might want touse two computers on the same desktop for developmenttasks, or two people in a meeting room might want toexchange information on their PDAs. Other than the.UbiComp vision, we often use multiple computers formore practical reasons; PCs, UNIXs, and Mats have
Permissiolro nkde
digitnlhrd copies of all or p;ut of this mnte&l forpeIWld Or ehssroom use is granted without fee provided that the copiesore not
mnde r
distributed for profit or conunercial
he copy-right notice, the title of the publication and its date
nppe,~,ndnotice s
liven tht copyright is by pemksion ofthe ACM, Inc. To copy otherwise,to republish
to post
on servers or to redistribute to fists, require specificpermission Nldlor fee
UJST 97 Bon& Alberta, Canada
Copyright 1997 ACM O-89791-881-9/97/10..%3.50
typical “mouse
in a multi-computerenvironment
their own advantages and disadvantages, and users haveto switch between these computers to take full advan-tage of each (e.g., writing a program on a UNIX whileediting a diagram on a Mac).However, using multiple computers without consideringthe user-interface introduces several problems. The firstproblem resides in a restriction of today’s input devices.Almost all keyboards and pointing devices are tetheredto a single computer; we cannot share a mouse betweentwo computers. Therefore, using multiple computers onthe same desk top often results in a “mouse (or key-board) jungle”, as shown in Figure 1. It is very confus-ing to distinguish which input device belongs to whichcomputer.The other problem is the fact that today’s user inter-face techniques are not designed for multiple-computerenvironments. Oddly enough, as compared with remotefile transmission, it is rather cumbersome to transfer in-formation from one computer to another on the samedesk, even though they are connected by a network. Acut-and-paste on a single computer is easy, but the sys-tem often forces users to transfer information between
How many computers do you have on your desktop?0( 1 1 2 I>30%1 7.7 % 1 38.5 % 1 53.8 %Q2. How often do
eed to transfer data between computerson the same desktop?VeryOftenSometimes Occasionally Neveroften69.4 %25.0 % 2.8 %0.0 % 2.8 %Q3. (under the Q2 situation) How do you transfer data?BYThrough ByBYThroughhandshared ftpe-mailfloppies Otherfiles62.9 % 62.9 %57.1 % 34.3 % 20.0 %22.9%Q4. HOW ften do you need to transfer data from your computerto another’s computer within a short distance?VeryOftenSometimes Occasionally Neveroften28.2 %23.1 % 35.9 %5.1 %5.1 %Q5. (under the Q4 situation) How do you transfer data?BYThrough ByBYThroughhandsharedftpe-mailfloppies Otherfiles54.1 % 56.8 % 37.8 % 73.0 % 10.8 % 18.9 %Table 1: How people transfer information betweencomputers within a proximity distance: A survey con-ducted on the members of Sony’s software laborato-ries. About 100 people received this survey by e-mail,and 39 of them answered. Note that the answers forQ3 and Q5 are duplicated, so the totals may exceed100%.
computers in a very different way. A quick survey re-veals that people transfer information from display todisplay quite irregularly (Table 1). Interestingly, quitea few people even prefer to transfer data
by hand (e.g.,
read a text string on one display and type it on anothercomputer), especially for short text segments such asan e-mail address or a universal resource locator (URL)for the World Wide Web . We consider these tenden-cies to be caused by a lack of easy direct data transceruser interfaces (e.g., copy-and-paste or drag-and-drop),between different but nearby computers.The first problem is partially solved by using more so-phisticated input devices such as a stylus. Today’s sty-lus input devices such as WACOM’s, provide untetheredoperation and thus can be shared among many pen sen-sitive displays. This situation is more natural than thatof a mouse, because in the physical world, we do nothave to select a specific pencil for each,paper. Withthe second problem, however, we have much room forimprovement from the viewpoint of user interface+Although some systems use multi-display configurations [3,17,141, direct manipulation techniques for multi-displayenvironments have not been well explored to date. We,believe that the concept of multi-display direct manip-ulation offers many new design challenges to the field ofhuman-computer interfaces.In this paper, we propose a new pen based interactiontechnique called “Pick-and-Drop”. This technique letsa user exchange information from one display to an-other in the manner of manipulating a physical object,This technique is a natural extension to the drag-and-drop technique, which is popular in today’s many GUIapplications. Figure 2 shows the conceptual diffcrcnccbetween the traditional data transfer method and Pick-andyDrop.
DESIGNING PICK-AND-DROPFrom Drag-and-Drop to Pick-and-Drop
is a direct manipulation technique thatis an extrapolation of drag-and-drop, a commonly usedinteraction technique for moving computer objects (e.g.,an icon) by a mouse or other pointing devices. With thetraditional drag-and-drop technique, a user first “grabsan object by pressing amouse button on it, then “drags”it towards a desired position on the screen with themouse button depressed, and (‘drops” it on that loca-tion by ~releasing the button. This technique is highlysuitable for. a mouse and widely used in today’s graphi-cal applications.However, simply applying the drag-and-drop to pen userim&faces presents a problem. It is rather difficult todrag an objeqt with a pen while keep the pen tip con-tacted, on the display surface. It is often the case thata user accidentally drops an object during the drag op-eration, especially when dragging over a large displaysurface.L, ,Development of OUT, rqposed Pick-and-Drop methodstarted as useful alternative to drag-and-drop for over-coming this problem. With Pick-and-Drop, the user firstpicks alp an computer object by tapping it with the pentip and then lifts the pen from the screen. After thisoperation, the pen virtually
the object. Then, theuser moves the pen tip towards the designated positionon the screen ,without contacting display surface. Whenthe pen tip, comes close enough to the screen, a shadowof the object appears on the screen (Figure 4) <as visualfeedback showing that the pen has the data. Then, theuser taps the screen with the pen and the object movesfrom the pen to the screen at the tapped position. Thismethod looks much more natural than that of drag-and-drop. In our real lives, we regularly, pick up an objectfrom one place and drop it on another place, rather thansliding it along the surface of something. We would alsoliketo mention that this Pick-and-Drop metaphor: mightbe more familiar to people who normally us!: chop sticksat meals.
Inter-Computer Operations
We soon realized that the more interesting part of thePick-and-Drop operation is in its multi-display capabil-ity. That is, with the Pick-and-Drop a, user can pickup a computer object from one displaf and drop it onanother (different) display. Pick-and-Drop is a directmanipulation technique that tries to ignore the bound-\.32
Figure 2: The conceptual difference between remote copy and Pick-and-DropPalmtopFigure 3: System configuration
ary between computers. We also regard this as one ofthe first manifestations of a multi-computer direct ma-nipulation technique.There are a number of opportunities where people needto exchange information from one computer to another.Examples include:Copying a file from your PDA to your colleague’sPDA.When working in front of a wall-sized computer dis-play (such as the LiveBoard [4] or the HoloWall [ll]),one may want to pick up a document from one’s PDAand attach it to the wall display.When using two or more computers simultaneously(e.g., a notebook PC and a desktop PC), one maywant to copy a text string from one computer andpaste it on the other computer.Although these operations can also be implemented byusing remote copy or shared file systems, we feel that itis more natural to allow a user to manipulate a computerobject as if it were a real (physical) object.
(a) 1 (b) / (C) /
Figure 4: Pen and icons: (a) the pen contacts thedisplay, (b) the pen lifts up, but remains close to thescreen, and (c) the pen is away from the screenPen-IDS
Storing data on a pen, however, makes the pen deviceheavy and unwieldy. We developed the multi-computerPick-and-Drop without making such modifications tothe pen by introducing the concept of
Pen IDS.
In ourdesign, each pen is assigned a unique ID. This ID isreadable from the computer when a pen is closer enoughto its screen. We are currently using a combination ofmodifier buttons (attached to the pen as a side switch)to represent IDS. We also assume that all computersare connected to the network (either wired or wireless).There is a server called the “pen manager” on the net-work. (Figure 3).When a user taps an object (typically an icon) on thescreen with the pen, the pen manager binds its objectID to the pen ID. This binding represents a situation inwhich the pen
holds the object (even thoughthe pen itself does not contain any storage). When theuser moves the same pen towards the other display, thepen manager supplies the type of the bound object tothe display. Then the shadow of. the data appears on thedisplay below the current pen position. At this moment,the pen does not touch the screen. Finally, when theuser touches the display with the pen, the pen managerasks the first computer to transfer the data to secondcomputer.

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->