by Puripant Ruchikachorn
ルジカジォーン プーリパン

A Master Thesis

Submitted to the Graduate School of the University of Tokyo on July 30, 2008 in Partial Fulfillment of the Requirements for the Degree of Master of Information Science and Technology in Computer Science Thesis Supervisor: Takeo Igarashi 五十嵐 健夫 Associate Professor of Computer Science

ABSTRACT We present an interactive system for designing knots that combines sketch-based interface and physical simulation together. Knots or self-occluded three-dimensional (3-D) configurations of one-dimensional (1-D) deformable objects such as rope are difficult to model using existing 3-D modeling applications, in which many geometrical view changes and operations are required. In our system, a knot can be created the same way it is drawn on paper by two main gestures, “pulling over” and “pulling under”, which extend a selected 1-D structure along the user’s drawn line over or under all other parts in a 3-D scene. With a simple depth inference algorithm, this system allows user to tie a rope or other 1-D objects into basic yet topologically correct 3-D knots. The integrated physical simulation provides direction manipulation interface, so the user can interactively pull, tighten, and loosen a knot while maintaining the rope’s physical properties. This allows the creation of a variety of knots and their tightened forms by two-dimensional (2-D) input device. The combination of a sketch-based interface for construction and physical simulation for direct manipulation technique enhances the design process to create more satisfying results faster and easier than using traditional geometric modeling systems or traditional physical simulation systems with 3-D input devices. Furthermore this interaction technique is shown to be applicable to 2-D objects like cloth or ones of other domains.

論文要旨 結び目などといった、紐状のオブジェクトの3次元構造のデザインのため、我々はスケッチに よるインタラクティブなシステムを提案する。既存の 3 次元モデリングシステムでは、結び目な どの作成には、様々な幾何学的操作や視点操作が必要とされた。一方、本システムでは、2次元 にスケッチを描くことで、自己遮蔽の多い結び目のモデルが生成できる。重要なジェスチャーは、 「pulling over」と「pulling under」で、それぞれ、pulling over は既存のオブジェクトに上から 重なるように、pulling under は下に潜り込むように、スケッチに沿って紐を伸ばす。本システム では単純な奥行き予測アルゴリズムを利用し、2次元スケッチと同じトポロジーを持ったモデル の生成を実現している。また、我々は、物理シミュレーションを利用し、紐を直接ドラッグアン ドドロップする操作を提案する。この直接操作により、2 次元スケッチだけでは表現できない様々 な結び目のモデリングや、結び目を固く縛る事などが可能になる。2次元スケッチによる初期生 成と直接操作による変形により、3次元空間で視点操作数が減り、結び目のデザインが簡単にな る。また、提案手法は布などの2次元オブジェクトにも応用可能である。


I would like to thank Igarashi-sensei for guiding this research into the right direction. His fascinating volume of work on interactive computer graphics have been a great inspiration to me and sparked the idea of this work. My thanks are extended to other members of Igarashi laboratory for their helpful comments and encouragement. I have owed too much to the Japanese Government (Monbukagakusho) Scholarship program which has provided me not only financial support but also a chance to come to Japan of where I had dreamed for a long time. Special thanks to Tokyo for hosting me for two years and a half. It has provided not only accommodation, but also experiences I had never got in my home country; Tokyo has become my second home. However I could not have been able to stand some lonely time here without all friendly people I have met in Japan (not necessarily Japanese). It is a great pleasure to meet everybody. I would like to list them all here but the space does not allow me to. Please know that I sincerely appreciate your time with me. Also, thanks to technology, it is possible to constantly keep in touch with people in Thailand like I had been close to them. I reserve the last but definitely not the least gratitude to my friends and family who have patiently supported my study and tolerated practically many years of “unemployment”. Thank you very much indeed.


1 Introduction 2 Related Work 2.1 2.2 2.3 2.4 2.5 2.6 Celtic Knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mathematical Knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Medical-related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-D Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sketch-based Modeling Systems . . . . . . . . . . . . . . . . . . . . . . Other Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 5 5 5 6 7 7 8 9 10 12 12 13 13 13 13 13 14 15 16 16 16 16 17 17

3 User Interface 3.1 Sketch-based Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 3.1.2 3.2 3.3 Pulling Over. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pulling Under. . . . . . . . . . . . . . . . . . . . . . . . . . . .

Direct Manipulation Operations . . . . . . . . . . . . . . . . . . . . . . Other Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 3.3.2 3.3.3 3.3.4 Fixing and Unfixing. . . . . . . . . . . . . . . . . . . . . . . . . Cutting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Loosening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tightening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Implementation 4.1 4.2 4.3 Sketch-based Operations . . . . . . . . . . . . . . . . . . . . . . . . . . Direct Manipulation Operation. . . . . . . . . . . . . . . . . . . . . . . Other Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 4.3.2 4.3.3 Fixing and Unfixing. . . . . . . . . . . . . . . . . . . . . . . . . Cutting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Loosening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


4.3.4 4.4

Tightening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17 17 19 19 19 24 24 25 27

Physical Simulation

5 Results and User Feedback 5.1 5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . User Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6 Conclusion 6.1 6.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .



List of Figures


Knots in loose and tightened forms. (a) (b)

. . . . . . . . . . . . . . . . . . .

3 3 3 4 7 7 7 7 7 10 10 10 10 10 10 10 10 12 15 15 15 15 20 20 20

Reef knot before and after (dangerous) capsizing. . . . . . . . . . Carrick bend before and after (stable) capsizing. . . . . . . . . .

1.2 2.1

Slipped overhand knot being created and tightened. . . . . . . . . . . 2-D trick to create the illusion that two rings are interlocked. . . . . . (a) (b) (c) (d) Two independent ropes. . . . . . . . . . . . . . . . . . . . . . . . Rope segment at the intersection is copied. . . . . . . . . . . . . The copied segment is pasted over the intersection. . . . . . . . . Broken illusion when one rope is moved out. . . . . . . . . . . .


Slipped overhand knot being created using another series of operations. (a) (b) (c) (d) (e) (f) (g) Pulling over. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pulling under. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cutting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pulling over from the middle of rope. . . . . . . . . . . . . . . . Loosening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fixing and direct manipulation. . . . . . . . . . . . . . . . . . . Tightening and unfixing. . . . . . . . . . . . . . . . . . . . . . .

3.2 4.1

Pull-over and pull-under operations illustrated. . . . . . . . . . . . . . Varied representations of a simple overhand knot. . . . . . . . . . . . . (a) (b) (c) Connected spheres. . . . . . . . . . . . . . . . . . . . . . . . . . . Mesh model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A piece of ribbon. . . . . . . . . . . . . . . . . . . . . . . . . . .


Many models of practical knots created by the author. . . . . . . . . . (a) (b) Double overhand knot . . . . . . . . . . . . . . . . . . . . . . . . Slipped overhand knot . . . . . . . . . . . . . . . . . . . . . . . .


(c) (d) (e) 5.2 (a) (b) (c) 5.3 (a) (b) 5.4 (a) (b) (c) (d)

Water knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jam hitch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bowline knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure eight knot . . . . . . . . . . . . . . . . . . . . . . . . . . . Threefold overhand knot . . . . . . . . . . . . . . . . . . . . . . Intermediate knot . . . . . . . . . . . . . . . . . . . . . . . . . . Small knots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generated chain link fence patterns . . . . . . . . . . . . . . . . Threefold overhand knot . . . . . . . . . . . . . . . . . . . . . .

20 20 20 21 21 21 21 21 21 21 22 22 22 22 22

Loosened and tightened forms of many knots. . . . . . . . . . . . . . .

Plane-filling results with self-occluded substructures. . . . . . . . . . .

Four knots in user study. . . . . . . . . . . . . . . . . . . . . . . . . . . Slipped overhand knot . . . . . . . . . . . . . . . . . . . . . . . . Jam hitch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bowline knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


List of Tables


Average time and the number of operations used to model four target knots in the user study. . . . . . . . . . . . . . . . . . . . . . . . . . . 23


Chapter 1 Introduction

Knot is a term normally defined as a product of tying or interweaving one or many segments of linear deformable material such as rope, while mathematical definition is stricter and includes only closed curve in 3-D space (excluding a simple circle) whose properties are officially studies in a branch of topology called knot theory. In fact, what we call knot is usually referred as braid in Mathematics. Knot theory plays an important role in many applied mathematics fields to understand behaviors of molecules and enzymes and is getting more interest in the community because of many recent breakthroughs like Jones polynomial. Many archaeological samples suggest that knot tying was done before historic times. In Qing Dynasty, knot became an art form with its 3-D look, intricate pattern, and color. This so-called Chinese knot has spread to Korea and Japan too. On the other side of the world, Celtic knot is a certain form of stylized knot for graphical purpose and decoration. It was firstly adopted by the ancient Celts, and later becomes a identity of Irish, Scottish, or Welsh. Also some other stylized elements like animals or plants are often added especially in early book decoration. Although the art of knot tying has almost lost in the past century, it has gained popularity and many have attempted to bring back the lost art in past twenty years. In everyday life, knot has many practical uses that are usually overlooked such as shoelacing. In sports and activities like sailing and rock climbing, a knot can mean life or death. In addition to its widely recognized practicality and aesthetic, knot now plays an important role in modern surgery too. A logical extension to the digital world is to model these objects in virtual space. To the computer graphics community, the more easily a complex model is created, the richer its visual content becomes. Interwoven models such as knitwear, lace, chain


link fence, or a part of armor suit are commonly seen in games and movies. With more computational power, adding such models can provide more realistic credibility or even more correct physical interaction to graphical scene. By its nature of self-occlusion and complicated topology, 3-D model is a suitable medium to represent knot. Currently most knot-related books and websites provide a series of still diagrams or photos to describe how to tie a knot. Although the mean is easy to interpret, creating such diagrams or photos requires much preparation and time. 3-D model is rarely used as learning material. In traditional 3-D modeling programs, knots can be modeled by first creating a spline in a 2-D view and then editing the depths of the necessary parts. These steps can be confusing and counterintuitive because the depth or the relationship between parts is usually determined during the creative process, not afterwards. In addition, the lack of physical properties such as collision and friction does not allow the knot to be tightened or modified as it could in reality. The knots that are created by traditional 3D modeling software packages are usually in their loosened forms as modeling a tight knot from the start is very hard. Those knots are not of many uses because knot is useful and found in everyday life only in its tightened form which can be totally different from its loosened form in terms of both appearance and physical properties as shown in Figure 1.1. There are two main reasons why faithful models of interwoven objects have not been widely used. One is due to technical limitation which will be overcome by faster and more advanced technology. The other is that current modeling tools are not designed for such objects. Existing modeling paradigm doesn ’ allow easy creation t and artists cannot freely add creative process into their work routine. Also designing interface for knot tying is challenging as it is claimed to be difficult for normal users because manipulating interweaving model requires a lot of creativity and practice to give exact and economical commands [32] especially for deformable model. The main goal of our research is to make design of 3-D knots easier by introducing specialized user interface tailored to physical knot tying. With sketching gestures and automatic depth inference, we can eliminate the necessity of manually setting each control point in a traditional approach. Even in the most sophisticated 3-D modeling systems, several geometric view changes are required to rotate working model into proper orientation in order to set correct depth. In our system, user can create a knot model as if drawing a knot diagram on paper from only one perspective. This system also adds interactive physical simulation into the design process, which supports the user’s experiments and helps in the overall visual quality of the design result. 2

(a) Reef knot before and after (dangerous) capsizing.

(b) Carrick bend before and after (stable) capsizing.

Figure 1.1: Knots in loose and tightened forms. Some postures may lead to dangerous and slippery results. Geometric modeling and physical simulation are usually separate processes. Typical geometric modeling systems require the user to specify each element’s position explicitly in 3-D space and completely ignore the physical properties. Although sketchbased modeling systems simplify the process by automatically computing various parameters from user-drawn sketches, most heuristics are geometrically motivated and do not consider dynamic physical behaviors. In contrast, most physical simulation systems are designed to predict what will happen in a given environment by mimicking the natural laws of physics, and are not intended to create original geometry reflecting the user’s intention. The use of physical simulation in our system is unique in that it is not aimed at simulating the real knot tying as in surgical simulations [8] but to help in designing arbitrary knot configurations as in geometric modeling systems. The combination of sketch-based interface and physical simulation is a nontrivial contribution. Sketching is proved to be easy to learn and good at communicating coarse information, but it is not good at specifying details. In contrast, physical simulation can automatically compute details and guide working design into plausible model, but overall configuration must be set manually. They fill complement each other’s missing gap and the combination of both is natural and powerful, but not yet 3

Figure 1.2: Slipped overhand knot being created by sketch-based operations and tightened by the integrated physical simulation in this system. fully explored in previous literature. In addition, because the resulting knot in a tightened form can be vastly different from its clear knot diagram that shows the rope in a loosened form (Figure 1.1), displaying the immediate result of the simulation can also indicate errors in knot tying steps whose small mistake can lead to a different or ineffective knot, or even an unknot. For interwoven models, knots are the most basic type, configurations of 1-D objects embedded in 3-D space. Our system is a proof of concept that a combination of sketch-based interface and physical simulation works well with these kinds of objects. Modeling knot is a logical starting point for broader problem of how to interweave objects of higher dimension. We believe that our result will be a valuable resource for those who will work on more general problem such as configuration design for 2-D cloth in the future. Also our interface could be directly applicable to other already flourishing application domains of rope simulation whose rope needs to be put at relative depth regarding other objects in its scene such as microsurgery [8] and cable routing in car assembly simulation [15]. The rest of the thesis is organized as follows. Chapter 2 provides a brief summary of previous literatures on various related topics. Chapter 3 and chapter 4 describe our system’s user interface design and implementation details respectively. In chapter 5 shown are the results and user study, and chapter 6 concludes this master thesis with discussions and future work.


Chapter 2 Related Work

This chapter provides a review of previous research and computer programs on related topics. With their brief description and difference from our work, listed are work on Celtic knot, mathematical knot, simulation in medical field, 2-D and 3-D modeling approaches, and other related work.


Celtic Knot

The Celtic knot has been extensively studied by many enthusiasts and computer scientists. Following Christian Mercat’s method, several computer programs such as Knots3D, KnotsBag, KnotScape, and Celtic Knot Thingy for knot construction have been released. Most of them are designed for aesthetically pleasing purposes, and not for other interactions which require physical simulation e.g. tightening or loosening. Recently nostalgic interest in ancient maximalist art has led to a research on 2-D computer-generated celtic design [22] which is later updated to correctly fill 3-D-model surface with such design [23].


Mathematical Knot

Another application of knot modeling is accurate visualization and simulation of mathematical knots. One of the applications widely used by Mathematicians is Knotplot [31]. Although it provides a graphical user interface for inserting links as a part of a knot, the resulting knot is still an abstract model showing no properties of the physical object such as collision or friction and user must specify the depth of each control point explicitly. However there is a sketch feature in KnotPlot. Admittedly it shares a similar


function to our system’s, but the experience is different. KnotPlot’s sketch feature is based on placing vertices onto 2D canvas and the user has to explicitly specify depth by either left- or right-clicking, while our system’s modeling process resembles drawing on paper. Our user study confirms that users can model many knots in less than a minute by following strokes (implicitly specifying depth) in static knot diagram, which is a common learning material for knot tying. The main difference between KnotPlot and our system is the focus. Though KnotPlot is an excellent tool to study mathematical knots, it lacks many crucial features for creating knots found in everyday life. KnotPlot cannot tighten knots. Because knots in everyday life are commonly found (and can be of practical use only) in their tightened forms, tightening is essential to create knot models for graphical applications mentioned in chapter 1. Firstly, there are no tightening-related operations such as fixing or unfixing which is a metaphor of a missing hand in an intrinsically twohand operation like knot tying. Secondly, KnotPlot has no visual simulation of local friction which will hold parts of a rope together. Hence, because of the different focus, our system can create forms of knots that KnotPlot cannot. For example, without painstaking manipulation, it is impossible to tie bowline knot (Figure 5.1(e) and 5.4(d)) in KnotPlot; we would get a topologically equivalent knot which looks totally different and has other different uses.


Medical-related Work

One of the most active applications of deformable linear object simulation is in medical field. Knot tying is an important part of suture simulation in microsurgery practice. Several related works have studied on sophisticated deformable models [28, 33, 19, 6], knot tying systems [29, 7], and surgery simulation systems [8]. Although these works provide satisfyingly realistic results, they require special input devices to manipulate deformable strings in 3-D space, while we provide solutions for commonly used 2-D pointing devices such as mouse or trackpad. In addition, because of their specific applications, rope or sutures in surgical simulation can be manipulated only at the working end, while we provide manipulation of other parts for more common applications. In the other word, their tying process must be realistic; in our case, final appearance of knot is more important and the process can be artificial. Though Realtime Knot-tying Simulation [7] provides the fundamental base for our rope simulation, its focus is different from ours as it doesn’t concentrate on user interface design.



2-D Techniques

Usually a knot is drawn on a two-dimensional (2-D) plane as a planar graph called a knot diagram. Most are drawn by hand or by a generic drawing program that supports layers. As all objects in a layer must have the same depth, the diagram cannot contain self-interweaving objects without using labour-intensive tricks such as cutting an object into many parts and assigning different layers to them in order to set different depths as shown in Figure 2.1. To overcome this drawback, many systems were built specifically for creating easy illusions using various stroke outlines, color fills, and particular interaction techniques [5, 4, 14, 35, 3]. These work well in many simple applications, but no real physical connection exists between separate parts, so when the viewpoint is changed, all objects must again be cut and rearranged.

(a) Two independent ropes.

(b) Rope segment at the intersection is copied.

(c) The copied segment is pasted over the inter- (d) Broken illusion when one rope is moved out. section.

Figure 2.1: 2-D trick to create the illusion that two rings are interlocked. (Images adapted from [35])


Sketch-based Modeling Systems

Most modeling systems for 3-D free-form design [18, 2] do not support self-occluded objects. Although several attempts have been made to reconstruct smooth solid shapes from static occluding contours [36, 24, 12], because of no integrated physical simulation, no further interaction is allowed after topological inference and hence they 7

produce only non-deformable objects and limited forms of interwoven models (e.g. loosened knots.) Although most sketch-based design systems allow the user to correct the results by deleting parts, redrawing, or oversketching, physical objects like rope or cloth are unlike drawings in that they require direct manipulation that follows a fast and relatively correct physical simulation. We are not the first to include physical simulation in the creative cycle though. MathPad2 [20] combines 2-D mathematical sketching and basic physical simulation for mathematics problem solving and visualization. Other works [17, 27] have integrated cloth simulations into the interactive design process, but to our knowledge, exists no application that incorporates self-collision of our target result, linear deformable model. Note here that our sketch-based gestures are similar to Drive [25], an existing sketch-based track modeling system. Though it involves with physical object design, it does not provide any tight coupling with physical simulation system. Anyway path design is intrinsically different from knot design because its form and function do not change with external force.


Other Related Work

Geometric substitution [13], which recursively replaces simple shapes with more complex geometry, can be used to create intricate knot models. However, the resulting models look good only with appropriate input geometry, and are often unpredictable. Also geometric texture synthesis [38] and knitwear rendering [37, 10] and simulation systems [21] can generate interwoven 3-D objects like net or cloth. Despite the beauty of their results and their ability to show different physical properties from subtly different details at yarn level [21], these systems lack an interactive user interface to control and add artistic value to the modeling process. Our system allows the user to create such models freely and can generate plane-filling structures as well (Figure 5.3). Furthermore there are many research indirectly involved in interactive configuration of 1-D flexible model as in cat’s cradle game [9] and cable and hose routing simulation in automotive product design [15]. In these systems, usually model construction is vaguely mentioned or just follows traditional modeling paradigm. Also knot models with proper representation can be used as an input for sophisticated robotic arm to tie real knot [26, 34, 30].


Chapter 3 User Interface

Our system initially has one main blank area on which the user can draw 2-D strokes, and the corresponding result will be displayed under the same view. The user’s input will be discarded when the mouse button is released, and the rope simulation will be applied, gradually changing its 3-D appearance according to physical constraints (see section 4.4 for more detail). After that, the interaction mode changes to the direct manipulation mode, which allows users to drag or untie selected parts of any rope on the screen, with physical simulation applied in real time. To add new rope, users can simply deselect all parts and start drawing again as the interaction mode switches back to the sketch-based interface. This implicit switching between two modes is intentional in order to allow smooth transition between two different operations which require different mindsets and provide both advantages of both world; intuitive gestures from sketch-based interface and realistic manipulation from direct manipulation technique. Note here that knot tying process in our system does not need to follow the laws of physics, but the resulting knot configuration represents some physically plausible configuration. For example, although the rope does not penetrate itself and the knot topology is conserved, unlike knot tying in the reality, our system allows the rope to be cut or joined freely, or its length to be extended. Although we could have designed the system to preserve the length of the drawn rope, we observed in early experiments that initially user often draws a short rope that is long enough for a tightened knot, but too short for its working form. In addition, as most knot tying diagrams display a rope with one or no working ends which implies a rope of infinite length, we decided to extend the rope length whenever needed or whenever possible. The main focus is at the knot itself, not the rope length, and user


can always use cut operation to get rid of unnecessary parts if desired.


Sketch-based Operations

While special hardware is usually used in other knot tying systems [29, 7], we stick with universal 2-D pointing devices such as mouse and trackpad since most users are well familiar with the apparatuses and their metaphors. The target user of our system is a normal user who wants to create a 3-D model of self-occluded objects found in everyday life.

(a) Pulling over.

(b) Pulling under.

(c) Cutting.

(d) Pulling over from the middle of rope.

(e) Loosening.

(f) Fixing and direct manip- (g) Tightening and unulation. fixing.

Figure 3.1: The same slipped overhand knot as shown in Figure 1.2 being created using another series of various operations supported in our system.


A question in knot tying interface design is how easy and intuitive tying operations in computer could be, compared to the same common real-world tasks which are often in 3-D space. The fact that any knot manipulation operations can be done in a projection plane by Reidemeister moves suggests a possibility to create and interact with a knot by using only 2-D devices. A user interface of a system for modeling interweaving linear deformable object can be in a form of traditional mathematical expression (as a list of vertices and edges), but sketch-based interface with well-defined gestures can be more intuitive, support more flexibility and artistry, and suit our target user well. With some distinct gestures specifically designed for self-occluded linear objects, the depth of each part can be correctly specified. These sketch-based gestures are inspired by the natural gestures that a human performs when tying a knot, like “pulling under” and “pulling over”. Also similar symbols are often found in illustrated knot diagrams and knot tying instruction as they can be clearly visualized and defined. A user of our system who wants to tie knot from an instruction can rotate and match a working knot with one in the diagram and use these two gestures to strictly follow tying steps easily. Noteworthy here that although these gestures are definitely not complete, they reflect the most common sketching behaviors normal people do when drawing selfoverlapped figures. Although we agree that a more exhaustive set of operations could be useful, we stick to our current two main gestures not only because they are simple and easy to understand but also because they are sufficient. Technically, a knot configuration can be represented by its (infinitely thin) central axis and even the most complicated knot can always be projected onto 2-D plane whose all crossings are of only two ropes, so all knot diagrams can be created by drawing a continuous line and going over or under parts of itself. Our modeling operations are similar to those of other 3-D sketch-based modeling systems [18, 2]. The corresponding result is displayed directly in the same space as the input, but in the case of our system, a linear object along a user’s sketch is created instead of a solid model. Sketching mode is activated when part of a rope is selected, with a big blue sphere as indicator. When the selected part is either end of the rope, sketching extends the model out in one direction by connecting the new stroke’s start point with the selected point or the end of selected rope (Figure 3.1(b)). If the selected part is in the middle of the rope, the rope is extended as if it were cut in the middle at the selected point, with both new ends extended along an input stroke, and joined together again 11

(Figure 3.1(d)). This extension is very useful for creating a knot that can be tied by forming a bight into a loop. Although all knots can be tied from working ends only, some knots such as a bow tie are easier to work on a bight and loop.

Figure 3.2: Pull-over and pull-under operations illustrated.


Pulling Over.

When the input stroke crosses itself or other existing elements on the screen, the new segment is pulled over the preexisting segments. In the other words, its z-depth is increased in the direction perpendicular to and out of the screen, as shown in Figure 3.1(a) and 3.1(d). This gesture is similar to sketching such objects on paper when a person would usually erase any existing lines along a path and then draw a new line on top of that path to show that this line should be over all other lines in the drawing. 3.1.2 Pulling Under.

When part of a rope is selected and the user starts sketching, the newly generated rope between the selected area and input stroke’s start point is extended and pulled under all other existing elements on the screen. In the other words, its z-depth is increased in the direction perpendicular to and into the screen as shown in Figure 3.1(b). This gesture is similar to sketching such objects on paper when a person would usually draw and stop drawing just before any lines meant to be over the drawn part and continue drawing right after those lines to show that these broken paths are indeed connected.



Direct Manipulation Operations

When a user clicks on any part of a rope, a blue solid sphere indicates the selected part and functions as a handle for dragging. As being dragged (Figure 3.1(f)), the selected part is moved in the plane parallel to display screen, and other parts follow under realistic rope constraints.


Other Operations

Since fixing, unfixing, and cutting are defined by the same interaction, we use the place of interaction and a separate interface element (drop-down list) to select only one of them at a time. Undo is present in the system and reverses both the action and all simulation steps in between. 3.3.1 Fixing and Unfixing.

Knot tying requires relatively complex maneuvers and it not only is in 3-D, but also uses both hands. However, in contrast to two-hand text input device like keyboard, a standard graphical input in most personal computer (PC) system has only one 2-D pointing peripheral such as mouse or trackpad. Although the system can imply 3-D depth from gestures, as a replacement for the missing hand, we introduce fixing as a metaphor for the static hand holding a portion of the rope. When the fixing mode is selected, double-clicking on any part of the rope toggles fixing constraint. As shown in Figure 3.1(f), a red solid sphere is displayed to indicate a fixed part. Although the fixed part is not included in the physical simulation cycle, it can still be dragged around by direct manipulation. 3.3.2 Cutting.

This operation can cut a rope short or break a knot apart. When the cutting mode is selected, double-clicking on any part of the rope cuts the rope at that point (Figure 3.1(c)). After cutting, either end at the cut result is selected. 3.3.3 Loosening.

Sometimes a tightened knot or loop has too small space to insert a new input stroke easily. To open some space, an area containing no knot strand is radially expanded


when double-clicked as if an acute cone point is wedged into the area until the mouse button is released (Figure 3.1(e)). 3.3.4 Tightening.

With the help of the fixing function, the user can grab a knot’s end and drag it to tighten the knot. Otherwise the user can command “tightening” which automatically pulls the two ends of a rope away from each other and, hence, tighten the knot.


Chapter 4 Implementation

Like other interactive systems with realistic physical constraints [17, 27], the result model generated by the system does not follow the user’s input sketch directly but instead makes adjusted strokes that are controlled by a mass-spring model. We use standard mesh models for visualizing a knot on the screen. Internal representation of a rope is a series of spheres connected together with forces according to the mass-spring model. The advantage of using two representations for a model is that working on spheres is computationally inexpensive compared to the more attractive mesh model (Figure 4.1). Since it is straightforward and relatively fast to display the internal structure, connected spheres, as one triangular mesh, living in both world is not really a pain, but actually a plus.

(a) Connected spheres.

(b) Mesh model.

(c) A piece of ribbon.

Figure 4.1: Varied representations of a simple overhand knot. Any operations or physical simulation updates the internal spheres by adding, deleting, fixing, or adjusting their positions, and a new mesh model is created to be displayed on the screen. Details of how each operation is implemented are given in the following sections.



Sketch-based Operations

The user’s input stroke is projected onto a plane parallel to the screen, and each sphere is added to the same rope if the stroke does not cross any existing ropes or objects on screen. The case of extending a rope from either end is the same. If a part of rope is extended from the middle, the system literally cuts the rope at that point and repeatedly extend both cut ends along the same input stroke to join them together again when the input stroke ends. In the pulling-over operation, when a newly added sphere encounters other spheres or objects in the viewing direction, the sphere and its neighbors are moved in the direction toward the user at a predefined distance from the closest object. In the pulling-under operation, when no other sphere or object exists in the direct path between the selected sphere and the input stroke start point, new spheres are placed uniformly on the sampled path and linked with other spheres generated along the input stroke. Otherwise, newly added spheres are pushed into the screen away from the user at a predefined distance from the farthest object. Then the extension path is resampled to ensure that all spheres are equally spaced and the physical simulation takes place and govern the integrity of the rope structure. Note here that the length of rope is not conserved so the result of the 3-D rope model length can be equal or longer than the input stroke length.


Direct Manipulation Operation.

When a sphere is grabbed and dragged, it is moved along the mouse trajectory a maximum distance of one sphere radius per simulation cycle to prevent penetration and topological inconsistency. After target position of the handle is set, the positions of other spheres are adjusted to maintain the rope’s internal constraint. Our rope motion is calculated in a follow-the-leader manner [8, 7] which helps the rope converge to a stable configuration more quickly and act reasonably realistic, similar to a stiff rope in an environment with no gravity.


Other Operations
Fixing and Unfixing.

When a sphere or more is fixed, it is moved in simulation cycle. Unfixing releases this positional constraint. However, like a normal sphere, fixed sphere can be directly


grabbed and dragged around via direct manipulation interface and affect other spheres. 4.3.2 Cutting.

In current implementation, selected sphere is removed and new mesh models for both parts are regenerated. The sum of the length of both ends is less than the original rope, but it leads to no topological inconsistency and instead helps open up space between two cut ends; it is a clear indicator that the selected rope is cut and allows user to select both ends more easily. 4.3.3 Loosening.

Double-clicking on empty space creates an invisible cylinder of infinite length, perpendicular to the viewing plane. The cylinder will gradually increase in size while the mouse button is being held. Any spheres on the rope which intersects this cylinder are moved to the surface of the cylinder along its radius. Run in parallel with the rope simulation cycle, this operation preserves the overall rope length and gives the impression of loosening a tightened loop or knot. 4.3.4 Tightening.

While tightening, two spheres at both ends of a rope are moved in opposite direction with constant speed away from each other. Added kinetic energy will tighten a rope up and will stop automatically when it is overcome by internal energy which keeps the integrity of rope when knot is tightened.


Physical Simulation

A rope is represented as spheres connected together with a simple classical method for rope simulation like mass-spring model. Although it cannot express twisting and other physical properties available in newer models [28, 33, 19, 6], the mass-spring model is easy to implement and gives satisfyingly accurate simulation result. Without much accuracy or convincing experience that simulation in engineering or entertainment field can provide, physical simulation in our system supports user design by giving immediate result and encourages user to experiment with tools and possibilities at interactive speed. To achieve interactive performance on each computer, some parameters like size of spheres can be adjusted. The bigger spheres are, the less integrity conserving and collision detection costs are. Also the model will be coarser and faster to display.


Like elements in normal mass-spring model, the spheres are linked with invisible springs whose force is linearly proportional to the difference between its rest length and the distance of two connected spheres. Two types of springs connect spheres on the same rope. First, stiff springs are attached to all pairs of neighboring spheres. Second, much weaker springs connect all pairs of next-nearest neighbor spheres to provide the bending effect. Also repulsive force is calculated in the same cycle and used as a method of collision detection here. The repulsive force is inversely proportional to distance of any pair of spheres (excluding neighbors) which touch each other. We follow discrete simulation steps widely used in many major systems including Brown et al.’s knot tying simulation system [7]. By adjusting multiplying factors to keep all ropes behave reasonably robust and topologically correct, all internal and external forces, both attractive and repulsive, are combined, applied once in each simulation cycle, and solved by the forth-order Runge-Kutta method to obtain velocity and position of each sphere. To prevent even the smallest penetration, which could dramatically change the topology of the model, a sphere cannot move farther than its radius in each simulation step. Hence, the smaller the spheres are, the slower the rope can be manipulated in one step and more steps are required to complete a move of the same distance. Note that rope surfaces can still contact or even sink into each other, but the relaxation process, which includes collision detection and runs in every simulation cycle, will pull them apart and not allow further sinking that could lead to penetration. Collision detection not only pulls penetrating spheres back, but also sets their velocity to zero, which effectively stops colliding spheres from moving in the next cycle. In addition to preventing recurrent collisions, the rope surface seems “sticky” or gives the visual impression of possessing friction. This is a desirable effect because it adds more realism and prevents knot from loosening.


Chapter 5 Results and User Feedback

Our system is implemented in Java and runs at interactive speed on a 1.0 GHz PowerPC computer. As shown in Figure 5.1 and Figure 5.2, we used the system to design many basic practical knots that commonly appear in knot-tying books and websites. The result model of our system is mesh model, so it can be easily exported, used in other systems, or added in 3-D scenes even as a static decoration.



The number of strokes used to create a knot model depends on its complexity and how to tie it. Most examples in this thesis require 3-4 strokes to model. Other operations were used only sparingly to correct some errors or set the final shapes of the results. Most of them took less than a minute to create and a few more minutes to be properly tightened and nicely posed for later use as an image in a publication or website. Usually tightening takes approximately two minutes to converge at a simulation rate of 30 cycles per second in typical examples of roughly 200 spheres. Most of the computation time is spent on collision detection, and the rope moves much more slowly when multiple contacts are present between surfaces.


User Feedback

We have conducted an informal user study to test our system. Eight first-time users (mainly graduate students in computer science) were given a brief demonstration of all features in the system, a five-minute tutorial under our guidance, and tasks to model four basic knots from images of their loosened forms. All tasks were completed within a few minutes. Figure 5.4 shows the target knots shown to a user and his resulting


(a) Double overhand knot

(b) Slipped overhand knot

(c) Water knot

(d) Jam hitch

(e) Bowline knot

Figure 5.1: Many models of practical knots created by the author. knots, and Table 5.1 shows average time and the number of operations the users used to create target models. Generally we got positive feedback from the users. They said that the gestures are easy-to-understand as they follow familiar symbols in knot diagrams. One test user mentioned that modeling knot with relative depth inference seems much easier than using spline tool available in traditional applications. And some users had absolutely no idea how to create such models otherwise. However there were complaints related to the system’s technical limitations such as slow tightening 20

(a) Figure eight knot

(b) Threefold overhand knot

(c) Intermediate knot

Figure 5.2: Loosened and tightened forms of many knots.

(a) Small knots

(b) Generated chain link fence patterns

Figure 5.3: Plane-filling results with self-occluded substructures. and wiggling initial rope appearance, though the wrinkle will be eventually resolved after a few simulation cycles. The user study clearly shows the advantage of our interface as a knot can be created in many ways. Some used the automatic depth implication implemented in our gestures, while others preferred rotating and direct manipulation interface. Our system allows these approaches or even a combined one in our exploratory interface space. (See Figure 1.2 and 3.1 for two different ways to tie a slipped overhand knot.)


(a) Threefold overhand knot

(b) Slipped overhand knot

(c) Jam hitch

(d) Bowline knot

Figure 5.4: Four knots we asked all test users to create in our user study. Shown in each subfigure are static target knot drawing, a test user’s loosened and tightened results respectively.


Table 5.1: Average time and the number of operations used to model four target knots in the user study. Avg. time to model Threefold overhand knot Slipped overhand knot Jam hitch Bowline knot (second) 13 18 34 41 Avg. time to tighten (second) 116 87 127 92 The number of strokes 4 3 5 4 The number of fixed parts 0 2 2 3

We believe that most computer users will quickly master our set of gestural operations as they follow familiar symbols in knot diagrams. Since it uses a standard pointing device like a mouse to build on the users’ acquaintance with the 2-D input space, the system presents a gentler learning curve than a system with unfamiliar 3-D hardware. Regarding knot tightening, the user doesn ’ have to fix loop in order to tighten t because friction behavior is also implemented in the system; if the loop is big enough, it will not get collapsed before the knot is tightened. Anyway fixing the loop can ensure that its size will be constant and the tightening may converge faster. A user can determine parts to fix easily as they are the same parts one would hold to prevent a loop in a real knot from collapsing. (Usually they are the beginning and the end of a loop.)


Chapter 6 Conclusion

In conclusion, this master thesis presents the problem of creating self-overlapped 3D models which are usually created by using unfamiliar 3-D input hardware which limits its use in surgical training. Our system suggests another possibility of using common 2-D pointing devices with intuitive combination of sketch-based interface and physical simulation to achieve such a task. The designed interface is based on widely understood symbols used in hand-drawn knot diagram; hence it is easy to understand even for novice users and allows them to create fairly complex models within few minutes. With many plausible extensions into designing similar system for objects of other classes like 2-D cloth, we hope that this research has paved the way for other user interfaces designed for creating and manipulating self-overlapped or interweaving objects.



Like many other sketch-based systems, our gesture commands are limited to basic ones that are easy to understand by most users. For example, our pull operations can go either over or under all other objects on the screen, but not in between. We believe that the set of operations provided in the system is adequate for novice users, but also agree that more extensive operations would satisfy serious enthusiasts and scientists working in professional fields like surgery simulation. Regarding performance issue, collision detection is the most obvious bottleneck in our system as it can slow down the system drastically if a knot is tightened and has many contacting surfaces. One way to solve this problem is to apply more sophisticated schemes such as continuous collision detection and algorithm to speed up the performance to cope with more ropes and detailed models. Also other more physi24

cally accurate models can be applied to the system in the future in order to represent various types of ropes. However, though rope simulation is an important part in our system, it should be considered as a mean but not an end. We acknowledge the performance issue of our simulation, but we want to emphasize again that our contribution is in the unique combination of both sketch-based interface and physical simulation, not one of each own. Currently, our system accepts only ropes with constant width along its length and all of them must have an equal predefined diameter. Also they all have the same set of other properties like coefficient of friction and reflectance distribution attribute. This leaves out the possibility of simulating knots made by different types of ropes. The only parameter we allow for different ropes is color and it can be set only in color chart before drawing stroke for new rope. Letting user choose these properties freely is interesting, but simple and intuitive interface for multivariate environment is still an active field of research in user interface.


Future Work

In addition to faster and more accurate simulation result we will gain from using sophisticated models as mentioned in section 6.1, the system can calculate many important values such as the knot efficiency, a ratio of breaking strength of the knotted and unknotted rope, along with each knot model. Despite its increased computational complexity, using an adaptive model [16, 19] could help improve interactive performance. Our current implementation can generate step-by-step tying instructions for a knot by capturing all important events before a newly drawn rope segment crosses over or under existing ones. However, these pictures of knot tying in progress may not reflect how a knot is actually tied because some steps may be repetitive and can be eliminated for clearer understanding, while other minor steps are so important that they should be magnified and explained in detail. One of our future work is to design a smart system that can generate step-by-step instructions specifically for self-overlapped and interwoven models similar to the system proposed by Agrawala et al. [1]. Also we want to extend the capability of current system to cover wider range of objects of other classes such as cloth in order to demonstrate other possibilities of the combination of sketch-based interface and physical simulation we has begun exploring. Though we can map existing results onto 2-D cloth as shown in Figure 4.1(c), modeling


bow tie or necktie is still an open challenging problem without another set of intuitive operations for 3-D orientation of 2-D objects like face flipping or folding. Lastly we plan to talk with real knot tyers and aficionados or those who tie knot as hobby to ask for comments and perform real evaluation with them. This is a very important step to guide our work into the right direction in the future.



[1] Maneesh Agrawala, Doantam Phan, Julie Heiser, John Haymaker, Jeff Klingner, Pat Hanrahan, and Barbara Tversky. Designing effective step-by-step assembly instructions. In SIGGRAPH ’03: ACM SIGGRAPH 2003 Papers, pp. 828–837, 2003. [2] A. Alexe, V. Gaildrat, and L. Barthe. Interactive modelling from sketches using spherical implicit functions. In AFRIGRAPH ’04, pp. 25–34, 2004. [3] Paul Asente, Mike Schuster, and Teri Pettit. Dynamic planar map illustration. ACM Trans. Graph., Vol. 26, No. 3, p. 30, 2007. [4] Ronen Barzel. Faking dynamics of ropes and springs. IEEE Comput. Graph. Appl., Vol. 17, No. 3, pp. 31–39, 1997. [5] P. Baudelaire and M. Gangnet. Planar maps: an interaction paradigm for graphic design. In CHI ’89, pp. 313–318, 1989. [6] Mikl´s Bergou, Max Wardetzky, Stephen Robinson, Basile Audoly, and Eitan o Grinspun. Discrete elastic rods. ACM Trans. Graph., Vol. 27, No. 3, pp. 1–12, 2008. [7] Joel Brown, Jean-Claude Latombe, and Kevin Montgomery. Real-time knot-tying simulation. Vis. Comput., Vol. 20, No. 2, pp. 165–179, 2004. [8] Joel Brown, Kevin Montgomery, Jean-Claude Latombe, and Michael Stephanides. A microsurgery simulation system. In MICCAI ’01, pp. 137–144, 2001. [9] R. Budiarto, M. Yamada, H. Itoh, and H. Seki. An interactive system for constructing cat’s cradle string diagram using ga. Computers and Industrial Engineering, Vol. 31, pp. 939–943(5), October 1995.


[10] Yanyun Chen, Stephen Lin, Hua Zhong, Ying-Qing Xu, Baining Guo, and HeungYeung Shum. Realistic rendering and animation of knitwear. IEEE Transactions on Visualization and Computer Graphics, Vol. 9, No. 1, pp. 43–55, 2003. [11] Joseph Jacob Cherlin, Faramarz Samavati, Mario Costa Sousa, and Joaquim A. Jorge. Sketch-based modeling with few strokes. In SCCG ’05, pp. 137–145, New York, NY, USA, 2005. ACM. [12] Frederic Cordier and Hyewon Seo. Free-form sketching of self-occluding objects. IEEE Computer Graphics and Applications, Vol. 27, No. 1, pp. 50–59, 2007. [13] Andrew Glassner. A tutorial on geometric replacements. IEEE Computer Graphics and Applications, Vol. 12, No. 1, pp. 22–36, January 1992. [14] Andrew Glassner. Celtic knots, part 1–2. IEEE Computer Graphics and Applications, Vol. 19, No. 5–6, 1999. [15] Mireille Gr´goire and Elmar Sch¨mer. Interactive simulation of one-dimensional e o flexible parts. In SPM ’06, pp. 95–103, 2006. [16] E. Hergenroether and P. Daehne. Real-time virtual cables based on kinematic simulation. In WSCG 2000, pp. 402–409, University of West Bohemia, Plzen, Czech Republic, February 2000. [17] Takeo Igarashi and John F. Hughes. Clothing manipulation. In UIST ’02, pp. 91–100, 2002. [18] Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. Teddy: a sketching interface for 3d freeform design. In SIGGRAPH ’99, pp. 409–416, 1999. [19] Matthias Teschner Jonas Spillmann. An adaptive contact model for the robust simulation of knots. In Computer Graphics Forum (Proceedings of Eurographics), Vol. 27, 2008. [20] Jr. Joseph J. LaViola and Robert C. Zeleznik. Mathpad2: a system for the creation and exploration of mathematical sketches. ACM Trans. Graph., Vol. 23, No. 3, pp. 432–440, 2004. [21] Jonathan M. Kaldor, Doug L. James, and Steve Marschner. Simulating knitted cloth at the yarn level. ACM Trans. Graph., Vol. 27, No. 3, pp. 1–9, 2008.


[22] Matthew Kaplan and Elaine Cohen. Computer generated celtic design. In EGRW ’03, pp. 9–19, Aire-la-Ville, Switzerland, Switzerland, 2003. Eurographics Association. [23] Matthew Kaplan, Emil Praun, and Elaine Cohen. Pattern oriented remeshing for celtic decoration. In PG ’04, pp. 199–206, Washington, DC, USA, 2004. IEEE Computer Society. [24] Olga A. Karpenko and John F. Hughes. Smoothsketch: 3d free-form shapes from complex sketches. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 589– 598, 2006. [25] James McCrae and Karan Singh. Sketching piecewise clothoid curves. EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, 2008. [26] Maiko Miyazaki and Katsushi Ikeuchi. Knotting rope by learning from visual examples. Technical report, IPSJ SIG Technical Reports, 2005. [27] Yuki Mori and Takeo Igarashi. Plushie: an interactive design system for plush toys. In SIGGRAPH ’07: ACM SIGGRAPH 2007 papers, p. 45, 2007. [28] D. Pai. Strands: Interactive simulation of thin solids using cosserat models. In Computer Graphics Forum, 21(3):347–352, 2002. Proceedings of Eurographics’02., 2002. [29] J. Phillips, A. Ladd, and L. Kavraki. Simulated knot tying. In IEEE International Conference on Robotics and Automation, May 2002. [30] M. Saha and P. Isto. Motion planning for robotic manipulation of deformable linear objects. In Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Orlando, FL, 2006. [31] Robert G. Scharein. Interactive Topological Drawing. PhD thesis, Department of Computer Science, The University of British Columbia, 1998. [32] C. H. S´quin. Analogies from 2d to 3d, exercises in disciplined creativity. In e BRIDGES - Mathematical Connections in Art, Music, and Science, July 1999. [33] J. Spillmann and M. Teschner. Corde: Cosserat rod elements for the dynamic simulation of one-dimensional elastic objects. In SCA ’07, pp. 63–72, 2007.


[34] Hidefumi Wakamatsu, Eiji Arai, and Shinichi Hirai. Knotting/unknotting manipulation of deformable linear objects. Int. J. Rob. Res., Vol. 25, No. 4, pp. 371–395, 2006. [35] Keith Wiley and Lance R. Williams. Representation of interwoven surfaces in 2 1/2 d drawing. In CHI ’06, pp. 65–74, 2006. [36] Lance R. Williams. Topological reconstruction of a smooth manifold-solid from its occluding contour. Int. J. Comput. Vision, Vol. 23, No. 1, pp. 93–108, 1997. [37] Ying-Qing Xu, Yanyun Chen, Stephen Lin, Hua Zhong, Enhua Wu, Baining Guo, and Heung-Yeung Shum. Photorealistic rendering of knitwear using the lumislice. In SIGGRAPH ’01, pp. 391–398, 2001. [38] Kun Zhou, Xin Huang, Xi Wang, Yiying Tong, Mathieu Desbrun, Baining Guo, and Heung-Yeung Shum. Mesh quilting for geometric texture synthesis. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 690–697, 2006.


Sign up to vote on this title
UsefulNot useful