Comprehensive gesture language NaiBoli v 0.

.1 While conceptualizing a mobile app, I realized there is not a single comprehensive notation for gesture modeling. There are few attempts which seem to be incomplete. So as usual, I have to develop my own solution. Before diving further, I like to call gesture language a name - NaiBoli. Nai (New) Boli (Dialect) While designing NaiBoli following factors were of prime concern: 1. Should be independent of device, platform, form factor, programming language, and natural language. 2. Should be easy to learn and use (manually as well as tool) 3. Gesture recognition is in its infancy and evolving at very fast pace, so new language must be evolvable. This is the first interation of NaiBoli and certainly not complete and very accurate. NaiBoli is a work in the progress. Enough fluff. Let start the real stuff. NaiBoli divides notations/symbols into following types Aadhar ( - Base) elements Tulika ( - Brush) elements Chaal ( - Action) elements Rupantran ( - Transformation) elements Parinami ( - Resultant) elements

Aadhar elements denote something on which action will take place. In future, more elements are expected like 3 dimensional spaces to cater Microsoft Kinect or Oura, occult, wrist band/watch, ring, glasses like google glass, etc. Single screen Device (Double boundary bounded area in shape of device screen) Wrist Wearable 3 Dimensional Space (Do not forget Microsoft Kinect, LeapMotion, and other gaming devices) Glasses (google glass)
1

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com

devices Ring (smarty ring, ring with three dots I am not able to recall its name)

Tulika elements denotes thing which makes gesture. It can be finger, stylus, eye, or any other body part or equipment. Pointer Tap Right palm with fingers Right eye Right eye with lid Right eye with eyebrow Right eye with lid & eyebrow Staring right eye Staring right eye with lid Staring right eye with eyebrow Staring right eye with lid & eyebrow Closed right eye Lips (closed mouth) Open mouth with teeth showing (Teeth) Body Pointer Hold Left palm with fingers Left eye Left eye with lid Left eye with eyebrow Left eye with lid & eyebrow Staring left eye Staring left eye with lid Staring left eye with eyebrow Staring left eye with lid & eyebrow Closed left eye Open Mouth with no teeth showing Tongue Body with fingers

Glasses (on right eye) Head

Glasses (on left eye) Fist

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com

Chaal elements denote action. Movement in a direction (swipe using finger/s, palm, and stylus) (At par with Tulika elements) Movement in opposing direction outward(Pinch out using fingers) Movement with pressure in a direction (drag in case of finger, palm, and stylus) (At par with Tulika elements) Movement in opposing direction inward (Pinch in using fingers)

Rupantran elements apply transformation on Tulika elements and add context to it. Rupantran elements also represent actions like drag or swipe. There are following classes of Rupantran elements. Class 1: Applied to a Tulika element only at a given point of time. These elements may attach with a Tulika element as sub script. Class 2: Applied to multiple Tulika elements. These elements glue two Tulika elements. Class 3: Applied to a Chaal element at a given point of time. These elements transform Chaal elements. Class 4: Applied to Aadhar elements. These elements to be shown out of boundary of device. Class 5: Applied to Tulika as well as Chaal elements.

Rupantran Class 1 elements No or Not (sub script to a Tulika element) Finger (sub script to a Tulika element) Palm (sub script to a Tulika element)

!
F P

Physical touch (sub script to a Tulika element) Stylus (sub script to a Tulika element)

Nothing or

Rupantran Class 2 elements sequential gesture separator (At par with Tulika elements)

Split separator(At par with Tulika elements)

/
3

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com

To (e.g. move from point A to B in 2 to 3 seconds) (At par with Tulika elements)

Rupantran Class 3 elements Fast (sub script to a Chaal element)

Slow (sub script to a Chaal element)

Rupantran Class 4 elements Device Tilt

or

Device Shake

Device movement in a particular fashion (e.g. move device in shape of 8 to calibrate its compass)

Device Hold during tilt motion

Rupantran Class 5 elements number of times (numeral) (At par with a Tulika element or sub script to a Chaal element) Logical OR (At par with or sub script to Tulika elements) e.g.: Go to store and bring milk or juice.

Time (seconds) (At par with a Tulika element or sub script to a Chaal element) To (e.g. move from point A to B in 2 to 3 seconds) (At par with Tulika elements)

sec

min

Parinami elements are resultant of application of Rupantran elements on Aadhar, Tulika or Chaal elements. Lets explore some of the examples of Parinami elements Action Tap and then tap again using finger Double Tap using finger Symbol Remarks

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com

Double Tap using stylus 3 finger swipe Tap using one finger, hold for 2 sec and then drag Tap using two finger, hold for 2 sec and then drag Tap using stylus, hold for 2 sec and then drag Tap and hold, then drag and then tap elsewhere using fingers Tap and then slow swipe using stylus Tap and then swipe for 1 second using stylus Tap and then swipe for 1 second using stylus Tap in air using figure and then swipe wink right eye 2 to 3 times Blink both eyes 2 times Close both eyes with force Protruded Lips Smiling lips Bilking eyes Winking right eye Shaking head in YES motion Shaking head in NO motion Moving head in circular fashion ( clockwise) Moving tongue up and down Tilting rectangular device (think of phone) between 10 to 30 degree and holding there
5

Do not forget Samsung Galaxy S4

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com

Tilting rectangular device (think of phone) between 10 to 30 degree then holding there for 2 to 5 second and bringing back to original position

NaiBoli is shared under common creative license. http://architecture-soa-bpm-eai.blogspot.com