Building Tablet PC Applications with Visual FoxPro

By Markus Egger, President and Chief Software Architect, EPS Software Corporation

Abstract
This whitepaper introduces the Tablet PC and discusses developing applications using
Microsoft Visual FoxPro to target the Tablet PC.
Contents

Introduction .............................................................................................................................3
So What Exactly is a Tablet PC?............................................................................................5
The Software ............................................................................................................................7
The Pen ....................................................................................................................................7
Developing for the Tablet PC .................................................................................................8
The InkOverlay Class ............................................................................................................11
The InkPicture Control ..........................................................................................................12
Recognizing Ink .....................................................................................................................16
Building a Real Application ..................................................................................................18
Conclusion .............................................................................................................................21

2 Building Tablet PC Applications with Visual FoxPro

Introduction
Last fall, Microsoft introduced an intriguing new hardware platform designed to run a superset of
Windows: The Microsoft Tablet PC. While this platform isnt going to revolutionize the PC world
overnight, it certainly offers some great features that are likely to be implemented by a large number
of vendors and device types in the near future. In the meantime, Tablet PCs are great for a number
of target groups, from managers, to sales personnel, and pretty much everyone who needs full PC
power while being mobile.

Of course, Tablet PC features don't just magically show up in regular applications (not at any level
of sophistication anyway). Therefore, developers need to specifically support Tablet PC features
such as "Digital Ink." Luckily, this task is made relatively simple using Microsoft's Tablet PC SDK.
Visual FoxPro Developers can use this SDK to Tablet-enable various areas of applications, from
data entry, to annotation, to ink collection (perhaps to collect signatures or the like).

So what exactly is a Tablet PC, and why would we want one? Well, for one, a Tablet PC is well
a tablet. It is flat. While this is the most obvious feature, it is not the most significant one. These
honors go to Pen Input. While this doesnt really require the device to be flat, it certainly provides a
much more natural experience. After all, Microsoft is trying to simulate a pen & paper environment
based on a digital platform. There are a number of reasons why this would be appealing. For one,
tablets provide great mobility. Unlike regular laptop computers, Tablet PCs can be carried around
resting on one arm, while they are operated with the other hand using mostly a pen. A clear
advantage over laptops, which are mobile, yet they usually need a surface to sit on (such as a lap
or a table) to free up both hands for input. Envision a nurse walking around the hospital trying to
take notes while balancing a notebook computer on the foot-board of a patients bed.

But there are more general uses for Tablet PCs. How many meetings have you attended lately?
The answer here is usually too many. Tablets are great in meetings, since their use generates a

EPS Software Whitepaper 3

much more natural meeting atmosphere than the meetings we have grown accustomed to over the
recent years, where almost all participants hide behind there notebook screens annoyingly tapping
their keys, and pushing other peoples buttons at the same time.

And from the developers point if view. Tablet PCs are just enormously cool. Personally, I have
wanted a tablet, ever since I saw my first Star Trek episode. On a more serious note, I use my
Tablet PC for reading, taking notes in meetings, doing code and interface reviews, article writing,
and even to program.

Those last two items may surprise you. Am I writing Visual FoxPro code with a pen these days?
Certainly not. One often overlooked fact is that Tablet PCs are regular notebook computers,
providing a keyboard, hard drive, CD Rom drives, and everything else you would expect a notebook
computer to have. You can imagine these devices as notebook computers with a fancy display that
can swivel around and be folded up with the display pointing to the outside:

Figure 1: Many Tablet PCs are convertibles that can transform from traditional notebook computers into
Tablet PCs in a single swivel.

I cannot stress this point enough: Tablet PCs are general purpose PCs. They simply come with
some extra software and a fancier screen.

4 Building Tablet PC Applications with Visual FoxPro

Note that there are also other form factors. Some tablets are just tablets. All the time. They are still
general purpose computers, meaning that they operate a regular Windows operating system and
have as much processing power, memory, and hard drive space as any other PC, but they do not
have a keyboard built in (although one can always hook up an external keyboard). But I would
consider these computers more of a specialty device. While pure tablets are great for certain
professions, most users are generally better of with a convertible.

Figure 2: A pure Tablet PC

Of course Tablet PCs arent the industrys first venture into the world of Pen Computing. Several
years ago, products such as the GRiDPad or Go PenPoint tried to establish a market for
themselves, later followed by Windows for Pen Computing, which was a Windows 3.x based pen
operating system and Microsofts first steps in the field. Unfortunately, none of these ventures ever
really got of the ground, mainly due to hardware restrictions. Devices were either to heavy, not
powerful enough, or both. Displays had low resolution and were rather dim making them hard to
read. And handwriting recognition was practically useless partly because algorithms simply werent
up to the task, but primarily due to a lack of processing power.

The first useful pen-based devices came to us in the form of PDAs. These devices are still great
today when it comes to being able to carry around a PIM in your shirt pocket. However, they still
have major limitations. Screens are not as great as they should be and the resolution leaves much
to be desired. More limiting however is that these devices arent general purpose computers. In
other words, while there are specialized versions of Microsoft Word and Outlook available, they are
not the same programs. If you wanted to install a regular version of Microsoft Visio, for example,
youd be entirely out of luck.

So What Exactly is a Tablet PC?

While there are a number of devices that fit the general description of a tablet pc, there are very
detailed specifications that define whether device can consider itself a true Microsoft Tablet PC. I
do not want to bore you with all the details here, but I do want to point out some of the more
interesting requirements.

For one, a Tablet PC has to have a digitizer. Thats the scientific term for the tablet that makes up
the screen. The digitizer is used to recognize the movement of the pen on and above the display.
The digitizer has to be rather sophisticated, and support a resolution much higher than the display.
This is important for a sophisticated handwriting experience. After all, pen & paper do not have
resolution limitations. The digitizer resolution must at least be 600dpi or 5-times the display
resolution (whichever is higher). For a typical 1024x768 display, this works out to a 615dpi sampling

EPS Software Whitepaper 5

resolution. This is quite significant when you consider the amount of data that is to be tracked.
While a 1024x768 display features some 785,000 pixels, the digitizer resolution works out to over
28 million pixels (points). On top of that, the digitizer also needs to be fast. A sampling rate of 100
times per second is the minimum. Most tablets sample 133 times per second.

Contrary to popular believe, the digitizer is not touch-sensitive. Instead, Tablet PCs use
electromagnetic digitizers. This means that you can touch the display with your hand or fingernail,
and you will not get a reaction from the device. So dont loose your pen! The electromagnetic
digitizer provides a number of advantages. For one, the digitizer can be mounted behind the
display. This makes the display much brighter than on similar devices. Also, it allows vendors to
add protective glass on top of the display making devices more rugged. More importantly, since the
digitizer uses magnetism to track the pen, it can sense the pen before it touches the display. At a
minimum, the pen has to be recognized 5mm above the glass, but most tablets recognize the pen
at about twice that distance, which translates to almost half an inch. The digitizer can also sense
the orientation (angle) of the pen and tell which end of the pen is pointed towards the display. Some
tablets can even sense the rotation of the pen, which allows for some interesting features in 3D
environments. Oh, and before I forget it, most digitizers are pressure sensitive. So if you want a fine
line, apply little pressure, and if you want a thick line, hold down a bit harder.

But there is much more. For one, Tablet PCs when used in tablet mode are generally operated
in portrait mode, rather then the usual landscape orientation:

Figure 3: In tablet-mode, most users prefer portrait orientation.

Portrait mode is very natural for handwriting uses, but it is also great to read Web pages and
eBooks. Tablet PCs have to support orientation switching on the fly without requiring a reboot.
Which leads us to the next feature: Being available quickly is a key feature for Tablet PCs. After all,
with pen & paper you wouldnt tell your fellow meeting attendees hold on for 5 minutes, while I
boot up my pencil. Therefore, Tablet PCs have to come out of sleep mode and be fully operational
with applications back up and running in 5 seconds or less. And to make sure we can use sleep
mode as often as possible, it is required that these devices can remain in sleep mode for 72 hours
on a single battery (4 hours in fully awake mode).

Also, Tablet PCs should be light. Mine is just under 3 pounds, which is incredible when you think
about it. After all, display and battery requirements are rather high, and these two components
account for most of the weight these days. On the other hand, if you ever do a presentation about
Tablet PCs and you cradle one of them on your left arm for an hour and 15 minutes, you realize that

6 Building Tablet PC Applications with Visual FoxPro

3 pounds can become pretty heavy. The first time I tried that and then sat down to type some
sample code in front of the audience, I realized that the quality of my left-hand-typing had dropped
below the quality of the handwriting recognizer. (I wonder whether I should renew my gym-contract
or work on my penmanship instead). Similar effects can be observed when trying to hold up a
tablet like a magazine while lying on the couch. On the other hand, you can use the effect to your
advantage by taking your tablet to the gym and reading your favorite Web site on an exercise bike.

The Software
Of course the hardware is only half the fun without the appropriate tablet OS. For Tablet PCs (as
opposed to other tablet-like devices), the operating system is Windows XP Tablet PC Edition. This
is Windows XP as you know and love it with some extra features and a few new applications. The
heart of the additions is Inking functionality. This is the software that takes the results of the
digitizer and draws them on the screen. But thats just the start. Not only is Ink drawn on the screen,
but Ink is considered a first class citizen of Windows. This means that once the ink is drawn, you
can manipulate it like everything else in Windows. You can copy and paste it, change its color,
erase it in full or in part, resize it, and much more. On top of that, the OS has the ability to recognize
Ink as handwritten text, hand-drawn objects (such as musical notes or geometric shapes), formulas,
and gestures (no not what you are thinking).

Windows XP Tablet PC Edition also ships with a few new applications such as Windows Journal (a
note-taking application), Sticky Notes, and the Tablet PC Input Panel. The later is a replacement for
a keyboard, allowing you to hand-write on a panel. The handwriting then gets recognized and fed
into whatever application you operate, making every application available to pen-based operation
(although with a somewhat limited user experience). The Input Panel also features a soft
keyboard allowing you to tap individual keys in the rare cases when you really cant get the darned
thing to recognize your scribbles.

The Pen
The Tablet PC input Pen is a rather intriguing device. Simple at first (after all, it appears to be a
simple stick with a pointy end), but rather delicate when you think about it: The Pen combines the
functionality of the mouse and the keyboard and then adds all the inking and gesture features on
top of that. At the same time, the Pen has to be quite durable. I cant count the number of times I
have dropped mine. Also, I hardly ever chew on my mouse, while the pen on the other hand well,
Im doing it right now.

So lets take a look at the pens features one by one. First of all, there is mouse-emulation. As the
pen moves over the display, the digitizer simulates mouse-input by feeding that information to the
Windows mouse driver (with pen coordinates translated to regular mouse coordinates). The input
rate here is much higher than with a regular mouse, not only because of the high sampling rate, but
also because the pen never stays still. When a user lets go of the mouse, there is no mouse-input.
It just sits there. The pen however is always in motion, even when the user tries to hold it still, and
very often, even if the user points the pen away from the panel (as it is often the case while the user
reads whatever is on the display), the pen is often still in range for the digitizer to recognize it.

The pen can also simulate a click. This is done by simply taping the display. A double-click is
accomplished with a double-tap. Quite intuitive. Many pens also have a barrel button, which is a
button on the pen that can be used for right-click operations. This however takes a while to get used
to (depending on the exact position of the button). Personally, I find the right-mouse gesture more
useful: Tap the screen and hold for a brief moment until the mouse-pointer changes shape and the
right-click menu appears.

EPS Software Whitepaper 7

This enormous amount of input as well as the great precision also leads to some problems. Getting
a tooltip to appear for instance can be difficult for novice tablet users. Clicking on an icon without
engaging in a click & drag operation can be hard. A similar phenomenon is often observed with
double-click (double-tap) attempts. The software tries to counter these issues by applying input
filters to electronically steady the users hand. It does a good job at that, but sometimes not good
enough and some extra practice is required.

The second major feature the pen provides is Ink input. This information is internally funneled
through entire different operating system channels than mouse input, as the mouse interface is
entirely inadequate for inking needs, because it is much too slow and inaccurate. For instance, ink
needs to be transferred from the recognized screen position of the pen to being displayed on the
screen very rapidly to emulate a real pen. After all, how weird would it be, if you used a pen on a
piece of paper but the line you draw lags half an inch behind your movement. This is rather
annoying and makes it almost impossible to generate real handwriting input. For this reason
(among others), inking is handled by its own set of algorithms and drivers. For users, this doesnt
matter. For most developers, it doesnt matter either except for very advanced applications where
ink as well as mouse recognition has to occur in a very time-critical fashion. In that case, there is no
guarantee whether mouse or ink information will be fed into the system first and at what intervals
and delays. However, this level of sophistication is beyond the scope of this paper.

Once the ink is in memory, it is up to the application to decide what to do with it. Most applications
store the actual ink information. Many applications try to recognize the ink as either text or
drawings. Almost all applications support some type of gesture recognition. At the very least, most
apps support gestures such as down and left or straight-left, which simulate the ENTER key and
back-space respectively. This allows the pen to be used as a full-featured keyboard replacement.

While features such as mouse-emulation are provided automatically, ink input, recognition, and
handling are not (except for the Input Panel, which we will ignore for now due to its unsatisfying
user experience level). Therefore, the subject of Digital Inking is whats most interesting to
developers. Luckily, Microsoft has released the Tablet PC SDK, which enables developers to make
their applications fully ink-aware.

Developing for the Tablet PC

You can download the Microsoft Tablet PC SDK from http://msdn.microsoft.com/TabletPC. The
SDK includes APIs in many different flavors. Anything from plain Win32API (C++ developers), to
.NET managed code, and COM interfaces are supported. For Microsoft Visual FoxPro (VFP)
developers, the COM version is most interesting. The COM API has been developed mostly with
VB6 in mind (surprise, surprise!). Unfortunately, this introduces some limitations for VFP
developers, but most of them are not too severe, and the API is quite useful and simple to use in
VFP.

What else do you need to develop for this platform? Well, a Tablet PC sure would be handy,
although you can install the SDK on any XP machine and use the mouse to simulate input to a
certain extent. Unfortunately, this doesnt work with all the provided objects. Especially the ActiveX
controls seem to be quite picky when it comes to these things. Also, drawing text with the mouse is
rather difficult (you can try it in MS Paint), so dont be surprised if the handwriting recognizer fails
miserably. You can buy digitizer peripherals for your PC which solves some of these issues, but I
prefer to use my Tablet PC for development (hook up an external keyboard and maybe even
another monitor). Using environments such as Visual Studio .NET, you also have the option to
develop the application on a desktop PC, but then run and debug the app on a Tablet PC remotely.

8 Building Tablet PC Applications with Visual FoxPro

Using Visual FoxPro, remote debugging is not an option. On the bright side however, VFPs
development environment isnt nearly the resource hog Visual Studio is, which makes it feasible to
develop on the actual device.

So assuming you have the SDK installed on your Tablet PC, we are ready to create our first ink-
enabled application. The COM SDK provides a number of COM objects, as well as a pair of ActiveX
controls. The simplest way to make your application ink-aware is though a control known as the
InkEdit Control. This control is almost identical to the RichEdit Control except for added ink
recognition functionality. It can be used as a replacement for VFPs textbox and editbox controls. To
use the control, simply drop in on a form and position it like you would with a regular textbox, with
one exception: Make the control a tad larger to provide some space to write into, since your users
will be inking straight into the control. This is it! Run your application, and try to write into the control
using your pen:

Figure 4: A straightforward inking example using InkEdit controls.

As you handwrite into the control, it will automatically try to recognize the handwritten text after a
few moments, and paste the result into the textbox (the RecognitionInterval property specifies the
delay). The ink then disappears (the previous screen capture is a trick shot showing the ink and
the recognized result in the first textbox). At this point, you are dealing with a regular RichEdit box
that could almost be mistaken for a VFP textbox.

Note that the recognition is very good, but not perfect. If the recognition was incorrect, right-click on
the recognized word (tap and hold with the pen) to bring up a smart tag with alternatives and
correction options:

EPS Software Whitepaper 9

Figure 5: The InkEdit control automatically handles alternate text recognition results.

Note that in addition to alternates, the SmartTag also has an option to store the actual ink (perhaps
the ink wasnt a word but some drawing insteador perhaps your handwriting is so terrible it cant
come close to recognizing what you typed and youd just like to move on and fix the problem later
when you have a keyboard). Alternately, you can get rid of the word altogether and rewrite it. This
is rather handy, because you do not have a backspace or delete key. Therefore, your only other
option is to gesture backspace though the entire word, which will probably cause you to either give
up, bite into your pen, or both.

Recognizing the ink as text is the default behavior. Alternatively, you can also store and display the
ink. To do so, change the InkInsertMode property from 0 to 1. This will keep the ink as ink, although
with one somewhat peculiar feature: The control always assume you want your ink insert at the
same point-size as the text size of the control. Therefore, the ink usually is shrunk down beyond
recognition. To fix this, change the text size of the control using the SelFontSize property. This
property is write-only and can only be set through code. Also, the font size only changes the font
size of the selected text. This means that we need to make sure all the text in the textbox is
selected, if we want to keep it at a constant size (the control supports advanced formatting of text
with different sizes and colors). Therefore, put the following code in the GotFocus() event of the
textbox:

THIS.SelStart = 0
THIS.SelLength = Len(THIS.Text)
THIS.SelFontSize = 36

This works OK, but to be honest it isnt great. One of the main problems with this approach is that
the ink is rendered at a rather low quality. The following image shows the result:

10 Building Tablet PC Applications with Visual FoxPro

Figure 6: The InkEdit control allows you to store ink rather than recognized text, but the quality of the ink is
rather poor.

Once you have gathered input (ink or recognized text), you will have to access the provided data to
store it away or manipulate it. You can do so easily through the Text and TextRTF properties. As
the name suggests, these provide access to unformatted text as well as the complete rich text
information (RTF), which is compatible to binary information generated by Microsoft Word and
WordPad. If you want to store format or ink information, use TextRTF. You can take that value and
store it in files or table fields, and later re-assign it to the property to reload the text.

There are a number of other properties you can set on this control that influence ink behavior.
Unfortunately, this control doesnt provide great flexibility, but at least it is a quick way to enable ink
and pen-input support in your app. For more advanced uses, there are better controls and we will
move on to those instead of spending more time with the InkEdit control.

The InkOverlay Class

While the InkEdit Control is a highly abstracted control designed to be easy to use, the InkOverlay
class is the opposite: it is the low-level interface to the inking system. It represents the preferred
way to implement ink input, granting you access to all the inking functionality down to individual
strokes of ink and even the individual points comprised in each stroke. You can implement ink
manipulation and low-level ink recognition. As a VFP developer, you are not likely to write custom
ink recognizers. You may want to be able to provide standard Ink functionality, such as selecting
and deleting ink, or simple ink manipulation, such as resizing or re-coloring strokes. On top of that,
this class enables you to handle and store ink at a very high level of quality, making use of all the
information provided to you by the high-resolution digitizer. In fact, not only do you get information
about stroke coordinates, but you can even influence things like stroke thickness, and whether you
want your application to be pen-pressure sensitive.

So how do we use this control? The concept is easy. You instantiate the InkOverlay COM-class and
attach it to the window-handle of a VFP form. This can be accomplished like so:

oForm = CreateObject("Form")
oForm.BackColor = Rgb(255,255,255)
oForm.Show()

oInk = CreateObject("MSInkAut.InkOverlay")
oInk.hWnd = oForm.hWnd
oInk.Enabled = .T.

EPS Software Whitepaper 11

As you can see, this is quite straightforward. The only trick is the assignment of the window
handles. Starting with version 7, VFP exposes a property called hWnd on all forms, granting us
access to the forms window handle. The only other tricky part here is that the InkOverlay is disabled
by default. Therefore, we need to enable it to activate inking, which makes the whole form ink-
enabled. The following image shows the result:

Figure 7: Using the InkOverlay class we can enable inking on entire forms.

While this works very well, I also have to point out a limitation of this approach: In Visual FoxPro,
we can only attach the InkOverlay class to forms since they are the only objects that expose (or
have) window handles. This is not the case in most Windows applications where practically all
controls are implemented as separate windows (although they do not look like it). Visual FoxPro
does not use a windowed approach for generating controls. Instead, controls are painted on the
background of the form (this behavior is a reminder that VFP once used to support the Macintosh
platform). This prevents us from attaching the InkOverlay class to controls such as textboxes or
containers as we could in other environments. This is unfortunate because only in very rare
occasions do we want the whole form ink enabled. (Even if we did we would likely want toolbar
buttons or some other controls on the form that we wouldnt want to ink over.)

The InkPicture Control

The only way around this is the use of another ink control called the InkPicture control. This control
is really designed as an annotation control allowing us to load an image in the background and then
ink over it. Luckily for us VFP developers, this control simply encapsulates the InkOverlay control,
giving us access to all its features. Attaching a background image is optional, allowing us to use this
control as an empty input area on a form. This makes this control the prime component for ink-
applications in VFP.

Using the InkPicture controls basic functionality is almost trivial. Simply drop it on a form and you
are ready to go. (Note: To follow along with the examples further down, name your object
oInkPict). By default, the control accepts ink input using black ink of medium thickness in a
pressure-sensitive way, but you can change all of that. For instance, you can add a simple button to
your form to change the ink color. The color is set on the DefaultDrawingAttributes member object
of the ink control:

THISFORM.oInkPict.DefaultDrawingAttributes.Color = GetColor()

12 Building Tablet PC Applications with Visual FoxPro

The DefaultDrawingAttributes object is rather interesting. It enables us to set attributes such as pen-
tip (round or rectangular), default line width and height, whether pen pressure should be ignored,
transparency, line rendering quality, ink type (regular or highlight ink) and so forth. Exploring all the
options is beyond the scope of this article, but I encourage you to experiment with these settings.

The InkPicture control supports 3 different interaction modes: Ink collection, eraser, and selection
mode. By default, the control is set to collect ink, but we can easily change that by changing the
EditingMode property of the control. The following image shows a simple VFP form that implements
this behavior:

Figure 8: The InkPicture control supports high-quality, multi-color inking. It also supports edit, erase, and
select modes.

As we change the drop-down list to change the behavior of the form, we will see the interface
change as demonstrated in this screen shot:

EPS Software Whitepaper 13

Figure 9: from top-left clockwise: Editing, erasing, lasso selecting, and working with a selection.

The InkPicture control also grants us access to the data collected by the control. Each InkPicture
control has an Ink object that encapsulates all the ink information contained within the control. Most
interesting for us is the Ink objects Strokes() collection, which represents the actual data. Each
individual stroke of ink (and all the individual points it is composed of) are stored in this collection.
We could use this collection to perform some low-level ink recognition by analyzing the data. In fact,
the API provides some functionality to make this task easier. For instance, there are ways to
retrieve stroke information as Bezier Spleens. We can also retrieve individual Cusps for each
stroke. Cusps are significant sections of a stroke. For instance, we could draw a triangle with a
single stroke, but there would be three cusps. As Visual FoxPro developers however, we are more
interested in collecting data and are not likely to create low-level ink recognition algorithms.

However, there are other ink-manipulation and ink-management options we might be very
interested in. Deleting ink programmatically to clear the display is a frequently required feature. This
can be accomplished using the Ink objects DeleteStrokes() method. The strokes that are to be
deleted have to be passed to the method in the form of a collection. This allows us to delete strokes
very selectively. Often, we may have the need to delete the selected strokes. In that case we can
simply pass a collection of the selected strokes to this method. Another option is to delete all
strokes. Heres a code example that demonstrated that behavior:

THISFORM.oInkPict.InkEnabled = .F.
THISFORM.oInkPict.Ink.DeleteStrokes(THISFORM.oInkPict.Ink.Strokes)
THISFORM.oInkPict.InkEnabled = .T.

Note that we need to disable inking on the control before we can perform the operation. This is
important because we want to make sure we do not start the delete operation while the user is still
drawing text. This could lead to some very nasty behavior. After the deletion, we re-enable the
inking, which not only allows the user to create more input, but it also forces a refresh of the control.

14 Building Tablet PC Applications with Visual FoxPro

Note that setting the InkEnabled property to .F. may actually fail if the user is in the middle of
drawing a stroke. For this reason, we should wait until the control is done collecting ink, which is
indicated by the CollectingInk property:

DO WHILE THISFORM.oInkPict.CollectingInk
* Do Nothing...
ENDDO

THISFORM.oInkPict.InkEnabled = .F.
THISFORM.oInkPict.Ink.DeleteStrokes(THISFORM.oInkPict.Ink.Strokes)
THISFORM.oInkPict.InkEnabled = .T.

This is a polling mechanism, and polling mechanisms are never very elegant. Unfortunately, this is
the only way to get around this issue.

Another reason for accessing the Ink object is to store and load ink. The Ink object has a Save()
method that can save the collected ink in two different ways with a number of sub-variations. There
is the native ink format, as well as the ability to store a GIF file. On top of that, each format can be
compressed at different levels. Also, the ink object has the ability to Base64 encode both GIF
formats as well as native ink format. This feature has been introduced to simplify transfer of ink data
using XML formats.

While GIF export is a nice feature to have, we are mostly interested in the native ink format. Heres
the code that retrieves the ink and puts it into a local variable:

LOCAL lcInk
lcInk = THISFORM.oInkPict.Ink.Save()

Calling the Save() method without parameters retrieves the ink information in its native format. If
you wanted to store a .GIF file, you would pass a 2 as the first parameter.

Loading ink is a bit trickier, although many people do not realize that. The ink object features a
Load() method. Using that method is straightforward. All we have to do is pass the ink as the only
parameter. Unfortunately, Visual FoxPro appears to have problems handling the ink in its native
format, when it is stored in a local variable. Visual FoxPro performs some conversions on the data
behind the scenes that renders the stored ink invalid. In order to prevent this behavior, you need to
use the CreateBinary() method to make sure the ink format is kept intact. The method is then smart
enough to figure out what format is used. The tricky part however is that this method only works if
no ink is in the control. In fact, no ink must have ever been in the control for this method to work.
(This has to do with the inner workings of the ink object and the ID assignment of each individual
stroke). For this reason, many developers fall into the trap of creating a load mechanism that works
only once.

The way to get around this problem is to instantiate a new ink object every time we perform a load
operation. The ink object is exposed as a COM class that can be instantiated in Visual FoxPro and
then simply be assigned to the ink reference of the InkPicture control:

LOCAL loInk AS MSInkAut.InkObject

loInk = CreateObject("MSInkAut.InkObject")
loInk.Load(CreateBinary(lcInk))

EPS Software Whitepaper 15

THISFORM.oInkPict.InkEnabled = .F.
THISFORM.oInkPict.Ink = loInk
THISFORM.oInkPict.InkEnabled = .T.

Voila! You have just learned how to avoid one of the most common ink management traps.

Oh, and before I forget it, the InkPicture control can show a background image, which is very helpful
if you like to create annotation functionality. Images are assigned to the Picture property of the
control. To accomplish this, you need to have a reference to the image in object form. Visual
FoxPro makes this easy through the use of the LoadPicture() method:

THIS.oInkPict.Picture = LoadPicture("xray.jpg")

The following screen shot shows the result:

Figure 10: The ability to load background images into the InkPicture control is very handy in annotation
scenarios.

Recognizing Ink
Of course, in most scenarios, ink does not do us any good as undecipherable scribbles. We need
the ability to turn handwriting into strings. Luckily, the Tablet PC SDK provides a number of different
ways to accomplish just that.

The simplest way is to use the ToString() method of the Strokes() collection:

WAIT WINDOW THISFORM.oInkPict.Ink.Strokes.ToString()

16 Building Tablet PC Applications with Visual FoxPro

This will return the top alternate. This is the string the recognizer is most confident to have
recognized. This is often exactly what you need and therefore usually adequate. However, you can
also go a step further and instantiate a recognizer object that allows you to take a closer look at
alternates and confidence levels. Creating a recognizer is a three-step process. First, the
Recognizers object (plural!) needs to be instantiated. This object gives us access to all the installed
recognizers. By default, you will have at least an English Handwriting recognizer (or whatever your
primary language is) as well as a Gesture Recognizer installed. On top of that, there might be
other recognizer use to recognize objects, formulae, musical notes, and so forth. For text
recognition, it is usually appropriate to ask the Recognizers object to return a reference to the
default recognizer using the GetDefaultRecognizer() function. On top of that, we need to obtain a
reference to our very own recognizer context, which is our recognition session that we do not share
with anyone else. Heres the code:

LOCAL loRecognizers AS MSInkAut.InkRecognizers

LOCAL loRecognizer AS MSInkAut.IInkRecognizer
LOCAL loContext AS MSInkAut.IInkRecognizerContext

loRecognizers = CreateObject("MSInkAut.InkRecognizers")
loRecognizer = loRecognizers.GetDefaultRecognizer()
loContext = loRecognizer.CreateRecognizerContext()

Whenever I need a recognizer context, I usually put this code into the Init() event of my form. This
allows me to store the context in a property on the form, where I can access it any time I need it
rather than re-creating it every time I need to recognize some text.

We are now ready to recognize text. The recognizer context operates based on strokes collections.
If we want to recognize all text in an InkPicture control, we can simply assign its Strokes collection
to the recognizer context and indicate that we do not expect further input:

loContext.Strokes = THISFORM.oInkPict.Ink.Strokes
loContext.EndInkInput()

The reason we need to call EndInkInput() is that we could dynamically assign additional strokes to
the recognizer, as the user writes more text. This allows for background recognition, which can be
very important if we attempt to recognize a lot of text, which can be a very time consuming task. If
we only want to recognize a few words or sentences however, we do not have to worry about this.

Retrieving the result is a bit more sophisticated than using the ToString() method. When we use a
recognizer context, we use the Recognize() method, which returns a RecognitionResults object as
well as a status code that is implemented as an out-parameter. Once we have the results object, we
can do a number of things. The first step usually is to look at the top alternate (TopAlternate object),
which provides us the string we are looking for, as well as a confidence level and LineAlternates.
Line alternates deal with spacing issues. For instance, line alternates might be Visual FoxPro,
VisualFoxPro, and Visual Fox Pro.

LOCAL loResults
LOCAL lnStat
loResults = loContext.Recognize(@lnStat)

? loResults.TopAlternate.String

EPS Software Whitepaper 17

? loResults.TopAlternate.Confidence

LOCAL lnCounter
FOR lnCounter = 1 TO loResults.TopAlternate.LineAlternates.Count
? loResults.TopAlternates.LineAlternates(lnCounter-1).String
ENDFOR

The confidence level will indicate how confident the recognizer is to have recognized the text
correctly. If the confidence level is low, we may want to take a closer look at other variations. We
can do so using the AlternatesFromSelection() method, which returns a collection of alternate
RecognitionResults objects:

LOCAL loMoreResults
loMoreResults = loResults.GetAlternatesFromSelection()
FOR lnCounter = 1 TO loModeResults.Count
? loMoreResults.Item(lnCounter-1).String
ENDFOR

Of course, recognizers arent perfect, and when you really think about it, how could they? I very
often cant read my own handwriting, and although the Tablet PC recognizers seem to do a better
job at reading my handwriting than I do myself, there will be some disastrous recognition attempts.
Often, we can increase the chances for successful recognition by providing the recognizer some
hints about the type of text we are expecting. For instance, we might be trying to recognize an email
address, a phone number, a Web address, or a numeric value. We can tell the recognizer about
this using the Factoid property. There are a number of different default factoids recognizers support.
We can set factoids by simply assigning a keyword to the Factoid property:

loContext.Factoid = "NUMBER"

Or:

loContext.Factiod = "EMAIL,NUMBER"

For a complete list of factoids supported by your recognizer, please consult the recognizer
documentation or the Tablet PC SDK documentation (for the default recognizers).

Building a Real Application

Based on the knowledge we have accumulated so far, we can now attempt to build a real Tablet PC
application. I envision an inventory management application for a user that works in a warehouse
and has to walk around looking at items, update the number of items in stock, and order more
goods. The following image shows that application in action:

18 Building Tablet PC Applications with Visual FoxPro

Figure 11: An inventory management Tablet PC application.

This application uses sample data that ships with Visual FoxPro (the Products table). I simply
added that table to the forms data environment and subsequently used a grid to display the tables
contents. But I wont bore you with those details here, since I assume that you understand how VFP
uses data.

What is more interesting, however, are the three ink-areas I have added to this form. These are
three InkPicture controls. To provide a better hint to the user about the inkable areas of the form, I
load a background image into each one of them. I believe the most obvious way to show that an
area of the form is inkable is by using the same background gradient rectangle the Input Panel
application uses. Note however that there are no established interface guidelines for Tablet PC
applications (yet). So this is just something I thought was a good idea, but other people may
disagree.

The way this form works is quite simple: The user can handwrite the name of a product in the
topmost ink area. The system will then recognize the text and search for the product in the
database and display it in the grid. If the list of products is too long, the user can write more text. If
the recognition failed, or the user wants to search for a different product, she can clear the ink area
using the Clear button at the top.

Once a small list of products has been selected, the user can pick one of the products from the grid
(or just stick with the first one that came up) and handwrite the number of actual items in the

EPS Software Whitepaper 19

bottom-left ink area as well as the number of items that are to be ordered in the bottom-right ink
area. Both ink areas at the bottom will attempt to recognize a number immediately, and clear the ink
once recognition has occurred. If the recognition failed, the user can simply re-write the number.
Once satisfied with the recognition results, the user can click Update Values to submit the new
information to the database.

Implementation of this application is surprisingly simple. You have already seen how to add
InkPicture controls to a form and load background images. You also know how to clear the ink, and
I am sure you can imagine that the Find button simply uses the ToString() method of the first
control to retrieve the desired list of products. However, the first ink area not only searches for
products when we press Find, but it will do so on an ongoing basis by starting a recognition attempt
half a second after the user is done writing. We can implement this behavior using a timer. This
timer has to be disabled by default, and its Interval is set to 500ms. The timer simply calls the
following method to retrieve the recognized text and subsequently the matching data.

LOCAL lcProduct
lcProduct = THISFORM.oInkSearch.Ink.Strokes.ToString()
* query the data here based on the product name
THISFORM.Refresh()

We can then use the InkPicture controls Stroke() event to enable the timer. This event fires every
time the user has completed a new stroke. Of course, the user might continue writing more text for
more than half a second after the first Stroke() event occurred. Therefore, we reset the timer every
time that event occurs, hence making sure it really only fires whenever the user pauses for at least
half a second. Heres the code we put into the Stroke() event handler:

LPARAMETERS cursor, stroke, cancel

THISFORM.tmrSearch.Enabled = .F.
THISFORM.tmrSearch.Enabled = .T.

Note that it doesnt matter if the user decides later to write more text, since this just re-triggers the
timer, and the ToString() method always returns the entire content of the control. This enables the
user to narrow down the selection by writing more text.

The two ink areas are the bottom are implemented similarly, but with some significant differences.
For one, we know that the user will write a numeric value. Therefore, we can provide the
NUMBER factoid. This increases recognition accuracy drastically, and it is almost impossible to
write the number so sloppy that the recognition would fail. The other difference is that we clear the
ink after recognition is complete, allowing the user to re-write a few numbers in case recognition
failed, rather than making them click a Clear button or have her try to scribble more into the control
to enhance recognition results. This wouldnt be very user-friendly for simple number entry.

The fact that we clear the ink after recognition means that we need to give the user a bit more time
before we start recognition. Otherwise, slow writers may see recognition triggered in between
writing 1 and 5 in the number 15, resulting in incorrect and frustrating results, putting the user
under time constraints that are not productive for them. For this reason, I set the timer interval to 2
seconds, which is the default the Input Panel uses. Personally, I think thats a little slow. A good
solution would be to make this an application option.

20 Building Tablet PC Applications with Visual FoxPro

Thats it! We have written our first Tablet PC application. Note that the application is pretty simple,
but not too far from a real-world implementation. When I wrote this application, I was quite surprised
myself how good the user experience was. There sometimes is some incorrect recognition, but it
really doesnt result in a bad experience since the application provides great fault tolerance and
recovery features by allowing the user to re-write and add to their search criteria.

Conclusion
Tablet PCs represent a fascinating platform, and the good news for developers is that it is rather
straightforward to write software for them. I do not think that Tablet PCs will revolutionize the
industry over night. But keep in mind that Tablet PCs are general purpose portable PCs. I really do
not see why future notebook computers wouldnt be equipped with digitizers added to their screens,
and allow those screens to be swiveled around, which is practically all it takes to provide tablet
functionality. I would expect to see this more and more, especially as all notebooks are constantly
getting lighter and thinner, making them more tablet-like anyway. And there is no downside to
providing tablet functionality. Even prices seem to be comparable.

Let me know what uses you have come up with for your Tablet PC and how Visual FoxPro plays
into that!

Markus Egger

EPS Software Whitepaper 21