Building GUIs with Julia, Tk, and Cairo, Part

I
23 May 2013 | Timothy E. Holy
This is the first of two blog posts designed to walk users through the process of
creating GUIs in Julia. Those following Julia development will know that plotting
in Julia is still evolving, and one could therefore expect that it might be
premature to build GUIs with Julia. My own recent experience has taught me
that this expectation is wrong: compared with building GUIs in Matlab (my only
previous GUI-writing experience), Julia already offers a number of quite
compelling advantages. Well see some of these advantages on display below.
Well go through the highlights needed to create an image viewer GUI. Before
getting into how to write this GUI, first lets play with it to get a sense for how it
works. Its best if you just try these commands yourself, because its difficult to
capture things like interactivity with static text and pictures.
Youll need the ImageView package:
Pkg.add("ImageView")

Its worth pointing out that this package is expected to evolve over time;
however, if things have changed from whats described in this blog, try
checking out the blog branch directly from the repository. I should also point
out that this package was developed on the authors Linux system, and its
possible that things may not work as well on other platforms.
First lets try it with a photograph. Load one this way:
using Images
using ImageView
img = imread("my_photo.jpg")

Any typical image format should be fine, it doesnt have to be a jpg. Now
display the image this way:
display(img, pixelspacing = [1,1])

The basic command to view the image is display. The optional pixelspacing
input tells display that this image has a fixed aspect ratio, and that this needs
to be honored when displaying the image. (Alternatively, you could set
img["pixelspacing"] = [1,1] and then you wouldnt have to tell this to the
display function.)
You should get a window with your image:

OK, nice. But we can start to have some fun if we resize the window, which
causes the image to get bigger or smaller:

Note the black perimeter; thats because weve specified the aspect ratio
through the pixelspacing input, and when the window doesnt have the same
aspect ratio as the image youll have a perimeter either horizontally or
vertically. Try it without specifying pixelspacing, and youll see that the image
stretches to fill the window, but it looks distorted:
display(img)

(This wont work if youve already defined "pixelspacing" for img; if

necessary, use delete!(img, "pixelspacing") to remove that setting.)
Next, click and drag somewhere inside the image. Youll see the typical
rubberband selection, and once you let go the image display will zoom in on
the selected region.

Again, the aspect ratio of the display is preserved. Double-clicking on the

image restores the display to full size.
If you have a wheel mouse, zoom in again and scroll the wheel, which should
cause the image to pan vertically. If you scroll while holding down Shift, it pans
horizontally; hold down Ctrl and you affect the zoom setting. Note as you zoom
via the mouse, the zoom stays focused around the mouse pointer location,
making it easy to zoom in on some small feature simply by pointing your
mouse at it and then Ctrl-scrolling.
Long-time users of Matlab may note a number of nice features about this
behavior:
The resizing and panning is much smoother than Matlabs
Matlab doesnt expose modifier keys in conjunction with the wheel
mouse, making it difficult to implement this degree of interactivity
In Matlab, zooming with the wheel mouse is always centered on the
middle of the display, requiring you to alternate between zooming and
panning to magnify a particular small region of your image or plot.
These already give a taste of some of the features we can achieve quite easily
in Julia.
However, theres more to this GUI than meets the eye. You can display the
image upside-down with
display(img, pixelspacing = [1,1], flipy=true)

or switch the x and y axes with

display(img, pixelspacing = [1,1], xy=["y","x"])

To experience the full functionality, youll need a 4D image, a movie (time

sequence) of 3D images. If you dont happen to have one lying around, you can
create one via include("test/test4d.jl"), where test means the test
directory in ImageView. (Assuming you installed ImageView via the package
manager, you can say include(joinpath(Pkg.dir(), "ImageView",
"test", "test4d.jl")).) This creates a solid cone that changes color over
time, again in the variable img. Then, type display(img). You should see
something like this:

The green circle is a slice from the cone. At the bottom of the window youll
see a number of buttons and our current location, z=1 and t=1, which
correspond to the base of the cone and the beginning of the movie,
respectively. Click the upward-pointing green arrow, and youll pan through
the cone in the z dimension, making the circle smaller. You can go back with
the downward-pointing green arrow, or step frame-by-frame with the black
arrows. Next, clicking the play forward button moves forward in time, and
youll see the color change through gray to magenta. The black square is a
stop button. You can, of course, type a particular z, t location into the entry
boxes, or grab the sliders and move them.
If you have a wheel mouse, Alt-scroll changes the time, and Ctrl-Alt-scroll
changes the z-slice.
You can change the playback speed by right-clicking in an empty space within
the navigation bar, which brings up a popup (context) menu:

By default, display will show you slices in the xy-plane. You might want to see
a different set of slices from the 4d image:
display(img, xy=["x","z"])

Initially youll see nothing, but thats because this edge of the image is black.
Type 151 into the y: entry box (note its name has changed) and hit enter, or
move the y slider into the middle of its range; now youll see the cone from
the side.

This GUI is also useful for plain movies (2d images with time), in which case
the z controls will be omitted and it will behave largely as a typical movieplayer. Likewise, the t controls will be omitted for 3d images lacking a temporal
component, making this a nice viewer for MRI scans.
Again, we note a number of improvements over Matlab:
When you resize the window, note that the controls keep their initial size,
while the image fills the window. With some effort this behavior is
possible to achieve in Matlab, but (as youll see later in these posts) its
essentially trivial with Julia and Tk.

 When we move the sliders, the display updates while we drag it, not just
when we let go of the mouse button.
If you try this with a much larger 3d or 4d image, you may also notice
that the display feels snappy and responsive in a way thats sometimes
hard to achieve with Matlab.
Altogether advantages such as these combine to give a substantially more
polished feel to GUI applications written in Julia.
This completes our tour of the features of this GUI. Now lets go through a few
of the highlights needed to create it. Well tackle this in pieces; not only will
this make it easier to learn, but it also illustrates how to build re-useable
components. Lets start with the navigation frame.

First steps: the navigation frame

First, let me acknowledge that this GUI is built on the work of many people who
have contributed to Julias Cairo and Tk packages. For this step, well make
particular use of John Verzanis contribution of a huge set of convenience
wrappers for most of Tks widget functionality. John wrote up a nice set of
examples that demonstrate many of the things you can do with it; this first
installment is essentially just a longer example, and wont surprise anyone
who has read his documentation.
Lets create a couple of types to hold the data well need. We need a type that
stores GUI state, which here consists of the currently-viewed location in the
image and information needed to implement the play functionality:
type NavigationState
# Dimensions:
zmax::Int
#
tmax::Int
#
z::Int
#
t::Int
#
# Other state data:
timer
#
fps::Float64
#
end

number of frames in z, set to 1 if only 2 spatial dims

number of frames in t, set to 1 if only a single image
current position in z-stack
current moment in time
nothing if not playing, TimeoutAsyncWork if we are
playback speed in frames per second

Next, lets create a type to hold handles to all the widgets:

type NavigationControls
stepup
stepdown
playup
playdown
stepback
stepfwd
playback
playfwd
stop
editz
editt
textz
textt
scalez
scalet
end

# z buttons...

# t buttons...

# edit boxes
# static text (information)
# scale (slider) widgets

It might not be strictly necessary to hold handles to all the widgets (you could
do everything with callbacks), but having them available is convenient. For
example, if you dont like the icons I created, you can easily initialize the GUI
and replace, using the handles, the icons with something better.
Well talk about initialization later; for now, assume that we have a variable
state of type NavigationState that holds the current position in the (possibly)
4D image, and ctrls which contains a fully-initialized set of widget handles.
Each button needs a callback function to be executed when it is clicked. Lets
go through the functions for controlling t. First there is a general utility not tied
to any button, but it affects many of the controls:
function updatet(ctrls, state)
set_value(ctrls.editt, string(state.t))
set_value(ctrls.scalet, state.t)
enableback = state.t > 1
set_enabled(ctrls.stepback, enableback)
set_enabled(ctrls.playback, enableback)
enablefwd = state.t < state.tmax
set_enabled(ctrls.stepfwd, enablefwd)
set_enabled(ctrls.playfwd, enablefwd)
end

The first two lines synchronize the entry box and slider to the current value of
state.t; the currently-selected time can change by many different
mechanisms (one of the buttons, typing into the entry box, or moving the
slider), so we make state.t be the authoritative value and synchronize
everything to it. The remaining lines of this function control which of the t
navigation buttons are enabled (if t==1, we cant go any earlier in the movie,
so we gray out the backwards buttons).
A second utility function modifies state.t:
function incrementt(inc, ctrls, state, showframe)
state.t += inc
updatet(ctrls, state)
showframe(state)
end

Note the call to updatet described above. The new part of this is the
showframe function, whose job it is to display the image frame (or any other
visual information) to the user. Typically, the actual showframe function will
need additional information such as where to render the image, but you can
provide this information using anonymous functions. Well see how that works
in the next installment; below well just create a simple stub function.
Now we get to callbacks which well bind to the step and play buttons:
function stept(inc, ctrls, state, showframe)
if 1 <= state.t+inc <= state.tmax
incrementt(inc, ctrls, state, showframe)
else
stop_playing!(state)
end
end
function playt(inc, ctrls, state, showframe)
if !(state.fps > 0)
error("Frame rate is not positive")

end
stop_playing!(state)
dt = 1/state.fps
state.timer = TimeoutAsyncWork(i -> stept(inc, ctrls, state, showframe))
start_timer(state.timer, iround(1000*dt), iround(1000*dt))
end

stept() increments the t frame by the specified amount (typically 1 or -1),

while playt() starts a timer that will call stept at regular intervals. The timer
is stopped if play reaches the beginning or end of the movie. The
stop_playing! function checks to see whether we have an active timer, and if
so stops it:
function stop_playing!(state::NavigationState)
if !is(state.timer, nothing)
stop_timer(state.timer)
state.timer = nothing
end
end

An alternative way to handle playback without a timer would be in a loop, like

this:
function stept(inc, ctrls, state, showframe)
if 1 <= state.t+inc <= state.tmax
incrementt(inc, ctrls, state, showframe)
end
end
function playt(inc, ctrls, state, showframe)
state.isplaying = true
while 1 <= state.t+inc <= state.tmax && state.isplaying
tcl_doevent()
# allow the stop button to take effect
incrementt(inc, ctrls, state, showframe)
end
state.isplaying = false
end

With this version we would use a single Boolean value to signal whether there
is active playback. A key point here is the call to tcl_doevent(), which allows
Tk to interrupt the execution of the loop to handle user interaction (in this case,
clicking the stop button). But with the timer thats not necessary, and moreover
the timer gives us control over the speed of playback.
Finally, there are callbacks for the entry and slider widgets:
function sett(ctrls,state, showframe)
tstr = get_value(ctrls.editt)
try
val = int(tstr)
state.t = val
updatet(ctrls, state)
showframe(state)
catch
updatet(ctrls, state)
end
end
function scalet(ctrls, state, showframe)
state.t = get_value(ctrls.scalet)
updatet(ctrls, state)

showframe(state)
end

sett runs when the user types an entry into the edit box; if the user types in
nonsense like foo, it will gracefully reset it to the current position.
Theres a complementary set of these functions for the z controls.
These callbacks implement the functionality of this navigation GUI. The other
main task is initialization. We wont cover this in gory detail (you are invited to
browse the code), but lets hit a few highlights.
Creating the buttons
You can use image files (e.g., .png files) for your icons, but the ones here are
created programmatically. To do this, specify two colors, the foreground and
background, as strings. One also needs the data array (of type Bool) for the
pixels that should be colored by the foreground color, and false for the ones to
be set to the background. Theres also the mask array, which can prevent the
data array from taking effect in any pixels marked as false in the mask.
Given suitable data and mask arrays (here we just set the mask to trues), and
color strings, we create the icon and assign it to a button like this:
icon = Tk.image(data, mask, "gray70", "black")
foreground=black
ctrls.stop = Button(f, icon)

# background=gray70,

Here f is the parent frame that the navigation controller will be rendered in.
A frame is a container that organizes a collection of related GUI elements. Later
well find out how to create one.
Assigning callbacks to widgets
The stop and play backwards buttons look like this:
bind(ctrls.stop, "command", path -> stop_playing!(state))
bind(ctrls.playback, "command", path -> playt(-1, ctrls, state, showframe)

The path input is generated by Tk/Tcl, but we dont have to use it. Instead, we
use anonymous functions to pass the arguments relavant to this particular GUI
instantiation. Note that these two buttons share state; that means that any
changes made by one callback will have impact on the other.
Placing the buttons in the frame (layout management)
Here our layout needs are quite simple, but I recommend that you read the
excellent tutorial on Tks grid layout engine. grid provides a great deal of
functionality missing in Matlab, and in particular allows flexible and polished
GUI behavior when resizing the window.
We position the stop button this way:
grid(ctrls.stop, 1, stopindex, padx=3*pad, pady=pad)

After the handle for the button itself, the next two inputs determine the row,
column position of the widget. Here the column position is set using a variable
(an integer) whose value will depend on whether the z controls are present.

The pad settings just apply a bit of horizontal and vertical padding around the
button.
To position the slider widgets, we could do something like this:
ctrls.scalez = Slider(f, 1:state.zmax)
grid(ctrls.scalez, 2, start:stop, sticky="we", padx=pad)

This positions them in row 2 of the frames grid, and has them occupy the
range of columns (indicated by start:stop) used by the button controls for the
same z or t axis. The sticky setting means that it will stretch to fill from West
to East (left to right).
In the main GUI well use one more feature of grid, so lets cover it now. This
feature controls how regions of the window expand or shrink when the window
is resized:
grid_rowconfigure(win, 1, weight=1)
grid_columnconfigure(win, 1, weight=1)

This says that row 1, column 1 will expand at a rate of 1 when the figure is
made larger. You can set different weights for different GUI components. The
default value is 0, indicating that it shouldnt expand at all. Thats what we
want for this navigation frame, so that the buttons keep their size when the
window is resized. Larger weight values indicate that the given component
should expand (or shrink) at faster rates.

Putting it all together and testing it out

Well place the navigation controls inside a Tk frame. Lets create one from the
command line:
using Tk
win = Toplevel()
f = Frame(win)
pack(f, expand=true, fill="both")

The first three lines create the window and the frame. pack is an alternative
layout engine to grid, and slightly more convenient when all you want is to
place a single item so that it fills its container. (You can mix pack and grid as
long as they are operating on separate containers. Here well have a frame
packed in the window, and the widgets will be gridded inside the frame.) After
that fourth line, the window is rather tiny; the call to pack causes the frame to
fill to expand the whole window, but at the moment the frame has no contents,
so the window is as small as it can be.

We need a showframe callback; for now lets create a very simple one that will
help in testing:
showframe = x -> println("showframe z=", x.z, ", t=", x.t)

Next, load the GUI code (using ImageView.Navigation) and create the
NavigationState and NavigationControls objects:
ctrls = NavigationControls()

state = NavigationState(40, 1000, 2, 5)

Here weve set up a fake movie with 40 image slices in z, and 1000 image
stacks in t.
Finally, we initialize the widgets:
init_navigation!(f, ctrls, state, showframe)

Now when you click on buttons, or change the text in the entry boxes, youll
see the GUI in action. You can tell from the command line output, generated by
showframe, whats happening internally:

Hopefully this demonstrates another nice feature of developing GUIs in Julia:

its straightforward to build re-usable components. This navigation frame can
be added as an element to any window, and the grid layout manager takes
care of the rest. All you need to do is to include
ImageView/src/navigation.jl into your module, and you can make use of it
with just a few lines of code.
Not too hard, right? The next step is to render the image, which brings us into
the domain of Cairo.

Building GUIs with Julia, Tk, and Cairo, Part

II
23 May 2013 | Timothy E. Holy

Drawing, painting, and plotting

In this installment, well cover both low-level graphics (using Cairo) and plotting
graphs inside GUIs (using Winston). Here again were relying on infrastructure
built by many people, including Jeff Bezanson, Mike Nolta, and Keno Fisher.

Cairo
The basics
The display of the image is handled by Cairo, a C library for two-dimensional
drawing. Julias Cairo wrapper isnt currently documented, so lets walk through
a couple of basics first.
If youre new to graphics libraries like Cairo, there are a few concepts that may
not be immediately obvious but are introduced in the Cairo tutorial. The key
concept is that the Cairo API works like stamping, where a source gets
applied to a destination in a region specified by a path. Here, the destination
will be the pixels corresponding to a region of a window on the screen. Well
control the source and the path to achieve the effects we want.
Lets play with this. First, inside a new window we create a Cairo-enabled
Canvas for drawing:
using Base.Graphics
using Cairo
using Tk
win = Toplevel("Test", 400, 200)
c = Canvas(win)
pack(c, expand=true, fill="both")

Weve created a window 400 pixels wide and 200 pixels high. c is our Canvas, a
type defined in the Tk package. Later well dig into the internals a bit, but for
now suffice it to say that a Canvas is a multi-component object that you can
often treat as a black box. The initial call creating the canvas leaves a lot of its
fields undefined, because you dont yet know crucial details like the size of the
canvas. The call to pack specifies that this canvas fills the entire window, and
simultaneously fills in the missing information in the Canvas object itself.
Note that the window is currently blank, because we havent drawn anything to
it yet, so you can see whatever was lying underneath. In my case it captured a
small region of my desktop:

Now lets do some drawing. Cairo doesnt know anything about Tk Canvases, so
we have to pull out the part of it that works directly with Cairo:
ctx = getgc(c)

getgc means get graphics context, returning an object (here ctx) that holds
all relevant information about the current state of drawing to this canvas.
One nice feature of Cairo is that the coordinates are abstracted; ultimately we
care about screen pixels, but we can set up user coordinates that have
whatever scaling is natural to the problem. We just have to tell Cairo how to
convert user coordinates to device (screen) coordinates. We set up a
coordinate system using set_coords, defined in base/graphics.jl:
function set_coords(ctx::GraphicsContext, x, y, w, h, l, r, t, b)

x (horizontal) and y (vertical) specify the upper-left corner of the drawing

region in device coordinates, and w and h its width and height, respectively.
(Note Cairo uses (0,0) for the top-left corner of the window.) l, r, t, and b are
the user coordinates corresponding to the left, right, top, and bottom,
respectively, of this region. Note that set_coords will also clip any drawing
that occurs outside the region defined by x, y, w, and h; however, the
coordinate system youve specified extends to infinity, and you can draw all the
way to the edge of the canvas by calling reset_clip().
Lets fill the drawing region with a color, so we can see it:
# Set coordinates to go from 0 to 10 within a 300x100 centered region
set_coords(ctx, 50, 50, 300, 100, 0, 10, 0, 10)
set_source_rgb(ctx, 0, 0, 1)
# set color to blue
paint(ctx)
# paint the entire clip region

Perhaps surprisingly, nothing happened. The reason is that the Tk Canvas

implements a technique called double buffering, which means that you do all
your drawing to a back (hidden) surface, and then blit the completed result to
the front (visible) surface. We can see this in action simply by bringing another
window over the top of the window were using to draw, and then bringing our
window back to the top; suddenly youll see a nice blue rectangle within the
window, surrounded by whatever is in the background window(s):

Fortunately, to display your graphics you dont have to rely on users changing
the stacking order of windows: call reveal(c) to update the front surface with
the contents of the back surface, followed by update() (or perhaps better,
Tk.update() since update is a fairly generic name) to give Tk a chance to
expose the front surface to the OSs window manager.
Now lets draw a red line:
move_to(ctx, -1, 5)
line_to(ctx, 7, 6)
set_source_rgb(ctx, 1, 0, 0)
set_line_width(ctx, 5)
stroke(ctx)
reveal(c)
Tk.update()

We started at a position outside the coordinate region (well get to see the
clipping in action this way). The next command, line_to, creates a segment of
a path, the way that regions are defined in Cairo. The stroke command draws
a line along the trajectory of the path, after which the path is cleared. (You can
use stroke_preserve if you want to re-use this path for another purpose later.)
Lets illustrate this by adding a solid green rectangle with a magenta border,
letting it spill over the edges of the previously-defined coordinate region:
reset_clip(ctx)
rectangle(ctx, 7, 5, 4, 4)
set_source_rgb(ctx, 0, 1, 0)
fill_preserve(ctx)
set_source_rgb(ctx, 1, 0, 1)
stroke(ctx)
reveal(c)
Tk.update()

fill differs from paint in that fill works inside the currently-defined path,
whereas paint fills the entire clip region.
Here is our masterpiece, where the background may differ for you (mine was
positioned over the bottom of a wikipedia page):

Rendering an image
Images are rendered in Cairo inside a rectangle (controlling placement of the
image) followed by fill. So far this is just like the simple drawing above. The
difference is the source, which now will be a surface instead of an RGB color. If
youre drawing from Julia, chances are that you want to display an in-memory
array. The main trick is that Cairo requires this array to be a matrix of type
Uint32 encoding the color. The scheme is that the least significant byte is the
blue value (ranging from 0x00 to 0xff), the next is green, and the next red.
(The most significant byte can encode the alpha value, or transparency, if you
specify that transparency is to be used in your image surface.)
Both Winston and Images can generate a buffer of Uint32 for you. Lets try the
one in Images:
using Images
img = imread("some_photo.jpg")
buf = uint32color(img)'
image(ctx, CairoRGBSurface(buf), 0, 0, 10, 10)
reveal(c)
Tk.update()

Rather than manually calling rectangle and fill, we use the convenience
method image(ctx, surf, x, y, w, h) (defined in Cairo.jl). Here x, y, w, h
are user-coordinates of your canvas, not pixels on the screen or pixels in your
image; being able to express location in user coordinates is the main
advantage of using image().
The image should now be displayed within your window (squashed, because we
havent worried about aspect ratio):

It fills only part of the window because of the coordinate system weve
established, where the range 0:10 corresponds to an inset region in the center
of the window.
While its a minor point, note that CairoRGBSurface takes a transpose for you,
to convert from the column-major order of matrices in Julia to the row-major
convention of Cairo. Images avoids taking transposes unless necessary, and is
capable of handling images with any storage order. Here we do a transpose in
preparation to have it be converted back to its original shape by
CairoRGBSurface. If performance is critical, you can avoid the default behavior
of CairoRGBSurface by calling CairoImageSurface directly (see the Cairo.jl
code).

Redrawing & resize support

A basic feature of windows is that they should behave properly under resizing
operations. This doesnt come entirely for free, although the grid (and pack)
managers of Tk take care of many details for us. However, for Canvases we
need to to do a little bit of extra work; to see what I mean, just try resizing the
window we created above.
The key is to have a callback that gets activated whenever the canvas changes
size, and to have this callback capable of redrawing the window at arbitrary
size. Canvases make this easy by having a field, resize, that you assign the
callback to. This function will receive a single argument, the canvas itself, but
as always you can provide more information. Taking our image example, we
could set
c.resize = c->redraw(c, buf)

and then define

function redraw(c::Canvas, buf)
ctx = getgc(c)
set_source_rgb(ctx, 1, 0, 0)
paint(ctx)
set_coords(ctx, 50, 50, Tk.width(c)-100, Tk.height(c)-100, 0, 10, 0, 10)
image(ctx, CairoRGBSurface(buf), 0, 0, 10, 10)
reveal(c)
Tk.update()
end

Here you can see that were aiming to be a bit more polished, and want to

avoid seeing bits of the desktop around the borders of our drawing region. So
we fill the window with a solid color (but choose a garish red, to make sure we
notice it) before displaying the image. We also have to re-create our coordinate
system, because that too was destroyed, and in this case we dynamically
adjust the coordinates to the size of the canvas. Finally, we redraw the image.
Note we didnt have to go through the process of converting to Uint32-based
color again. Obviously, you can use this redraw function even for the initial
rendering of the window, so theres really no extra work in setting up your code
this way.
If you grab the window handle and resize it, now you should see something like
this:

Voila! Were really getting somewhere now.

Unlike the complete GUI, this implementation doesnt have the option to
preserve the images aspect ratio. However, theres really no magic there; it all
comes down to computing sizes and controlling the drawing region and
coordinate system.
One important point: resizing the window causes the existing Cairo context(s)
to be destroyed, and creates new ones suitable for the new canvas size. One
consequence is that your old ctx variable is now invalid, and trying to use it for
drawing will cause a segfault. For this reason, you shouldnt ever store a ctx
object on its own; always begin drawing by calling getgc(c) again.

Canvases and the mouse

A Canvas already comes with a set of fields prepared for mouse events. For
example, in the complete GUI we have the equivalent of the following:
selectiondonefunc = (c, bb) -> zoombb(imgc, img2, bb)
c.mouse.button1press = (c, x, y) -> rubberband_start(c, x, y, selectiondonefunc)

rubberband_start, a function defined in rubberband.jl, will now be called

whenever the user presses the left mouse button. selectiondonefunc is a
callback that we supply; it will be executed when the user releases the mouse
button, and it needs to implement whatever it is we want to achieve with the
selected region (in this case, a zoom operation). Part of what
rubberband_start does is to bind selectiondonefunc to the release of the
mouse button, via c.mouse.button1release. bb is a BoundingBox (a type
defined in base/graphics.jl) that will store the region selected by the user,
and this gets passed to selectiondonefunc. (The first two inputs to zoombb,
imgc and img2, store settings that are relevant to this particular GUI but will not
be described in detail here.)
The mouse inside a Canvas is an object of type MouseHandler, which has fields
for press and release of all 3 mouse buttons and additional ones for motion.
However, a few cases (which happen to be relevant to this GUI) are not
available in MouseHandler. Here are some examples of how to configure these
actions:
# Bind double-clicks
bind(c.c, "<Double-Button-1>", (path,x,y)->zoom_reset(imgc, img2))
# Bind Shift-scroll (using the wheel mouse)
bindwheel(c.c, "Shift", (path,delta)->panhorz(imgc,img2,int(delta)))

The delta argument for the wheel mouse will encode the direction of scrolling.

The rubber band (region selection)

Support for the rubber band is provided in the file rubberband.jl. Like
navigation.jl, this is a stand-alone set of functions that you should be able to
incorporate into other projects. It draws a dashed rectangle employing the
same machinery we described at the top of this page, with slight modifications
to create the dashes (through the set_dash function). By now, this should all
be fairly straightforward.
However, these functions use one additional trick worth mentioning. Lets
finally look at the Tk Canvas object:
type Canvas
c::TkWidget
front::CairoSurface # surface for window
back::CairoSurface
# backing store
frontcc::CairoContext
backcc::CairoContext
mouse::MouseHandler
redraw
function ...

Here we can explicitly see the two buffers, used in double-buffering, and their
associated contexts. getgc(c), where c is a Canvas, simply returns backcc.
This is why all drawing occurs on the back surface. For the rubber band, we
choose instead to draw on the front surface, and then (as the size of the rubber
band changes) repair the damage by copying from the back surface. Since
we only have to modify the pixels along the band itself, this is fast. You can see
these details in rubberband.jl.

Winston
For many GUIs in Julia, an important component will be the ability to display
data graphically. While we could draw graphs directly with Cairo, it would be a
lot of work to build from scratch; fortunately, theres an excellent package,
Winston, that already does this.
Since theres a nice set of examples of some of the things you can do with
Winston, here our focus is very narrow: how do you integrate Winston plots into
GUIs built with Tk. Fortunately, this is quite easy. Lets walk through an
example:
using Tk
using Winston
win = Toplevel("Testing", 400, 200)
fwin = Frame(win)
pack(fwin, expand=true, fill="both")

We chose to fill the entire window with a frame fwin, so that everything inside
this GUI will have a consistent background. All other objects will be placed
inside fwin.
Next, lets set up the elements, a Canvas on the left and a single button on the
right:
c = Canvas(fwin, 300, 200)
grid(c, 1, 1, sticky="nsew")
fctrls = Frame(fwin)
grid(fctrls, 1, 2, sticky="sw", pady=5, padx=5)
grid_columnconfigure(fwin, 1, weight=1)
grid_rowconfigure(fwin, 1, weight=1)
ok = Button(fctrls, "OK")
grid(ok, 1, 1)

Finally, lets plot something inside the Canvas:

x = linspace(0.0,10.0,1001)
y = sin(x)
p = FramedPlot()
add(p, Curve(x, y, "color", "red"))
Winston.display(c, p)
reveal(c)
Tk.update()

Youll note that you can resize this window, and the plot grows or shrinks
accordingly.
Easy, huh? The only part of this code that is specific to GUIs is the line
Winston.display(c, p), where we specified that we wanted our plot to
appear inside a particular Canvas. Of course, theres a lot of magic behind the
scenes in Winston, but covering its internals is beyond our scope here.

Conclusions
Theres more one could cover, but most of the rest is fairly specific to this
particular GUI. A fair amount of code is needed to handle coordinates: selecting
specific regions within the 4d image, and rendering to specific regions of the
output canvas. If you want to dive into these details, your best bet is to start
reading through the ImageView code, but its not going to be covered in any
more detail here.
Hopefully by this point you have a pretty good sense for how to produce onscreen output with Tk, Cairo, and Winston. It takes a little practice to get
comfortable with these tools, but the end result is quite powerful. Happy
hacking!