ADLib: AN ARDUINO COMMUNICATION FRAMEWORK FOR AMBIENT DISPLAYS*

Robert Adams and Russ Shearer School of Computing and Information Systems Grand Valley State University Allendale, MI +1 616 331 2060 adams@cis.gvsu.edu, shearerr@mail.gvsu.edu

ABSTRACT Ambient displays have been a major area of research within the ubiquitous computing field since the early days of Mark Weiser's vision. Although many forms of ambient displays have been built, they all require a significant amount of development time. In this paper we introduce ADLib, a communication framework built to provide easy communication with an Arduino-based ambient display. The ADLib framework consists of a protocol to encode information between a host computer and an Arduino, an Arduino library for accepting and parsing incoming data, and a desktop application that eases the sending of data to the Arduino. The ADLib framework virtually eliminates the need to think about how to send information to the ambient display, allowing the developer to focus on its representational elements. 1 INTRODUCTION Ambient displays have been a mainstay of ubiquitous computing research starting with Mark Weiser's time at Xerox PARC. Ambient displays are devices that are "distinguished from more typical informational displays in that they are designed to be minimally attended and perceivable primarily from outside of a person's direct focus of attention, providing pre-attentive processing without being unnecessarily distracting." [6] Perhaps the earliest known example is Natalie Jeremijenko's "Dangling String" installation at PARC: a long piece of string suspended from the ceiling that rotated in connection with the amount of network traffic on the LAN. Since then, numerous ___________________________________________
*

Copyright 2011 by the Consortium for Computing Sciences in Colleges. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the CCSC copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Consortium for Computing Sciences in Colleges. To copy otherwise, or to republish, requires a fee and/or specific permission.

16

CCSC: Midwestern Conference researchers have proposed novel ambient displays to monitor such things as the weather [11], bus routes [9], email volume [1], and posture [8]. Building ambient displays is challenging. An ambient display is not simply a physical device, but rather a system often comprising a physical device to display information (a data sink), a data source gathering and processing raw information, and a protocol used to transmit data between the two. Several choices are available for building the physical devices themselves, from off-the-shelf components like the Chumby [7], to custom-built hardware. The tools available for constructing the data source of an ambient display are virtually limitless, although they generally fall into two categories: sensors on the display itself, or a server reading information and passing it to the device. In this paper, we focus on the latter. The final component is the protocol used to communicate information between the server and the ambient display. Again, the choices are wide: serial, Bluetooth, Ethernet, etc. One popular choice for building ambient displays is the Arduino [5]. Arduino is an inexpensive open-source prototyping microcontroller aimed at making hardware prototyping accessible to those without an extensive background in electronics. The main strengths of the Arduino are its ease of use and the tremendous volume of helpful information and people found on the Web. The Arduino allows one to quickly and easily build ambient displays. Although the Arduino platform addresses the "display" portion of the ambient display system, it does not address either the data source or protocol. The Arduino is capable of communication across several different protocols. For example, Ethernet capabilities can easily be added to an Arduino, and an Ethernet library similar to the UNIX socket library [3] is available for use. The challenge is that prototyping ambient displays using the Arduino requires a protocol be developed for each prototype. Our contribution is a communication library (ADLib (Ambient Display Library)) specifically developed for ambient displays. ADLib allows designers to focus on the ambient device and data source components of the system without having to reinvent a communication protocol. The rest of this paper is organized as follows. Section 2 discusses related work. Section 3 provides an overview of the ADLib framework, while section 4 covers each element of the framework in detail. Section 5 discusses our evaluation of the framework, and section 6 presents our conclusions and future work. 2 RELATED WORK Several libraries exist for communicating with an Arduino. The simplest is the serial library [4]. Using the library, the Arduino can send and receive data from a computer attached via a USB cable. The serial library provides no guidance on the organization of data between the Arduino and the host computer. Rather, the developer is required to invent a communication protocol. One library that provides structured serial communication is the Messenger library [5]. This library provides the Arduino the ability to split incoming strings on a delimiter.

17

JCSC 27, 1 (October 2011) Although this is more useful for building ambient displays, it still leaves most of the details of what to send in each field up to the developer. Along with serial communication, an Arduino can also communicate over an Ethernet connection via an attached Ethernet card (a "shield"). Like the serial library, the standard Ethernet library [3] provides the ability to send and receive text strings, leaving the format and content of those strings entirely up to the developer. 3 ADLIB FRAMEWORK Our contribution consists of a protocol, Arduino library, and desktop application to support the rapid development of ambient displays. Using the ADLib framework, developers can focus on the mechanics of creating ambient displays with the Arduino, rather than having to reinvent a communication protocol and desktop server for each display. The ADLib protocol defines the type and structure of data to be sent between a host computer and the Arduino. The ADLib Arduino library handles the details of receiving, analyzing, and splitting the incoming data. The desktop application allows virtually any data collection application to send data to the ambient display. The communication model supported by ADLib is quite straightforward (Figure 1). An application program reads data to be delivered to the ambient display. The data collection application passes the information to the ADLib desktop application, which bundles the data in the ADLib protocol, and delivers it to the Arduino. Finally, the ADLib Arduino library accepts the incoming data, parses it, and calls a user-created handler function. The handler function is responsible for updating the ambient display.

Figure 1: Data flow in the ADLib framework.

18

CCSC: Midwestern Conference 4 ADLib ELEMENTS 4.1 Communication Protocol As described above, the ADLib communication model is one in which the ambient display is updated based on data collected on a host computer. Therefore, the ADLib protocol is unidirectional, designed to allow a host computer to send data to an Arduino to update an ambient display. No provision is made to allow the Arduino to send data back to the host computer. ADLib is a message-based protocol that supports four types of messages, two string-based and two numeric: error, text, normalized, and numeric (summarized in Table 1). An error message is used to send an error string to the Arduino, while a text message sends a plain (non-error) string. A normalized message is used to send a numeric value that has been normalized to the range 0 to 100. A numeric message is designed to send an arbitrary numeric value, along with the range minimum and maximum, as well as the amount of change from the last message (delta). Message Type Error Text Normalized Numeric Fields deviceID, string deviceID, string deviceID, value deviceID, value, min, max, delta Example 1, "Invalid data source" 2, "Hello" 1, 65.3 1, 34, -100, 100, -45

Table 1: ADLib Protocol Summary One novel feature of the ADLib framework is the support of multiple "devices" or "discrete information sources" [10] per ambient display. For example, an articulated flower arrangement may have one flower associated with each data source. Different flowers could open or close depending on the state of the data. Support for multiple devices is achieved via a deviceID field in the protocol. The deviceID is an integer that uniquely represents some element of the ambient display. It is up to the ambient display to associate a deviceID with a particular set of LEDs, servos, etc. 4.2 Desktop Application The ADLib desktop application is responsible for accepting data, bundling it in the ADLib protocol, and transmitting it to the ADLib Arduino library. The design goal of the desktop application is ease of use. Therefore, we decided to build the application with a command-line interface (CLI). The desktop application is programming language and architecture independent, allowing virtually any data collection application to use the framework. The only requirement is that the data collection application must be able to invoke an external command. Virtually all programming languages have this feature, allowing the data collection application to be written in whatever language is most suitable for collecting the data. The data collection application can then simply invoke the ADLib CLI to pass the data to the Arduino. So far, two desktop applications have been built, one in C# and one in Python. Both function identically, taking all data as command-line parameters. The basic usage of the client is: client [parameters], where parameters specify (1) how to communicate with the 19

JCSC 27, 1 (October 2011) Arduino, (2) which ambient device on the Arduino should receive the message, and (3) the data payload. These parameters are discussed next. The ADLib client currently supports two forms of communication, serial and Ethernet. For serial communication, the --serial=device parameter is used where device is the hardware device name. For Ethernet, two parameters are necessary: --ip=ip_address and --port=port. These specify the IP address and port on which the Arduino is listening. As mentioned above, ADLib supports multiple "devices" per ambient display. The --device=device_id parameter is used to address the message to a specific device on the Arduino. Finally, the four types of message described in the previous section have corresponding command-line parameters as shown in Table 2. Message Type Error Text Normaliyed Numeric Parameters --type=0 --text="Invalid data source" --type=1 --text="Hello" --type=2 --normalized=65.3 --type=3 --current=34 --min=-100 --max=100 --delta=-45

Table 2: ADLib Desktop Application Message Parameters. These parameters correspond to the examples shown in Table 1. As mentioned above, the power of the ADLib desktop application is the fact that it can be used by virtually any program written in virtually any programming language. For example, in Python one must only construct the command-line to execute and then pass it to the subprocess.Popen() function:
subprocess.Popen("client --serial=/dev/tty.usbmodem411 --device=1 --type=2 --normalized=65.3", shell=True)

4.3 Arduino Library The goal of the ADLib Arduino library is to ease communication with the host computer. Using the library requires three steps. First, the library must be initialized to indicate how to communicate with the desktop host (e.g., serial or Ethernet). Second, callback functions must be registered to handle the expected types of incoming data. Third, an ADLib function that reads and parses data must be called. As described above, ADLib supports two forms of communication, serial and Ethernet. The ADLib library must be initialized to indicate which form of communication to use (typically in the Arduino setup() function). This is accomplished by one of two function: ADLib_StartSerial() for serial communication, or ADLib_StartEthernet(mac_address, ip_address, subnet_mask, gateway) for Ethernet communication. After the library has been initialized, callback functions must be registered to handle data coming from the host computer. Because the ADLib protocol supports four different messages types, four distinct message handlers can be registered. All four callback 20

CCSC: Midwestern Conference functions are passed the data received from the host computer as a struct. Listing 1 shows an example handler for numeric data as well as how it is registered.
void handleNumericData(struct ADLib_NumericDataType *p) { // Handler can access p->minValue, p->maxValue, // p->currentValue, and p->deltaValue. } ADLib_registerNumericHandler( &handleNumericData );

Listing 1: An example callback function to handle numeric data, and how the function is registered with ADLib. After the desired data handlers have been registered, the final step is to call the ADLib function that reads and parses data from the host computer and delivers it to the callback function. This is accomplished through the single function ADLib_Parse(isBlocking). The boolean parameter indicates whether or not to block waiting for data. Putting all of these steps together, a complete Arduino sketch using ADLib is shown in Listing 2 (include files are omitted for space).
void setup() { Serial.begin(9600); ADLib_StartSerial(); ADLib_registerNumericHandler( &handleNumericData ); } void loop() { ADLib_Parse( ADLIB_BLOCKING ); } void handleNumericData( ADLib_NumericDataType *p ) { // update the ambient display }

Listing 2: A complete example of using the ADLib library. 5 EVALUATION So far we have built two ambient displays with the ADLib library. The first one is used to monitor the CPU utilization of a VMware cluster hosting 350 virtual servers. The ambient display consists of a translucent plastic flower pot with an Arduino connected to a mini-breadboard in the base. LEDs in the breadboard are controlled by the Arduino to indicate the current utilization of the VMware cluster. The flowerpot glows green while the CPU utilization of the virtual machine farm is below 50% utilization, and then fades from green to red between 50% and 100% utilization. An evaluation of the effectiveness of this ambient display is outside the scope of this paper (see Future Work below). The intent here is to evaluate the ADLib framework itself.

21

JCSC 27, 1 (October 2011) A data collection program was written in Windows PowerShell to gather information on the VMware cluster. That data is condensed into a numeric value and passed to the ADLib desktop application. On the Arduino, a program structured like the one shown in Listing 2 was created to update the LEDs appropriately. The data collection program consists of 17 lines of PowerShell code. Of those, only one was necessary for communicating with the ADLib desktop application. This represents 5% of the data collection program. Similarly, the Arduino program consists of 500 lines of code, of which only 5 were necessary for using the ADLib library. This represents 1% of the Arduino program. The second display we built displays school closing information. A data collection program reads school closing information from a web page and sends that information to the ambient display. An LED glows green if school is open, blue if there is a one hour delay, orange if there is a two hour delay, or red if school is closed. A Python program was written to gather school closing information from a local TV station's web page and pass that information to the ADLib desktop application as a normalized value. Of the 56 lines of Python code, only 1 line was necessary to invoke the desktop application. This is less than 2% of the program. On the Arduino side, a program was written to accept incoming data and change the color of an LED accordingly. The Arduino program consists of almost 200 lines of code, of which 4 (2%) were required by ADLib. Another evaluation metric was to compare the size of the messages between the desktop computer and the Arduino. Two obvious protocol encodings are XML and JSON. Both of these suffer from a lack of terseness. Although this isn't often a problem when dealing with desktop machines, memory is at a premium in an embedded system. Our experiments using XML and JSON led us to conclude that neither was suitable for communicating with the Arduino. However, we have not given up on either, and we plan to investigate the use of JSON in the future. 6 CONCLUSIONS AND FUTURE WORK As the previous section demonstrates, the ADLib framework requires very little effort from the ambient display developer, allowing her to focus on the representational elements of the ambient display rather than on the details of communication. In the two displays described above, practically no time was spent considering device communication. Instead, the code from Listing 2 was used as a starting point, and handler code was added to change the LEDs. From the evidence, it is clear that ADLib has met its design goal of ease of use. Although the ease-of-use design goal has been met, another area is being considering that would improve ADLib. [2] and others have discussed the issue of "trust" in an ambient display. One way trust may be realized is by providing the user some indication that the system is still functioning normally. In the VMware flowerpot example, how can the user be confident that (a) the data collection program is still working, and (b) it is sending data to the flowerpot? Similar questions can be asked of all ambient displays. In the school closing display, the LED slowly fades from black to white to black periodically to indicate that the system is still alive. This required a rather complex use of timers to detect missing updates, control the rate of fade, etc. We are 22

CCSC: Midwestern Conference considering ways that the ADLib library could be updated to remove some of the complexity of timers. REFERENCES [1] Altosaar, M., Vertegaal, R., Sohn, C., Cheng, D., AuraOrb: using social awareness cues in the design of progressive notification appliances. In Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments (OZCHI '06), Jesper Kjeldskov and Jeni Paay (Eds.). ACM, New York, NY, USA, 159-166. [2] Antifakos, S., Kern, N., Schiele, B., Schwaninger, A., Towards improving trust in context-aware systems by displaying system confidence. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services (MobileHCI '05). ACM, New York, NY, USA, 9-14. [3] Arduino, Arduino Ethernet Library, 2009, http://arduino.cc/en/Reference/Ethernet. [4] Arduino, Arduino Serial Library, 2009, http://arduino.cc/en/Reference/Serial. [5] Arduino, Arduino Messenger Library, 2009, http://arduino.cc/playground/Code/Messenger. [5] Banzi, M., Getting Started with Arduino (Ill ed.). Make Books, Sebastopol, CA. [6] Hazlewood, W., Connelly, K., Makice, K., Lim, Y., Exploring evaluation methods for ambient information systems. In CHI '08 extended abstracts on Human factors in computing systems (CHI '08). ACM, New York, NY, USA, 2973-2978. [7] Huang, A., Chumby: An Experiment in Hackable Pervasive Computing. IEEE Pervasive Computing 7, 3 (July 2008), 55-62. [8] Jafarinaimi, N., Forlizzi, J., Hurst, A., Zimmerman, J., Breakaway: an ambient display designed to change human behavior. In CHI '05 extended abstracts on Human factors in computing systems (CHI EA '05). ACM, New York, NY, USA, 1945-1948. [9] Mankoff, J., Dey, A.K., Hsieh, G., Kientz, J., Lederer, S., Ames, M., Heuristic evaluation of ambient displays. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '03). ACM, New York, NY, USA, 169-176. [10] Pousman, Z., Stasko, J., A taxonomy of ambient information systems: four patterns of design. In Proceedings of the working conference on Advanced visual interfaces (AVI '06). ACM, New York, NY, USA, 67-74. [11] Vogel, D., Balakrishnan, R., Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST '04). ACM, New York, NY, USA, 137-146. 23