Systems for Context Awareness: Challenges & Opportunities

Pankaj Mehra
CTO, Fusion-io, Inc.

Abstract
We examine contextualization as a way of improving the business value of user information. Focusing then on behavioral data, we outline the benefits of activity recognition from the points of view of improved abstraction and compression. We briefly visit the high level design of context engines, before focusing exclusively on system aspects. Our experiences suggest that rapid contextualization favors unified scaling, an architecture in which memory and storage functionalities are converged near the CPU.

the moments they open an app (e.g., the Amazon app on their iOS device), or type a search query into Google. A retailer who wishes to participate in the online bidding, or to make a recommendation on the basis of the query, has but a fraction of a second within which to extract, correlate, and decorate the users intent to purchase any entitiesor entity classesimplied by the users behavior.

Contextualization
The information economy has evolved from an attention economy (a battle for eyeballs) to an intention economy (a battle to figure out the large context of end users). As explained in my previous work (Mehra, 2012), the large context of users is a complex, multi-dimensional data set, sometimes integrated over space and time, and across different collection points and privacy domains. Large context is everything that a user experiences, reads, and especially does, speaks or writes. It shapes how users perceive a message and interpret specific and often ambiguous words such as apple or call. It also determines how devices and services interpret users activities and utterances. The benefits of gathering and analyzing large context data are many, however almost all of them arise from contextualization. The most straightforward way to understand contextualization is to view it as real-time (cf. historical) integration and analysis of context data. Implicit in that view is the idea that the most important part of user context is that which concerns the here and now, also known as situational context. However, given the oft noisy and ambiguous nature of context data, the larger context provides the patterns and statistics with which to disambiguate and derandomize the real-time input. The faster and deeper the contextualization, the greater is the benefit. By way of illustration consider an example from online retail. A users intent to purchase may become apparent in

Figure 1. The Benefits of Timely Contextualization

As shown in Figure 1, online information about users is hard for one vendor to corner; therefore, as the information spreads digitally to various subscribers (and thereby becomes priced in or expected), its differential value to a particular business is significantly diminished. Likewise, there is a rapid decline in the business value of context data with the passage of time, as staleness, change, and even competitive offers, creep into the situation. These aspects of contextualization link it closely with the idea of delivering the right information to the right person at the right time. In the words of Gartner analysts A. Johnson, et al. (Johnson, 2010), timely integration and processing of context data sharpens relevance at moments of truth.

Behavior Data and Activity Context

Activity-context awareness focuses on capturing and analyzing human behavior (what is a user doing?) in the

hope of understanding why. Various theories in psychology (Hopfinger, 2000) and philosophy (Dennett, 1981) link intentions to actions, and lend credence to statistical and analytical work in the computing realm. In fact, Hopfinger argues in favor of a top-down model of human attention in action contexts, meaning our intentions dictate what we find useful in the course of an action. And Dennett argues that human behavior is best modeled using the intentional stance. Some passively generated user data (unrelated to actions), such as the users gender and age, might still possess considerable predictive power in situations. However, its relatively static nature suggests it should serve as the target, rather than the source, of the real-time information integration that is the very definition of contextualization. Behavior data may come from sensed context, such as that generated by a Jawbone Up device. Or it may come from a highly contextualizing app, such as iMapMyRun. Or, perhaps from a trace of search/browsing behavior. Additional sources of behavior data may include digital communications, such as e-mail, and social interactions, such as the acts of posting, sharing, liking, following, and commenting in online social networks, such as Facebook and Twitter. Surveillance cameras or sophisticated video monitors in home and retail situations may yield derived context of human behavior, such as putting things into shopping cart, or as in the case of Feuz and Cooks paper in this workshop, the action of doing something in the kitchen (probably, cooking). The utility of activity-driven interpretation of sensed context into derived context is evident in the work of Han, et al. in this workshop, in their paper about Platys. In information-theoretic terms, an activity-centric interpretation of raw behavior signal produces a form of compressive sensing known as activity recognition, which represents sufficient detail about the behavior as to reproduce the essential intention-related aspects of it, while correctly modeling the rest of it as noise. Just imagine if a future version of a Maps application that integrates with the instrumentation in your car had to always work with the raw trace of your entire GPS location history, instead of being place-aware about where your home is, where your work is, and then perhaps integrating your current GPS location. Ultimately, activity recognition simplifies contextualization.

the vehicle for guiding meaning-driven integration of the different dimensions of context data, which have origins in different devices and data sources. In this workshop, the tradition continues with the paper by Rodriguez, et al. about multi-dimensional ontologies. We can model the (even larger) context of activities as a theory, i.e., a collection of facts (entities and their interrelationships) and rules. By lifting these facts and rules to a particular (sparsely described) situation, context enables inference. Inference is the foundation that makes devices and services look smart and capable of working with meager clues about users intentions and preferences. Beyond representation of activities and activity contexts, the discipline of building activity-context aware systems must deal with speeds and scales. It is all about inetgrating a lot of data, and bringing the analysis of it to bear on moments of truth, those fleeting opportunities where a tiny fraction of that data has high value.

Systems for Activity-Context Awareness

Context engines are an emerging class of computing platforms designed to sift through billions of context clues in order to rapidly identify the few key entities of interest in a particular situation. By retrieving and analyzing a vast number of potentially relevant context clues, a context engine seeks to sharpen relevance. Due to the degree of information compression involved, context engines are designed differently than databases and search engines. In particular, it is no longer apt to pile up mountains of unprocessed data, and then apply either the needle-in-a-haystack style index building of search algorithms, or the heavy query processing approach of database management systems. Instead, as illustrated above by the idea of activity recognition, a context engine seeks to apply multiple layers of data reduction through abstraction, even in the store path. The implications of this on the information infrastructure underlying context engines are quite profound. In the course of helping large enterprises and intelligence agencies with designing their engineering systems to cope with massive amounts of time-critical data, I have discovered that certain system engineering principles need refinement and amplification. Take for instance the idea of scaling out: addressing bigger problems by adding more nodes to a system, rather than building a bigger system. It is still the right idea for building context-aware systems. However, there have been two separate interpretations of this idea, and only one of them fairs well against the stringent requirements of rapid contextualization.

Principled Context Engineering

Since the pioneering work of Guha and McCarthy (2003) on modeling context, reasoning with context theories has been a very big part of doing something intelligent with context data. That thread has continued well through recent work, with the advent of ontologies as

In the first, computational resources are scaled out separately from storage resources. So, for instance, in such designs, one finds a compute grid that has high-bandwidth connectivity to a storage grid or a data grid. Recent interpretations of this design layer an in-memory ad hoc query framework (SAP Hana) atop a Hadoop Distributed File System (HDFS) based hindsight-oriented analytics and data reduction framework that exploits sequential throughput and cheap capacity of HDD-based storage. The designs offer excellent benchmark throughputs but have terrible cost/scale slopes and in context-aware computing terms, take hours (sometimes days) to contextualize new information. What we have found by interviewing customers is that the slope is dictated by the availability of enough DRAM to fit the working set of complex analytical calculations needed for entity extraction, correlation and master data management, the fundamental operations of contextualization. In the second, a unified scaling concept is employed. Each node packs high-speed flash-as-memory storage that lives closer to the CPU than traditional HDD-based storage. Low-latency processing is available in both the store path and in the query path, leading to contextualization speeds that can be measured in mere tens of seconds. The designs not only exhibit both excellent latency and throughput performance, but the cost/scale slope can be lower due to both unification and due to more aggressive use of flash as memory, at a cost about 5x lower than DRAM. Neither of these is sufficiently fast for the future of context-aware computing. Aggressive improvements in memory-storage hierarchy and improvements in latencytolerant techniques (Dean and Barroso, 2013) are both needed in order to bring scales of the context engineering infrastructure up, and the costs down, enough to truly power the future of context awareness.

References
Pankaj Mehra (2012): Context-Aware Computing: Beyond Search and Location-Based Services. IEEE Internet Computing 16(2): 12-16. J.B. Hopfinger, et al. (2000): The neural mechanisms of top-down attentional control. Natural Neuroscience 3(3): 284-291. Daniel C. Dennett (1997): "Chapter 3. True Believers: The Intentional Strategy and Why it Works", in John Haugeland, Mind Design II: Philosophy, Psychology, Artificial Intelligence. Massachusetts: Massachusetts Institute of Technology. ISBN 0-262-08259-4 (first published inScientific Explanation, 1981, edited by A.F. Heath, Oxford: Oxford University Press; originally presented as a Herbert Spencer lecture at Oxford in November 1979; also published as chapter 2 in Dennett's book The Intentional Stance). R.V. Guha and J. McCarthy (2003): "Varieties of Contexts," Proc. 4th Int'l and Interdisciplinary Conf. Modeling and Using Context (CONTEXT 03), LNAI 2680, Springer, pp. 164177. V. Agashe, et al. (2013): Harness Insights from Hadoop with MapReduce and Text Data Processing, SAP, online. Jeffrey Dean and Luiz Andr Barroso (2013): The Tail at Scale, Communications of the ACM, 56(2).

Conclusions
The key opportunity and the source of many key challenges in context-aware computing is rapid contextualization. Activity-awareness and in particular activity recognition is seen as an abstraction vehicle for context data, whose realization should result in reduction of both noise and dimensionality. Even so, the challenge of processing massive amounts of context clues and to rapidly contextualize them against a mass of accumulated information remains a challenge for future systems. It appears that a combination of unified scaling out, storage-memory convergence, and architecting systems for latency tolerance could result in activity context aware systems that are also practical at scale.