Beyond the Smart Home

Tatsuya Yamazaki Universal City Group, Knowledge Creating Communication Research Center National Institute of Information and Communications Technology Abstract
Due to progress in wired and wireless home networking, sensor networks, networked appliances, mechanical and control engineering, and computers, we can build smart homes, and many smart home projects are currently proceeding throughout the world. However, we have to be careful not to repeat the same mistake that was made with home automation technologies that were booming in the 1970s. That is, [total?] automation should not be a goal of smart home technologies. I believe the following points are important in construction of smart homes from users’ viewpoints: development of interface technologies between humans and systems for detection of human intensions, feelings, and situations; improvement of system knowledge; and extension of human activity support outside homes to the scopes of communities, towns, and cities In the 1990s, so-called ubiquitous [1] or pervasive computing technologies arose from progress in wired and wireless home networking, sensor networks, networked appliances, mechanical and control engineering, and computers. Using these developed technologies, researchers have started smart home projects in every corner of the world. Each project has a unique goal, and some interesting discoveries have been made. At this stage, however, we have to be careful not to repeat the same mistake that was made with HA technologies; as Intille advises, “Researchers and technologies are more cautious in predicting the future of the home,” [2]. He also describes, “A survey of ongoing work shows that there is a bias in research toward creating automatic home environments that eliminate the need to think about tasks, such as controlling heating and lighting, going to the grocery store, scheduling home appliances, and cooking.” I dare say that we must go beyond smart homes that automate all tasks for us. Toward the goal of “going beyond smart homes,” the following points should be considered: development of interface technologies between humans and systems for detection of human intensions, feelings, and situations because interfaces should not be unidirectional, but interactive and spiral; improvement of system knowledge, for example, by incorporating Web technologies and making it flexible and adaptable to changing lifestyles; and extension of human activity support outside homes because we live not only in houses but also in communities, towns, and cities. Related works are listed in section 2, and section 3 describes in detail the smart home built in our project. In section 4, the Universal City project we are conducting is introduced as an example of smart home extension.

1. Introduction
We can trace the roots of smart homes back to 1970s home automation (HA) technologies. Research has shown that, except for infrared remote controllers of consumer appliances, these technologies failed to improve our lifestyles significantly for several reasons. First, determining economic benefits of HA technologies was difficult. Namely, effects brought about by these technologies were small in comparison with their introduction costs, and they did not contribute to total household energy savings. Also, systems based on HA technologies were so inflexible that new network infrastructures had to be constructed, and these systems were not adaptable to family lifestyle variations. Moreover, HA technologies lacked applications and services that users wanted. Technological capability often lags behind user demand although users sometimes attribute this problem to killer applications and research and development.

2. Related works
Real-life living space test beds are sometimes called smart homes or sensor-embedded houses, and much

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 © 2006

not in a laboratory. In that project. The group wants sensor-driven pervasive technologies to empower people with information that helps them make decisions without stripping them of their sense of control over their environments. Welfare Techno-Houses were constructed in Japan [3]. for example. researchers built a three-story. and evaluation the future of domestic technologies. the House_n group [6] is working toward a vision where computer technology is ever-present. The Ubiquitous Home differs from other test beds in three aspects. Ubiquitous Home layout Although details of daily human behaviors over the long term can be measured in the Sensing Room. The University of Sherbrooke in Canada laboratory is situated at the University of Sherbrooke. The layout of the Ubiquitous Home is shown in Figure 1. In the EasyLiving project. and appliances across data networks. the Sensing Room was constructed at the University of Tokyo [8]. 1. In 1995. Outline We also constructed a real-life test bed. 5040-square-foot home that functioned as a living laboratory for interdisciplinary design. at the University of Tokyo. Research on smart homes built within the last decade is reviewed in this section. that is.research has previously been reported on them. in [13] 3. development. The sensing room of the EasyLiving project at Microsoft Research [9] is similar to the Sensing Room Fig. The MavHome is a home environment that detects home environmental states through sensors and intelligently acts upon the environment through controllers. At the Massachusetts Institute of Technology. Active Home is located at Information and Communications University of Korea where several experiments of context-aware services based on human behaviors are carried out [12]. we have enhanced sensor ubiquity in it. The Aware Home [4] project is noteworthy among smart home research. The University of Florida's Mobile and Pervasive Computing Laboratory is developing programmable pervasive spaces in which a smart space exists as both a runtime environment and a software library [10]. sensors. Ubiquitous Home 3. In the Ubiquitous Home. called the "Ubiquitous Home. It is an information and communications technology (ICT) housing test facility for the creation of useful new home services that will be made possible by linking devices. Extended from Robotic Room 2 [7]. where automated light and monitor control based on users’ situations are demonstrated. The University of Texas at Arlington has conducted the MavHome project since the beginning of the twenty-first century [5]." for home context-aware service experiments [14]. but in a subtler way than is often advocated in popular culture and engineering paper motivation sections. First. there are limitations to imitating real-life behaviors when people are confined to only a small room. UbiHome is another smart space in Korea. The concepts of these experimental houses were to promote independence for elderly and disabled people and improve their quality of life. researchers developed prototype architectures and technologies for building intelligent environments.1. the atmosphere there is different from a real-life one. experimenters can collect real-life data as if living in their own houses.00 © 2006 . we have installed many cameras and microphones in each room 2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.

a Japanese-style room is provided as a living space for extended family members. The former uses 315-MHz waves. When a person wearing an Figure 3. Camera and microphone in ceiling Figure 4. The Ubiquitous Home is equipped with various sensors to monitor human activities. Infrared sensor Four accelerometers or vibration sensors are attached to the underside of the bedroom floor in all four corners. Active system scanners.45-GHz band. bedroom. washroom. and bathroom. there are three corridors called "catwalks. Between the apartment and the Japanese-style room. 3) in the ceiling to gather audiovisual information. Each room has cameras and microphones (Fig. 4) installed throughout the flooring contain 18-by-18-cm binary detection units and are used to track residents’ movement or detect furniture positions. Respecting residential privacy in such ubiquitous camera and microphone environments is an important issue to consider. Although the role of accelerometers is similar to that of floor pressure sensors.00 © 2006 . Floor pressure sensor Figure 5. Ubiquitous Home robot The Ubiquitous Home is an apartment comprised of a living room. One is active and the other passive. RFID tag passes through an entrance. In addition to the apartment. the latter the 2. Infrared sensors (Fig. 2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20. We have also set up a remote Japanesestyle room in which we can test extended family connectivity and create a test bed for specific Japanese services. such as a grandmother and/or grandfather. Floor pressure sensors (Fig. Two radio frequency identification (RFID) systems are installed in the Ubiquitous Home. an antenna reads information on the tag. Passive system antennas are embedded inside walls around each room entrance. is a computer room called the Network Operating Center (NOC). at foot positions in the kitchen and along the corridor are used to detect residents’ movement. detect RFID tags when a person wearing them enters the room. In the space. Above the ceiling is a space where researchers can work. we have introduced robots in home services (Fig. Moreover. there is a 40-cm space between the foundation and floor of the Ubiquitous Home to accommodate free access. dining room/kitchen study. 2). they are more promising for detection of human behaviors because of their higher precision than floor pressure sensors." Researchers use them for moving around and installing machines and cables. Figure 2. located above the ceiling of each room. Furthermore. 5) installed above the entrance to each room.and various sensors to monitor locations in every space of the home.

Although the definition of user context may change based on the situation or service. if a word in a TV program in the EPG appears in the viewing history. a plasma display panel. “who. we should consider their contexts. When he/she leaves. it does so using network commands and selects TV programs that the resident is likely to watch by comparing the Electronic Program Guide (EPG) and the resident’s TV viewing history. 4. The refrigerator has a camera inside to capture images of its contents. The resident’s location is also detected from the active RFID tag. With the above contexts the following contextaware services are implemented in the Ubiquitous Home. personal identification is obtained from an active RFID tag worn by the resident or face recognition by a robot camera. The robot shown in Fig. Context-aware services implemented in the Ubiquitous Home To provide a service to users.” and “when” are usually the main contexts.Plasma panel and liquid crystal displays (Fig. For example. on the other hand. and the images are displayed on the display panel closest to the resident. (3) Forgotten-property check service An RFID tag is attached to each article a resident removes from the Ubiquitous Home. serves as an intermediary between the inhabitants and the unconscious-type robot by providing a human-machine interface. 6) and speakers are installed throughout the Ubiquitous Home to provide residents with audiovisual contents.00 © 2006 . 2. Figure 6. that TV program is recommended. It autonomously controls appliances on the home network based on sensor information. and time is provided by a computer clock that is adjusted by a Network Time Protocol (NTP) server. In the Ubiquitous Home. the robot selects a recipe related to that word. upon which. (1) TV program recommendation When a resident orders the robot to turn on the television in the living room. 3. We have also implemented service that demonstrates collaboration of a refrigerator. Plasma display on wall The Ubiquitous Home is a highly networked and sensor-embedded home and is considered as the unconscious-type robot that is a robot of which residents are not consciously aware. In addition.” “where. From the Ubiquitous Home to the Universal City 2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20. The resident can continue searching for recipes by talking with the robot until the desired recipe is found. it is displayed on the display panel in the living room or the kitchen. and the area RFID system. (2) Cooking recipe display When a resident says a food-related word.2. articles he/she should bring with him/her are listed for each outside destination. This is the visible-type robot that is a robot of which residents are aware and on which they rely. he/she can verify which articles to take with him/her at the entrance hall using the RFID tag reader installed in the shoebox.

it is vital to observe real life human activities and behaviors and model human activities and cognizance based on data obtained through such observations. when considering necessary frameworks for building the Universal City. The goal must not be to construct automatic home environments. Community-adaptive communications technologies are communications infrastructure technologies necessary for building "Ubiquitous and Universal Towns. In other words. The Internet provides indiscriminant access to information from anywhere in the world as technologies mutually linking entities scattered around the world. communities linking them. We have to construct smart spaces that connect insides and outsides of homes seamlessly. From the environmental side.” I stated some important points that need to be addressed: development of interface technologies between humans and systems for detection of human intensions. Although we have not been able to achieve this goal yet. and transmit it to them. and extension of human activity support outside homes to the scopes of communities. I presented the Ubiquitous Home project that we have previously conducted and the Universal City Figure 7. towns. iii) barrier-free communications technologies as exemplified by intelligent mobile terminals and barrier-free information interfaces technologies. improvement of system knowledge. and situations vary for individual users at various moments. we are conducting research and development for constructing a society in which an entire town will keep a gentle eye on residents thereof and visitors thereto. including the elderly. To do so. and children. Based on the accumulated research and development results on i) home network infrastructure technologies utilizing a real living-environment-type home ubiquitous environment test bed such as the Ubiquitous Home.00 © 2006 . visible light communications. power line communications (PLC). since research and development on communications technologies is being conducted in a multifaceted manner." Preferences. including wireless and wired communications. it is vital to enable continuous and precise transmission by storing useful real-world information with detailed information on time and location coordinates at distinct corners of every town. To this end. and sensor networks. The key technologies necessary for building the Universal City are depicted in Fig. feelings. We will now refer to people who use systems and receive information and services as "users." I believe such technologies will adapt to physical sizes. the challenges are to extract useful knowledge from information acquired by networked sensors. and situations. To make seamless information transmission/reception possible between heterogeneous communications. users’ activities cannot be limited to inside their homes. In addition. For this purpose. and cities. we must "know" real users. Where referring to all these factors as a user environment. we have started the Universal City project to build a nextgeneration intellectual living environment hospitable to everyone. useradaptation technologies and community-adaptive communications technologies will comprise the pillars of the plan. Toward the goal of “beyond smart homes.As described in the introduction. Key technologies in Universal City Information and communications technologies will be used further in a variety of situations at home and in offices. such as outdoor and community activities and lifelong learning. Even if we are not transmitting it in real-time. abilities. technologies being used differ at each location. and support itself in a variety of aspects. Furthermore. Conclusion I addressed issues that technologists and researchers in smart home research and development should consider. specify people who need such information. people with disabilities. ii) technologies for providing situation-adaptive-type services. we will conduct studies on user-adaptation technologies on a range of accumulated and analyzed data. We must go beyond smart homes that automate all tasks for us. Networked robots Service content Universal City Universal interface Ubiquitous Home At anytime Sensor models to establish technologies for extrapolating user intentions from the data. From and to anywhere Every product Everyone 5. we will address challenges of constructing the Universal City during the next five years. 7. we have to develop universal designs adaptable to each user environment. including homes. and towns containing other facilities and enable flexible information delivery suitable for their content within and/or between entities. we must conduct research and development on technologies for comprehensively using those communications. as well as on 2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20. To achieve user-adaptation technologies.

” Med. Blair MacIntyre.. King. [8] Taketoshi Mori. [6] Stephen S. “The Goal: Smart People. 2001.” IEEE Pervasive Computing. [12] Jae-Seon Lee. Kyoung-Shin Park. Intille. Physics. E. Vol. B.” C. Thad E. Mayers. “Fully automated health monitoring system in the home.” Proc. Y. March 2006. J. Meunier. J. H. Aritoki Takada. Assistive Technology Research Series 19. Christopher G. Gregory D. [5] S. Ogawa. 2000. Tamura. “EasyLiving: Technologies for Intelligent Environments.” Scientific American. pp.” 5th International Conference on Simulations in Biomedicine. 1999. [13] Yoosoo Oh and Woontack Woo. “The Aware Home: A Living Laboratory for Ubiquitous Computing Research. Augusto (Eds. W. and M. and Tomomasa Sato. June 2005. C. on Ubiquitous Computing Systems (UCS2004). Yoda. April 2003. pp.. T. Mann. A. 2002. Kidd. 80–86.). and Shafer. Kern. [7] Taketoshi Mori. pp. S. Nugent. Abowd. A. March 2005. 117–122.00 © 2006 . O. 2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20. J. 94–104. “Gator Tech Smart House: A Programmable Pervasive Space. “The Role of Prediction Algorithms on the MavHome Smart Home Architectures. Irfan A. Eng. J. Sept.” Proc.project that we will move on to during the next five years. Vol. 3–6. “The Computer for the Twenty-First Century. 6. “WindowActive: An Interactive House Window On Demand. [3] T.CoBuild'99. Elzabadani. Springer-Verlag. Katsutoshi Asaki.. [2] Stephen S. 2004. Krumm. 20. 2004. “IEEE Computer magazine. “Designing a home of the future. IOS Press. Slovenia. 4th International Conference On Smart homes and health Telematics (ICOST2006). B. Helal. Orr. and E. pp. IROS (IEEE/RSJ International Conference on Intelligent Robots and Systems) 2001.” Proc.. [10] A. September 1991. and T. Smart Homes and Beyond. “Accumulation and Summarization of Human Daily Action Data in One-room-type Sensing System. Heierman. References [1] Mark Weiser. [4] Cory D. and Giroux S. pp. Cook. Intille. and Min-Soo Hahn. pp. Elizabeth Mynatt.-Y..” Proc. Hiroshi Noguchi. pp. 1998. Kaddourah. [11] Pigot. Meyers. Kerhervé. Hiroshi Noguchi. 9. Togawa. [9] Brumitt. 2002. pp. Tokyo. Bhattacharya. M. H. Das. Jansen.” IEEE Wireless Communications (Special Issue on Smart Homes). A. Lin. Robert J. of First International Workshop on Networked Sensing Systems (INSS2004).” Proc. No. Dec. J. in LNCS 1927. “Sensing Room: Distributed Sensor Environment for Measurement of Human Daily Behavior. Lefebvre. 481–484. G.” Handheld and Ubiquitous Computing 2000.. of the Second International Workshop on Cooperative Buildings .” 1st Korea-Japan Joint Workshop on Ubiquitous Computing and Networking Systems (UbiCNS 2005). Atkeson. 573–579. “Human Action Detection and Context-aware Service Implementation in a Real-life Living Space Test Bed. pp. 40–43. 64–74. June 2006. April–June. 2349–2354. 77–84. and Tomomasa Sato.” pp. “The role of intelligent habitats in upholding elders in residence. D. and Wendy Newstetter. Not Smart Homes.. B.. [14] Tatsuya Yamazaki. K. Starner. Essa. “A unified Application Service Model for ubiHome by Exploiting Intelligent Context-Awareness. TridentCom 2006 (Second International Conference on Testbeds and Research Infrastructures for the DEvelopment of NeTworks and COMmunities). Symp.. B.. of Second Intern. III.