Games in AR: Types and Technologies

Andrea Phillips Neogence, Inc.

ABSTRACT To unlock its full potential, augmented reality gaming will need to integrate several types of technology, including RFID; motion, image and face recognition; location awareness. Different types of play mechanism enabled by integration of each of these technologies are discussed. 1 INTRODUCTION

games are very similar to existing board and tabletop games. 2.1 Stationary Marker-Driven AR Games

Augmented reality seems like the perfect playground. In theory, it allows games of pretend where the imaginary characters act of their own accord, and new sports with complicated rules and elaborate scoring systems handled invisibly by computer. In practice, though, augmented reality alone is not enough to bring these games to life. The potential of AR gaming will be unlocked when it is integrated with other technology – most of which already exists – including RFID, GPS, and motion detection. As they are integrated, the potential for AR gaming will explode. 2 STATIONARY AR GAMES

These games may be very similar in feel and execution to existing games. One might wonder at whether it’s worthwhile to develop such a game; but consider the success of solitaire card games for the PC. Any implementation of a popular game that allows some of the gamemanagement overhead to be controlled by the computer brings a value-add of low effort that could boost the adoption of AR versions of already familiar games. 2.1.1 Arena dueling games These games are similar to Magic: The Gathering, Pokémon, and Yu-Gi-Oh. The players (generally two) conduct a battle by gradually placing new markers (probably cards indicating creatures, abilities, or other concepts) into the field. The computer mediates the markers' interactions with one another and keeps track of score and other vital statistics. 2.1.2 Battle chess This is a popular game concept, from Star Wars to Harry Potter. It involves a traditional strategy game, such as chess, in which the pieces perform animations in AR that will vary depending on their relative position versus other pieces on the board. 2.1.3 Tabletop strategy games The non-AR analogue to this is Warhammer 40K. In AR tabletop strategy games, markers are

Stationary AR games based on marker technology are the first and lowest hurdle we expect to be leaped. In fact, several developers are working on such games already, though none have yet made it to market; they should between Q4 2009 and Q3 2010. For the most part, these Twitter: @andrhia

moved freely around on a table; the player inputs actions such as attacks or building fortifications, and the game computes damage and other statistics. 3 MOBILE AR GAMES

score. We anticipate set-collection games run as marketing efforts, with markers placed on promotional materials such as posters, billboards, and packaging. 3.1.3 Assassin/laser-tag style games Players tag themselves, either with markers printed on stickable labels, or by purchasing and wearing items such as T-shirts and hats with preprinted markers. Players disperse into an environment and then try to tag one another. A tag would be complete when one player targets and "shoots" another player's marker through the viewfinder of the mobile AR device. 3.2 Mobile Markerless AR

The next step in the evolution of augmented reality gaming will come when the AR device is untethered. This requires a portable camera, display, computer, and some interface mechanism. While wearable computing devices and AR headsets would be ideal, a smartphone should be sufficient. 3.1 Mobile Marker-Driven Games

A mobile AR device relying on markers will still be extremely limited, but mobility does open up new genres of play. They should be possible even under current technology; such games should start to appear on the market between Q3 of 2009 and Q2 of 2011, depending on market saturation of the devices for which development first occurs. 3.1.1 Puzzle hunts These are likely to be limited-adoption smallscale pervasive games. A designer places markers in predetermined locations. Upon view via the mobile AR device, each marker displays a clue toward the location of the next marker. This style of puzzle hunt could be repurposed to serve as a self-guided walking tour of a neighborhood, theme park, or museum; or even to convey a narrative, such as a mystery. 3.1.2 Set-collection games Markers are interspersed into an environment, and the player must seek out each marker. Upon view, the marker would likely play a set animation, possibly appearing to interact with other elements in its environment. When each marker is located, the computer adds it to an online storehouse and adds points to the player's

While the games using AR markers will certainly be entertaining in their own right, it is only when the technology allows markerless AR that the games designed will start to become native to augmented reality, allowing completely new mechanisms of play rather than mere adaptations of existing ones. In these games, a player would view his or her surroundings through either a viewfinder on a mobile device or a camera/display headset. Software would detect objects and perspective and overlay AR information accordingly. Some simple implementations of markerless AR operating on a small scale should appear by Q1 of 2010. First implementations are likely to include simple arcade and puzzle games. 3.2.1 Shooting galleries AR objects are displayed in the environment and players must locate them in the AR viewfinder and shoot them. The objects could be asteroids, aliens, ghosts, or

3.2.2 Whack-a-mole Objects appear and disappear from the field of view very quickly and the player must target them or otherwise catch them before they disappear. (This is only one in a nearly limitless variety of possible fast-reflex games.) 3.2.3 Hidden object Virtual objects are concealed in the detected surroundings, and the player must locate them (presumably in as little time as possible.) If local object detection is sufficiently sophisticated, AR objects may be "hidden" behind and under real, physical objects. 3.3 Location Detection

games such as Quake or Halo. This game type could easily become multiplayer if the AR devices are communicating with one another; this game then becomes similar to marker-driven assassin/laser-tag games, but without the need for a marker. 3.3.2 Mazes The player must navigate AR constraints tethered to real-world locations in order to navigate a maze. The AR constraints could appear as shrubs, walls, or simply lights in the air, and the path of the maze could either correspond to existing streets, or be entirely artificial and drawn within a large open space such as a sports field or gymnasium. 3.3.3 Mobile platformers This is a close cousin to games such as Pac-Man or Super Mario Brothers. The player navigates challenges in an environment -- say, collecting coins and avoiding ghosts -- to reach a specific destination or a high score. 3.4 Other Technologies

Games using markerless AR and location detection – probably via a GPS-and-compass combination – should appear by Q4 2012 at the outmost, and likely quite a bit sooner. Note that two varieties of location detection are possible. In one, the system relies on static maps of an area to draw a playing field. These would likely be implemented outdoors first, as the interior of any given building would be highly unlikely to be mapped. In dynamic mapping, the computer would parse the environment for objects and continue mapping on the fly as the player moves and perspective and location both shift. Ultimately, an efficient system will likely rely on both techniques: Systems would access a Google Maps-style datastore with size, perimeter and location information for large static objects, while dynamic mapping would account for the presence of mobile objects such as vehicles and people. It's unlikely that a completely integrated system will exist until sometime in 2013, and possibly not for quite some time afterward. 3.3.1 Mobile first-person shooter A single player both seeks out and hides from AR opponents in the style of first-person shooter

Some of the possibilities of augmented reality gaming rely more on input or data collection interfaces than any other single technology. Stationary games associated with a desktop or laptop computer can rely on a full keyboard and mouse/trackpad setup. Mobile devices can make use of touchpads, or simply use a static point on the viewfinder as the cursor point. Speech recognition, particularly when integrated with a mobile AR device, will allow for more complex functions and games. But motion detection and face or image recognition will allow the kinds of games that often come to mind when “augmented reality games” are discussed. 3.5 Motion Detection

Games relying on physical motion on the part of the player will open the avenue to entirely new

categories of sport and exercise. To be sure, there are larger uses for motion detection in AR, including an interface based on a gesture vocabulary or the ability to interact with a virtual object in the same way as a physical one, but these games are not easily tackled by existing taxonomies. 3.5.1 Exergames Motion detection allows just about any kind of exergame that could be imagined; the concept of a game like Kinetic for the PS2 and Eyetoy could realize its full potential, as would boxing, fencing, and dance games. The possibility is strong that this technology could also be used to teach real skills that require a high awareness of proprioception: martial arts, yoga, golf, etc. 3.5.2 Quidditch and other fantasy sports Obviously augmented reality won't let anybody fly on a broomstick, but fantasy elements of a game like Quidditch -- balls that move of their own accord and with their own goals -- could definitely be ported to an augmented reality interface successfully. The best of these games will be completely native to AR, and may not have clear antecedents in traditional games or sports. 3.6 Face and Image Recognition

3.6.1 Pervasive MMORPGs These technologies together would allow roleplaying games to be played pervasively on a scale never before seen. Networked AR devices could alert players to others in close physical proximity, and the ability to identify common objects would allow the game to overlay them with a pervasive environment consistent to the game world. Pervasive MMORPGs will likely be a very long time in development, due to the many technologies required. We anticipate that one or more will launch sometime between 2015 and 2020, depending on how quickly AR technology saturates the market.

Identity recognition -- of people or of objects -- is another technology that will significantly advance the gaming possibilities of AR. RFID chips embedded in everyday objects will allow the AR game to detect the existence, location and purpose of everyday items such as furniture, vehicles, or clothing. Facial recognition will allow the AR environment to paint individuals with clothing, accessories, or other personal attributes in keeping with a game. Together, these technologies would allow one of the most ambitious goals of augmented reality gaming: the pervasive mobile MMORPG.