Multimodal layer surveillance map

based on Anomaly detection using

multi-agents for smart city security

Hochul Shin1, Ki-In Na1, Jiho Chang1, Taeyoung

Uhm2
 Background / Introduction

– Use of cameras everywhere around us

– Safe city projects and other facilities need surveillance
– Smart cities are expected to provide residence with convenience through multiple
agents
– Agents can collect environmental data and used for advertising and security
monitoring.
– Collecting data from all the agents
Literature Review

They all worked with Security Robot Applications

– Meghana used RFID tags and metal detector sensors to detect metal bombs in outdoor
environments such as large facilities, critical infra structure, and borders.
– Chakravarty proposed a mobile robot that repeats a manually trained path and detects
visual anomalies that did not exist during training.
– . Zhang proposed a robot system to patrol around a dam using a multi-sensor-based
system.
– Di Paola proposed an autonomous mobile robot for surveillance in an indoor
environment and implemented a multi-sensor platform, including a monocular camera,
a laser scanner, and an RFID sensor
Literature Review

– US government public data site has – UCF101

a dataset of over 183 500 US cities, – Mivia
with an average of 2791 new
– TV Human Interaction Dataset
datasets growing every month
– SCFace – UCSD

– UTKFace – Bilkent

– IMDB-WIKI – TRANCOS

– UCFCC50 – BIT Vehicle

– GTI Vehicle
Research Gap

– A method for transmitting and processing large amounts of data from multiple
agents is necessary for real-time applications
– Large amount of data
– Review on different data
– Intensity variations (variations occur due to different viewpoints, occlusions,
different movements in agents)
Methodology

Map-based surveillance
The observation map is generated in two stages:
– Each agent converts large multi-modal data in a multi-layered local map and
transmits it to the server
– The server integrates each local observation map to develop a global multi-
layered observation map
– The server computes the global probability map for the current observation map.
– Information is converted in to a surveillance map.
Methodology

Multi-modal data collection and processing agent

– Multi-modal data collection from heterogeneous multiple agents
– Local multi-layered observation map building
Multi-modal data collection from
heterogeneous multiple agents
– Moving agents, fixed agents
– Monitoring efficiency can be increased by running a fixed agent and a mobile agent.
– Mobile agents collects data by using RGB camera, thermal camera, and 3D light
detection
– It estimates global position and orientation by using Global Positioning System.
– Fixed agent is installed on a building where monitoring is required.
– It only detects human and vehicles with RGB camera.
Local multi-layered observation map
building
– Each agent can develop a local multi-layered observation map centered on itself
using multi-modal sensors.
– The observation map is then sent to the data analysis server.
– People and vehicles are used as objects
– The object map represents the moving direction, speed and density of object.
– The thermal map converts temperature in a temperature distribution in 3D space.
Cont..
Conclusion

– This research article presents a data framework that efficiently process

multimodal data generated from multi agents.
– Surveillance map consists of observation map and a probability map.
– It can also express environmental information.
– It allows smooth data transmission and processing via multi-agents.
– It transforms multimodal data in a local observation map and a global
surveillance map in server.
– Global observation is used to generate a probability map.
Conclusion

– Here they only used observation data for human and vehicles.
– It can also be used to detect unusual situations like anomaly.
– It presented an anomaly detection algorithm which comprises in two stages
– Candidate extraction and filtering.
– It is expected that this algorithm will apply not only to security services but also
to various applications using environmental information for multi agent systems
in smart cities.