Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Digital Object Identifier 10.1109/ACCESS.2023.0322000

Allen Robot: A Modular and Adaptive Solution

for Efficient Customer Service in Shopping Malls
HAMZA ANWAR12 , HIRA MOHIUDDIN1 , EHTISHAM AHMED1 , KHAWAR ALI1 , RAMSHA
SHAHID1 , RIZWAN TAHIR1 , and DR. ANAYAT ULLAH12
1
Control, Automotive and Robotics Lab (CARL), National Centre for Robotics and Automation, Department of Electronic Engineering, Balochistan University of
IT, Engineering and Management Sciences, Quetta, Pakistan.
2
Department of Electronic Engineering, Balochistan University of IT, Engineering and Management Sciences, Quetta, Pakistan.
Corresponding author: Hamza Anwar (e-mail: hamzaanwar5893@gmail.com)

ABSTRACT Social service robots have emerged as versatile and interactive solutions in various domains,
revolutionizing human-robot interaction and shaping the future of service-oriented industries. This research
addresses the growing demand for affordable social service robots, particularly during the COVID-19
pandemic when there is a need for reduced human contact. To meet this demand, a state-of-the-art social
service robot named "Allen" has been developed. Allen is specifically designed for deployment in malls
and theme parks. The study makes significant contributions by integrating advanced technologies and
incorporating human-centric design principles into the robot’s development. These enhancements enable
Allen to effectively engage with visitors, enhance their experiences, and provide efficient assistance. The
robot’s appearance resembles that of a minion and is equipped with sensors, including LiDAR and depth
cameras. It utilizes Simultaneous Localization and Mapping (SLAM) techniques, path planning algorithms,
and obstacle avoidance algorithms for navigation. Allen interacts with customers through graphical and
speech-based interfaces and employs Natural Language Processing (NLP) techniques for query answering.
The robot integrates state-of-the-art NLP algorithms, such as ChatGPT, to facilitate more natural and human-
like user interactions. Field trials conducted in a shopping mall have yielded positive results, indicating
improved customer satisfaction, increased footfall, and an enhanced customer experience. These findings
demonstrate the effectiveness of Allen as a social service robot in real-world scenarios.

INDEX TERMS Social Service robots, natural language processing, NLP, chatGPT, Alan, smart robot,
human-machine interaction, HMI, human-robot interaction, HRI, interactive robotics.

I. INTRODUCTION algorithms, and Large Language Models (LLMs) [5]. These

technologies have paved the way for robots to possess traits
S OCIAL service robots have become ideal counterparts in
public spaces, serving a wide range of industries such as
hotels, healthcare, and entertainment. They are categorized
such as communication, cooperation, guidance, and decision-
making [6]. These traits play a crucial role in the successful
deployment of the social service robot. Effective commu-
by ISO 8373-2021 standards, as shown in Fig. 1. The global
nication enables robots to interact with users, understand
market for social service robots has experienced a boost in
their needs, and provide appropriate assistance. Cooperation
recent years. This growth has been driven by the impact of
facilitates seamless integration into existing workflows by
the COVID-19 pandemic, which has boosted the demand for
enabling collaboration with humans or other robots. Provid-
human-less interaction in public areas [1] [2]. This growth is
ing guidance empowers robots to offer directions or instruc-
reflected by the increase in sales of social service robots up
tions, enhancing the user experience. Additionally, robust
to 13% to 4.6 million units in 2019, with potential growth of
decision-making enables autonomous choices in complex
10% to 5.1 million units in 2020 [3]. This growth is further
situations. The significance of these capabilities is particu-
estimated to reach USD 84.8 billion by 2028 [4], projecting
larly pronounced when deploying these robots across diverse
a compound annual growth rate (CAGR) of 15.4% as shown
industries like hospitality, malls, and education. [7] exten-
in Fig. 2.
sively explored the deployment of social service robots across
This vastly increasing demand for service robots is rein- sectors such as hospitality, malls, and education. Compre-
forced by the advancements in computer vision, navigational

VOLUME 11, 2023 1


FIGURE 1: Taxonomy of service robots as proposed by ISO
8373:2021
FIGURE 2: Service robot market forecast by the year 2028
[4]
hensive insights emphasize the impact of these robots on
consumer-facing services, focusing on two critical factors:
physical appearance and user experience (UX). Milman et
al. [8] explored visitor expectations and preferences regard-
ing anime, cartoon-like, human-like, and animal-like robots.
The findings suggested a slight preference for human-like
robots, although mobility and visitor interactions were not
extensively evaluated. However, another study [9] showed
that a positive user experience (UX) boosts willingness to use
social robots in malls, with a stronger influence from hedonic
characteristics than pragmatic ones.
Based on the aforementioned characteristics; physical ap-
pearance and user experience (UX), several well-known
robots, including Sophia [10], Honda Asimo [11], and Paro
the Seal [12] as shown in Fig. 3 have been developed. Re-
cent studies have discussed many other deployments of such
robots in the entertainment industry, particularly shopping
malls. Robots such as Sylvester [13] have demonstrated their
usefulness in transporting items; however, they lacked the
interactive capabilities required to engage with humans, pro-
vide social and emotional support, and enhance the user
2 VOLUME 11, 2023
experience. In the realm of social service robotics, numerous
researchers have acknowledged the necessity of a human-
robot interaction (HRI) system. A smartphone-based HRI
system for a robotic sales clerk is discussed in [14], which
leads to increased sales and improved customer interaction.
Inverse reinforcement learning (IRL) for HRI was used and
implemented in a geminoid-f [15] android in a mall [16]. The
android was deployed as a receptionist without the ability to
move or guide the customers physically. We have found that
most of these robots have certain limitations, such as limited
movement, sub-optimal user interfaces, and minimal creative
participation.
FIGURE 3: (a) Honda Asimo (Credited: Honda Global) (b)
Paro, the seal (Credited: PARO Robots) (c) Sophia Robot
(Credited: Hanson Robotics)
To address these shortcomings, researchers have incorpo-
rated pre-built robotic platforms to provide consumers with
a more comprehensive and fulfilling experience. Japan intro-
duced its Robovie robot in various shopping malls over sev-
eral years [17] [18] [19] [20] [21]. Several other deployments,
including SoftBanks Robotics’ Pepper robot [22] have been
discussed in various studies, such as libraries [23], restau-
rants [24], train stations [25], museums [26], and shopping
malls [27]. Deploying such pre-built robotic platforms can
be expensive. However, their deployment can significantly
improve the customer experience and boost efficiency by
performing routine services and assisting customers with in-
quiries, directions, and promotional activities.
In economically developing nations, especially those fac-
ing specific market conditions, social service robots offer the
potential to enhance customer services, improve efficiency,
save costs for mall owners and retailers, advance technology,
and increase accessibility and affordability for low-income
consumers. Nevertheless, the adoption and implementation
of these robots may entail considerable expenses. This high
value for deployment arises from the significant costs asso-
ciated with robot training, technical assistance, and mainte-
nance. Additionally, the design of these systems primarily
targets adult interactions, potentially making them less ap-
pealing to children when deployed in malls and theme parks.
This research investigates the potential usefulness of the
developed social mobile robot named Allen, detailing its
design and functioning. Allen is a minion-shaped, medium-
sized social service robot, targeted at the consumer audience.
It is built locally with the vision of being deployed in public
places such as shopping malls and theme parks. The Allen
robot guides the visitors through the various attractions and
answers their queries, as well as interacts with children. It
could play various games, sing poems, and show videos
to them. This study explores the effectiveness of the robot
towards customer satisfaction, increasing the footfall in the
marketplace, and improving the overall experience of the
visitors in the malls and theme parks.
A. RESEARCH OBJECTIVES
Modern shopping malls are becoming increasingly enormous,
with hundreds of different retailers. Customers may become
perplexed and disoriented while attempting to locate a spe-
cific store in the mall. This leads shopping mall managers
to hire guides to improve service quality and the consumer
shopping experience, which raises labor expenses. A service
robot, on the other hand, could be a more efficient and cost-
effective alternative, providing a more appealing information
delivery system to customers through interactive robotic sys-
tems. Based on the aforementioned requirements, Allen robot
can provide consumers with guidance and information, as
well as entertainment services for youngsters.
Mapping a mall presents a significant challenge due to
its complex and constantly evolving nature. To address this
challenge, the Allen robot’s guidance system leverages a
combination of algorithms that enable simultaneous local-
ization and mapping (SLAM), path planning, and obstacle
avoidance. The SLAM technique called Cartographer [28],
which is a grid-based SLAM approach with graph optimiza-
tion, is employed to create an initial map of the environment.
This approach is particularly well-suited for large landscapes
where feature tracking can be computationally demanding.
To improve the precision of SLAM’s localization, the system
utilizes an Extended Kalman Filter (EKF) that combines data
from an Inertial Measurement Unit (IMU), dead reckoning,
and the cartographer’s localization. Once the map is obtained,
the system utilizes the ROS navigation stack for real-time
path planning. The stack includes Adaptive Monte Carlo
Localization (AMCL) [29], a particle filter-based localization
technique, combined with the Dynamic Window Approach
(DWA) planner [30] and the Dijkstra algorithm [31] for local
and global path planning, respectively.
Human-robot interaction is a crucial aspect of social ser-
vice robots. To deliver a satisfactory experience, the interac-
VOLUME 11, 2023
tive system must exhibit human-like qualities, enabling it to
effectively engage with customers and provide them with both
information and enjoyment. The Allen robot employs a com-
bination of graphical and speech-based interaction systems,
which is regarded as the most optimal technique of HRI since
it replicates the fundamental form of communication between
people. For this reason, an animated screen is provided to
the Allen robot to match its minion personality. The robot’s
animated eyes, which move in response to its surroundings,
create a lifelike appearance that draws attention from cus-
tomers and passersby. Its naturalistic eye movements and
attentiveness while listening to customers enhance its social
presence and make it a center of attraction in public settings.
Additionally, it can communicate with customers and answer
their queries through a natural language processing (NLP)
framework.
In general, the main contributions of this work can be
summarized as follows:
• Design and development of a low-cost minion-shaped
service robot, deployable in shopping malls and theme
parks.
• Implementation of Cartographer SLAM to lower the
computational needs in large environments.
• Implementation of a safe navigation system using a
combination of various algorithms to make it robust in a
crowded environment.
• Design of a multi-modal interaction method for a robot
shopping assistant to improve the shopping experience
for customers.
• Integration of ChatGPT into the robot’s interactive sys-
tem to enhance its ability to provide nuanced and sophis-
ticated responses to customers.
• Evaluation of the robot system in malls through field
trials and found promising outcomes.
The paper is organized into several sections to provide a
comprehensive analysis of the robot’s design, functioning,
and effectiveness. Section I provides an overview of the his-
torical background of mobile robots and their significance
in our daily lives. Section II highlights the design principles
and modeling techniques used in the development of the
Allen bot. It provides an in-depth discussion of the software
stack implemented on the robot. Section III presents the
experimentation and field trials in a mall. Finally, Section IV
summarizes the major contributions of the research and high-
lights future directions for improving the performance and
functionality of social mobile robots like Allen. Overall, the
study demonstrates the potential usefulness of social mobile
robots in enhancing customer satisfaction, increasing footfall
in the marketplace, and improving the overall experience of
visitors in public places.
II. METHODOLOGY
A. PHYSICAL SYSTEM DESIGN
The initial phase entailed conducting extensive research on
the design of existing social service robots on the market,
3
 (a) (b) (c)
FIGURE 4: (a) The top part of the robot includes mounts for the camera sensor and screen. (b) The bottom part of the robot
features wheels, motors, and LiDAR. (c) The complete 3D printed body of the robot

with a focus on defining important physical features and TABLE I: Allen robot’s specifications
functionalities needed for the bot while keeping the target au- Items Names and Specifications
dience, particularly youngsters, in mind. Speedy prototyping Physical Specifications
Height 80 cm
and design iteration were achieved through the utilization of Diameter 38 cm
AutoCAD Fusion 360, an industry-level CAD/CAM design Weight 6.5 kg
software. The design displayed in Fig. 4c was adopted using Display 23.98 x 15.86 x 0.85 cm
a combination of 3D modeling and simulation techniques to Technical Specifications
2 x LiPo 3C batteries:
guarantee that the bot was functional and addressed industry- Battery
11.1 V, 5000 mAh
specific demands. TTL Half Duplex,
Fig. 4 shows the physical design and model of the Allen Motor Communication Asynchronous Serial
Communication
robot. The robot is modeled after the famous animated char- Wireless WiFi IEEE 802.11n
acter, Minion. The design was chosen because of its pop- High-level processor Jetson Nano
ularity among children and adults, using its familiarity to OpenCR board: 32-bit
Low-level processor ARM Cortex M-7 pro-
increase customer accessibility and likability. Consequently, cessor
it improves the quality of the user experience when compared LiDAR LDS-01
Sensors
to traditional robot designs. Intel RealSense D455
Dynamixel-XM
The Allen robot was printed in-house on an Ultimaker 3D smart servo motors:
printer in two parts. The bottom part works as an enclosure Motors Integrated ARM
for the processing units, sensors, motors, wheels, and chassis Cortex M-3 processors
and built-in encoders
of the robot as shown in Fig. 4b. Fig. 4a shows the top part
as a complete minion, with the robot’s camera mount and a
touch screen mount. The 23.98 cm x 15.86 cm x 0.85 cm efficiently. The Jetson Nano board communicates with an
screen acts as its face, and running animations, and behaves OpenCR board, which acts as a low-level controller for the
as a communication medium between the customer and the robot. It contains an integrated accelerometer and gyroscope
robot. To guarantee that the robot is eco-friendly, we chose and implements the robot’s kinematic model. This board
PLA filament, a biodegradable thermoplastic substance, to communicates directly with two Dynamixel-XM smart servo
3D print our robot. It stands 80 cm tall and weighs approx- motors. Our approach to SLAM and navigation involves the
imately 6.5 kg. The design is circular, with a diameter of 38 integration of a low-powered LDS-01 LiDAR and an Intel
cm. The bot’s stability is achieved through the installation of RealSense D455 depth camera. This combination enables
components using screws, bolts, and hot glue. Before testing, efficient navigation in the presence of large obstacles as well
rigorous quality checks were performed to ensure optimal as ground clutter. The camera also recognizes customers and
functionality and enhance overall performance. their emotions to carry out more efficient social behaviors.
The robot runs on two LiPo 3C batteries, giving it an opera-
B. HARDWARE ARCHITECTURE tional time of about 3 hours. The specifications of the Allen
For effective and efficient deployment, the Allen robot is robot are given in Table I.
equipped with cutting-edge sensors and powerful processing
systems. A Jetson Nano development board was utilized to C. SOFTWARE STACK
perform the robot’s high-level processing tasks. The board not Our social service robot is powered by a robust software stack
only has a small form factor and better power efficiency but that includes a variety of components and tools. The core
also deploys and runs AI and deep learning-based algorithms software is based on the Robot Operating System (ROS),
4 VOLUME 11, 2023
 FIGURE 5: Block diagram for the operation of SMART robot
which provides a flexible and scalable framework for building
complex robotic systems. In addition to ROS, we use a range
of software tools for perception, navigation, and control,
including OpenCV, TensorFlow, and MoveIt! These tools
enable the robot to perform a wide range of tasks, from object
recognition and tracking to path planning and manipulation.
We also leverage cloud-based services to provide natural lan-
guage processing and speech recognition capabilities, which
allow the robot to interact with visitors more intuitively and
naturally. Overall, our software stack is designed to provide
a robust and scalable platform for the development of social
service robots in a variety of environments.
1) System Architecture
The Allen robot exhibits remarkable implementation with
two innovative pipelines. The communication and interactive
pipeline enable the robot to seamlessly address customers’
requests in areas such as entertainment, general information,
and navigation. The robot utilizes a variety of sensors, includ-
ing a camera and microphone, to detect emotions and capture
customer queries, respectively. It then tailors its interaction
accordingly. The built-in display features kid-friendly rhymes
and movies for enjoyment. The robot can also have intuitive
conversations with members of the general public. Naviga-
tional queries are directed to the navigational stack, enabling
VOLUME 11, 2023
flexible navigation to manipulate the robot’s trajectory.
Furthermore, the navigational pipeline provides the robot
with remarkable maneuvering abilities, enabling it to navigate
through diverse scenarios with ease. To travel a given path,
it uses different algorithms such as localization, path plan-
ning, and motion planning within a map constructed by the
SLAM algorithm. The navigational features are affected by
the directional inquiries, so that the robot may lead customers
to a certain franchise or halt and answer when a consumer
approaches. Fig. 5 depicts the integration of the navigational
and interactive pipelines respectively, highlighting the Allen
robot’s exceptional skills and innovative position in setting a
new benchmark for customer service robots.
2) Robot Model: Differential Drive
Our robot employs the differential drive model, featuring two
independent wheels mounted on a shared axis on each side as
well as two caster wheels for design stability. By accurately
controlling the wheel velocities, the robot achieves different
types of motion, such as translation and rotation around its
instantaneous center of curvature (ICC). This enables the
robot to follow curved trajectories or rotate in place as shown
in Fig. 6. Both velocities can be varied to specify the trajectory
taken by the robot. The two velocities can be given by eq. 1
and eq. 2.
5
 of engaging interactions with the audience. Moreover, the
training of Alan AI using tailored behavioral data ensured
personalized and contextually relevant responses, enhancing
overall user satisfaction. The model effectively delivered effi-
cient and tailored conversations, while considering the emo-
tional state of users through emotion detection capabilities.
However, during our implementation of the Alan AI frame-
work, we encountered limitations in generating precise re-
sponses for more general questions. To overcome this chal-
lenge and enhance the conversational experience, we ex-
FIGURE 6: Kinematics of a differential drive robot) plored alternative solutions. Consequently, we evaluated the
integration of ChatGPT, a widely adopted large language
model known for its advanced capabilities. Although incorpo-

L
 rating ChatGPT improved conversation quality, it introduced
ω R+ = vr (1) slightly higher latency compared to Alan AI. This latency
2
increase can be attributed to ChatGPT’s larger model size
 
L and increased computational requirements. Despite this, the
ω R− = vl (2) benefits of ChatGPT outweighed the slight latency increase.
2
The integration of ChatGPT expanded the range of inter-
where L is the distance the wheels are parted, is the angular
actions that our robot, Allen, could facilitate. It empowered
velocity of the robot and R is the distance from the ICC to the
Allen to engage visitors, particularly children, in activities,
midpoint of the wheel axis. R and can be calculated by eq. 3
provide entertainment, and offer basic information such as
and eq. 4:
weather updates. These additional capabilities significantly
 
L vl + vr
 enhanced the overall user experience and made Allen a more
R= (3) valuable and versatile assistant within the mall environment.
2 vr − vl
To make the most of the strengths of both frameworks,
vr − vl we developed separate application software for Alan AI and
ω= (4)
L ChatGPT within our HRI framework. This approach allowed
us to selectively switch between the two frameworks based
on specific application requirements. When prioritizing low
latency and user-friendly conversations, we utilized the Alan
AI application. Conversely, for more general queries requir-
ing comprehensive and accurate responses, we seamlessly
switched to the ChatGPT application, leveraging its extensive
training on diverse text data. This strategic utilization of the
appropriate application software optimized the conversational
experience, enabling Allen to deliver suitable and effective
responses in various scenarios. Thus, we achieved a balance
between real-time interaction and the generation of human-
like and contextually relevant responses.
To provide a visual representation of the interaction process
within the Allen robot, we developed an interactive pipeline,
as depicted in Figure 7. This flowchart demonstrates the
seamless integration of both Alan AI and ChatGPT language
models into our HRI framework. The integration of these
language models transformed the way Allen engages with
FIGURE 7: Interaction pipeline visitors, ensuring a more human-like and effective interaction
experience.
Although integrating ChatGPT introduced a slight increase
3) Interactive System Design in latency, the advantages it offered in terms of enhanced con-
Initially, we integrated the Alan AI conversational model into versation quality, personalized responses, and expanded capa-
our multi-modal Human-Robot Interaction (HRI) framework bilities outweighed this drawback. The separate application
due to its low latency and user-friendly conversational capa- software for Alan AI and ChatGPT enabled us to effectively
bilities. Alan AI facilitated near-real-time interactions, result- harness the strengths of both frameworks. It allowed us to
ing in a seamless and responsive conversational experience. address the limitations of Alan AI by delivering more accurate
Its intuitive conversation flow design simplified the creation and engaging conversations and providing users with person-
6 VOLUME 11, 2023
alized and contextually relevant responses. Ultimately, the
integration of ChatGPT into our multi-modal HRI framework
contributed to a more immersive and satisfying experience for
mall customers.
4) Navigation system
The Allen robot’s navigation system, depicted in Fig. 8,
incorporates multiple modalities. The system utilizes the
Robot Operating System (ROS) framework and implements
a navigation pipeline. This pipeline includes a Simultaneous
Localization and Mapping (SLAM) algorithm, which enables
the robot to build an initial map of the environment where
it is deployed. By leveraging these technologies, the Allen
robot can navigate autonomously within its surroundings. We
tested our robot in a controlled environment within a lab. The
robot was able to navigate through it efficiently and avoid
obstacles successfully. We also tested the robot in a real-
world environment, and the robot was able to navigate the
environment successfully without any problems.
FIGURE 8: Navigational pipeline
a: Simultaneous Localization and Mapping (SLAM)
The deployment of the Allen robot in various environments
necessitated the creation of an accurate map. This map was
built by using Google’s online LiDAR-based SLAM tech-
nique called Cartographer. This technique employed a grid-
based approach with graph optimization as its backend. The
advantage of Cartographer SLAM lay in its ability to effi-
ciently map larger environments while being compatible with
low-powered single-board computers (SBCs). A 2D LiDAR
was used to capture depth information about the environment.
Fig. 4b shows the placement of LiDAR in an enclosure. By
creating distinct local and global maps, we were able to use
this data to create a 2D grid map. The most recent submap
(local) was compared with recent laser scans to create the final
submap. A completed submap participated in loop closure
detection utilizing newly scanned features. This enabled us
to update the world map of the environment while the error
for the pose estimation accumulated. To cope with this error
accumulation, another set of readings was taken from the
integrated IMU to estimate the pose of the robot in the en-
VOLUME 11, 2023
vironment in combination with SLAM. These readings were
used for pose optimization. Ceres-based global optimization
was carried out on the estimated map and pose information.
Continuous map updating and submap-based loop closures
contributed to the optimization process when revisiting spe-
cific locations. Fig. 9 presents the real-time implementation
of SLAM in both the testing and deployment environments.
Specifically, Fig. 9a illustrates its implementation in the test-
ing environment, while Fig. 9b showcases its application in a
mall setting. The complete maps of both controlled and mall
environments are shown in Fig. 10.
b: Navigation and Path Planning
We chose the ROS navigation stack to enable our Allen robot
to navigate through cluttered environments efficiently. This
was an obvious choice due to its scalability, robustness, and
adaptability. The ROS navigation stack is packed with effi-
cient algorithms to implement robust, versatile, and reliable
navigation. To configure the ROS navigation stack for our
robot, we first needed to select the appropriate global planner
and local planner. Dijkstra’s algorithm (named navfn in ROS)
for the global planner and the Dynamic Window Approach
(DWA) for the local planner was selected. The parameters
for each of the planners were set. Parameters for Dijkstra’s
algorithm include the following:
• Heuristic function: The heuristic function is used to esti-
mate the cost of the path from the robot’s current location
to its goal location. We used the Euclidean distance as
the heuristic function.
• Step size: The step size is the distance that the robot will
move in each planning iteration. We set the step size to
0.1 meters.
The parameters that we set for the DWA planner included
the following:
• Robot radius: The robot radius is the radius of the robot.
We set the robot radius to 0.19 meters.
• Max speed: The max speed is the maximum speed that
the robot can move. We set the max speed to 0.2 meters
per second.
• Min speed: The min speed is the minimum speed that
the robot can move. We set the min speed to 0.0 meters
per second.
We have set a small amount of goal tolerance so that the
robot can easily stop when it reaches its targeted position. We
have also set a full-length global map as well as a 5x5 local
map with a 0.05 resolution. The inflation layer is set to be
1.75.
The robot was able to navigate through areas with larger
obstacles efficiently. However, it had difficulty navigating
through areas with ground clutter, such as curbs or steps. This
was because these obstacles were below the placed height
of the 2D lidar. To address this issue, we integrated a depth
camera into the robot. The depth camera can generate a 3D
point cloud of the robot’s surroundings, including the ground
7
 (a)
(b)
FIGURE 9: Real-time SLAM implementation in (a) testing environment (b) mall.
clutter. The obtained point cloud was integrated into the ROS
navigation stack shown in Fig. 11.
We also chose the Adaptive Monte Carlo Localization
(AMCL) algorithm to localize our robot in the deployed
environment. AMCL is a probabilistic localization algorithm
that uses particle filters. The combination of these algorithms
allowed the robot to detect and avoid obstacles while navigat-
ing through the environment.
5) Adaptive Navigation Pipeline
We developed an adaptive navigation pipeline for our robot
that allows it to move randomly while not in use. To establish
this communication between the navigational and interactive
pipelines, we have developed APIs using the Flask frame-
work. The robot initially checks its status to see if it is
currently responding to a query. If not, it generates a random
location within the given map and navigates to that location.
As soon as the robot receives a directional query, it stops
its random motion and selects the static location from the
tagged list of positions on the map. A path is then generated,
and the robot assists the customer in reaching their desired
destination. Once the guidance pipeline is complete, the robot
resumes generating random locations and driving randomly.
D. SERVICES PROVIDED: AN OVERVIEW
The Allen robot is a versatile and capable mobile robot de-
signed to provide directional assistance and customer engage-
ment in large environments. It is designed to provide a variety
of services in a shopping mall. The robot is equipped with a
variety of sensors and actuators, including a camera, a mi-
8 VOLUME 11, 2023
crophone, and a speaker. Allen also has a variety of software
capabilities, including the ability to navigate autonomously,
recognize emotions, and generate speech.
The robot has a prebuilt map of the mall that it uses to
direct customers to their desired destinations. It is designed
to appeal to both adults and children, with its child-friendly
minion-shaped design. The robot can interact with customers
based on their mood and age. For example, it can talk to adult
customers, provide them with weather information, entertain
them with jokes, and answer their directional as well as
informational queries. It can also engage children by telling
them jokes, reciting poems, showing them movies, or playing
games with them. The combination of the robot’s autonomous
navigation, customer interaction, and directional assistance
capabilities makes it a valuable asset for any shopping mall.
TABLE II: Customer’s reaction to the robot
Common (Percentage) Surprised (Percentage) Disappointed (Percentage)
25 63 12
TABLE III: Customer’s behavior towards the robot
A B C D
38% 25% 27% 10%
III. EXPERIMENTATION AND FIELD TRIALS
The Allen robot was designed for rapid and efficient de-
ployment, owing to its in-house manufacturing process. The
robot’s modular construction facilitated easy replacement
and upgrading of its components, while its adaptable design
catered to the specific needs of each customer.

(a)
(b)
FIGURE 10: (a) Map of the controlled environment used
for testing the robot. (b) A 2D map of the mall’s children’s
playground area used for the deployment of our autonomous
robot.
The performance assessment of the Allen robot was con-
ducted in two distinct phases. The initial phase involved test-
ing the robot in a controlled environment. During this phase,
the Allen robot exhibited efficient communication capabili-
ties with the study participants and demonstrated high accu-
racy in navigating the environment. However, a limitation was
observed regarding the robot’s ability to detect ground clutter,
which had a negative impact on its navigation efficiency. To
overcome this limitation, a depth camera was integrated into
the robot’s navigation system, allowing for the incorporation
of a 3D point cloud of the environment. This enhancement
significantly improved the robot’s ability to accurately detect
potential obstacles in its path.
In the second phase, the robot was deployed in a shopping
mall, where it successfully navigated and guided customers to
their desired destinations. The robot engaged in conversations
with customers of various ages and interests, providing enter-
tainment through games, poems, and stories. Fig. 12 show-
cases the vibrant interaction between Allen and customers.
Additionally, it offered information and conversed with cus-
tomers using its ChatGPT interface, showcasing its versatility
in addressing the dynamic needs of different customers.
Overall, the test results demonstrate Allen’s potential to
provide a variety of services in a shopping mall setting. The
robot has the capability to enhance customer service, boost
VOLUME 11, 2023
(a)
(b)
FIGURE 11: Visualization of the robot’s navigation with the
2D map integrated with the 3D point cloud
sales, and create a more enjoyable shopping experience.
A. CUSTOMER REVIEWS
Customers’ feedback on the Allen robot was analyzed based
on their comments. Their feelings about the robot were di-
vided into three categories: common, disappointed, and sur-
prised. As shown in Table II, most customers were surprised
by the robot. Notably, approximately 12% of customers were
disappointed with the robot. These customers gave various
reasons, such as the robot not meeting their expectations of
an intelligent robot like in science fiction movies. They also
believed that the robot would not provide any positive impacts
on the market.
Table III shows customers’ behavior regarding the func-
tionality of the robot. Options A, B, C, and D in Table III
represent the options that the robot is now useful but needs
improvements, the robot is now useful and good enough, the
robot is now useless but promising, and the robot is useless
and impractical.
In general, customers who interacted with the Allen robot
were generally positive about the robot. They appreciated the
robot’s ability to provide information and assistance, as well
as its ability to entertain them. Overall, its testing has been
successful. It shows room for improvement but the potential
to provide various services in a shopping mall. The Allen
robot is a versatile and promising new development that has
the potential to revolutionize the retail industry.
9
 FIGURE 12: Interaction of customers with the Allen robot in the mall
IV. CONCLUSION
This paper has presented the deployment of the Allen robot
in shopping malls as a means of revolutionizing the retail
industry by providing efficient and versatile customer service.
The robot’s modular design and in-house manufacturing pro-
cess enabled easy customization to meet specific customer
requirements. The robot’s adaptive navigation pipeline and
multimodal communication capabilities have allowed it to
navigate through cluttered environments, engage with cus-
tomers of various ages, and provide information, entertain-
ment, and directional assistance. Additionally, the integration
of ChatGPT into the Allen robot further enhanced its conver-
sational capabilities and contributed to its success in shopping
malls. By switching from the Alan AI conversational model
to ChatGPT, the robot’s ability to generate real-time and
efficient responses was significantly improved. The inclusion
of ChatGPT allowed the robot to engage in more human-
like and effective conversations, addressing a wider range of
inquiries and providing accurate and detailed information to
customers.
The successful performance of the Allen robot in con-
trolled and real-world environments has demonstrated its
potential for application in the retail industry. The customer
feedback analysis predominantly indicates satisfaction with
the robot’s capabilities, with only a small percentage express-
ing disappointment or concern. These concerns have also
provided valuable insights into areas for improvement, partic-
ularly in meeting higher expectations of an intelligent robot,
which will be crucial for advancing the robot’s capabilities.
Further research and development in the field of service
robots will contribute to improving their capabilities and
exploring new opportunities for integration across various
sectors. The adoption of intelligent robots like Allen holds
promise for enhancing customer service, and differentiation,
and meeting evolving customer needs in the dynamic retail
landscape.
ACKNOWLEDGMENT
This research is conducted at Control Automotive and
Robotics Lab (CARL-BUITEMS), funded by National Cen-
ter of Robotics and Automation (NCRA) with the collabora-
tion of the Higher Education Commission (HEC) of Pakistan.
10
REFERENCES
[1] C. Clifford, ‘‘COVID-19 pandemic proves the need for ’social robots,’
’robot avatars’ and more, say experts,’’ Apr. 2020.
[2] Y. Shen, D. Guo, F. Long, L. A. Mateos, H. Ding, Z. Xiu, R. B. Hellman,
A. King, S. Chen, C. Zhang, and H. Tan, ‘‘Robots Under COVID-19
Pandemic: A Comprehensive Survey,’’ IEEE Access, vol. 9, pp. 1590–
1615, 2021.
[3] T. Hornyak, ‘‘Entertainment robots the latest craze worldwide as the pan-
demic rages on,’’ Jan. 2021.
[4] ‘‘Service Robotics Market Industry Report Size, Share, Trends Forecast,
2028.’’
[5] S. Schneider, W. Clas, and D. Brščić, ‘‘Human-Robot Interaction in Public
Spaces,’’ in 2022 17th ACM/IEEE International Conference on Human-
Robot Interaction (HRI), pp. 1287–1289, Mar. 2022.
[6] M. Sarrica, S. Brondi, and L. Fortunati, ‘‘How many facets does a “social
robot” have? A review of scientific and popular definitions online,’’ In-
formation Technology & People, vol. 33, pp. 1–21, Jan. 2019. Publisher:
Emerald Publishing Limited.
[7] M. Blaurock, M. Čaić, M. Okan, and A. P. Henkel, ‘‘A transdisci-
plinary review and framework of consumer interactions with embod-
ied social robots: Design, delegate, and deploy,’’ International Journal
of Consumer Studies, vol. 46, no. 5, pp. 1877–1899, 2022. _eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1111/ijcs.12808.
[8] A. Milman and A. D. Tasci, ‘‘Consumer reactions to different robotic
servers in theme parks,’’ Journal of Hospitality and Tourism Technology,
vol. 13, pp. 314–332, Jan. 2022. Publisher: Emerald Publishing Limited.
[9] D. Golchinfar, D. D. Vaziri, G. Stevens, and D. Schreiber, ‘‘Let’s Go to the
Mall: Investigating the Role of User Experience in Customers’ Intention to
Use Social Robots in a Shopping Mall,’’ in Designing Interactive Systems
Conference, DIS ’22, (New York, NY, USA), pp. 377–386, Association for
Computing Machinery, June 2022.
[10] ‘‘Sophia.’’
[11] ‘‘Honda Global | ASIMO.’’
[12] ‘‘PARO Therapeutic Robot.’’
[13] D. Roy, ‘‘Public Awareness Robot Sylvester,’’ International Journal for
Research in Applied Science and Engineering Technology, vol. 10, pp. 5–
15, 8 2022.
[14] R. Matsumura and M. Shiomi, ‘‘An Animation Character Robot That
Increases Sales,’’ Applied Sciences, vol. 12, p. 1724, Jan. 2022. Number:
3 Publisher: Multidisciplinary Digital Publishing Institute.
[15] ‘‘Meet Geminoid F, a Smiling Female Android - IEEE Spectrum.’’
[16] Z. Chen, Y. Nakamura, and H. Ishiguro, ‘‘Android as a Receptionist in a
Shopping Mall Using Inverse Reinforcement Learning,’’ IEEE Robotics
and Automation Letters, vol. 7, pp. 7091–7098, July 2022. Conference
Name: IEEE Robotics and Automation Letters.
[17] D. Brscic, T. Ikeda, and T. Kanda, ‘‘Do You Need Help? A Robot Providing
Information to People Who Behave Atypically,’’ IEEE Transactions on
Robotics, vol. 33, pp. 500–506, Apr. 2017.
[18] D. F. Glas, K. Wada, M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita,
‘‘Personal Greetings: Personalizing Robot Utterances Based on Novelty
of Observed Behavior,’’ International Journal of Social Robotics, vol. 9,
pp. 181–198, Apr. 2017.
[19] T. Kanda, M. Shiomi, Z. Miyashita, H. Ishiguro, and N. Hagita, ‘‘A Com-
munication Robot in a Shopping Mall,’’ IEEE Transactions on Robotics,
vol. 26, pp. 897–913, Oct. 2010.
VOLUME 11, 2023
[20] C. Shi, S. Satake, T. Kanda, and H. Ishiguro, ‘‘A Robot that Distributes
Flyers to Pedestrians in a Shopping Mall,’’ International Journal of Social
Robotics, vol. 10, pp. 421–437, Sept. 2018.
[21] M. Shiomi, K. Shinozawa, Y. Nakagawa, T. Miyashita, T. Sakamoto, T. Ter-
akubo, H. Ishiguro, and N. Hagita, ‘‘Recommendation Effects of a Social
Robot for Advertisement-Use Context in a Shopping Mall,’’ International
Journal of Social Robotics, vol. 5, pp. 251–262, Apr. 2013.
[22] S. R. A. Inc, ‘‘Meet Pepper: The Robot Built for People | SoftBank Robotics
America.’’
[23] O. Mubin, I. Kharub, and A. Khan, ‘‘Pepper in the Library" Students’
First Impressions,’’ in Extended Abstracts of the 2020 CHI Conference on
Human Factors in Computing Systems, (Honolulu HI USA), pp. 1–9, ACM,
Apr. 2020.
[24] R. Stock and M. Merkle, ‘‘Can humanoid service robots perform better than
service employees? a comparison of innovative behavior cues,’’ in Hawaii
International Conference on System Sciences, 1 2018.
[25] S. Thunberg and T. Ziemke, ‘‘Are People Ready for Social Robots in Public
Spaces?,’’ in Companion of the 2020 ACM/IEEE International Conference
on Human-Robot Interaction, (Cambridge United Kingdom), pp. 482–484,
ACM, Mar. 2020.
[26] L.-D. Daczo, L. Kalova, K. L. F. Bonita, M. D. Lopez, and M. Rehm,
‘‘Interaction Initiation with a Museum Guide Robot—From the Lab into
the Field,’’ in Human-Computer Interaction – INTERACT 2021 (C. Ardito,
R. Lanzilotti, A. Malizia, H. Petrie, A. Piccinno, G. Desolda, and K. Inkpen,
eds.), vol. 12934, pp. 438–447, Cham: Springer International Publishing,
2021. Series Title: Lecture Notes in Computer Science.
[27] M. E. Foster, R. Alami, O. Gestranius, O. Lemon, M. Niemelä, J.-M.
Odobez, and A. K. Pandey, ‘‘The MuMMER Project: Engaging Human-
Robot Interaction in Real-World Public Spaces,’’ in Social Robotics
(A. Agah, J.-J. Cabibihan, A. M. Howard, M. A. Salichs, and H. He, eds.),
vol. 9979, pp. 753–763, Cham: Springer International Publishing, 2016.
Series Title: Lecture Notes in Computer Science.
[28] W. Hess, D. Kohler, H. Rapp, and D. Andor, ‘‘Real-Time Loop Closure in
2D LIDAR SLAM,’’ in 2016 IEEE International Conference on Robotics
and Automation (ICRA), pp. 1271–1278, 2016.
[29] D. Fox, ‘‘Kld-sampling: Adaptive particle filters and mobile robot local-
ization,’’ in NIPS 2001, 2001.
[30] D. Fox, W. Burgard, and S. Thrun, ‘‘The Dynamic Window Approach to
Collision Avoidance,’’ Robotics & Automation Magazine, IEEE, vol. 4,
pp. 23–33, Apr. 1997.
[31] H. Wang, Y. Yu, and Q. Yuan, ‘‘Application of Dijkstra algorithm in robot
path-planning,’’ in 2011 Second International Conference on Mechanic
Automation and Control Engineering, pp. 1067–1069, July 2011.
HAMZA ANWAR received the B.S. degree in
Electronics Engineering from the Balochistan Uni-
versity of IT, Engineering & Management sciences
(BUITEMS), Quetta, Pakistan, in 2015.
He worked in Pakistan Telecommunication
Company Limited (PTCL) from 2016 to 2018 as
an Assistant Manager. He has been associated with
multiple local educational institutions and is cur-
rently working in the Control, Automotive and
Robotics Lab (CARL) under the National Centre
for Control and Automation (NCRA) as a research associate. His research
interests include Artificial Intelligence, Neural networks, Cognitive neural
networks, and Robotic systems design.
VOLUME 11, 2023
SECOND B. AUTHOR (M’76–SM’81–F’87) and
all authors may include biographies. Biographies
are often not included in conference-related pa-
pers. This author became a Member (M) of IEEE
in 1976, a Senior Member (SM) in 1981, and a
Fellow (F) in 1987. The first paragraph may con-
tain a place and/or date of birth (list place, then
date). Next, the author’s educational background
is listed. The degrees should be listed with type of
degree in what field, which institution, city, state,
and country, and year the degree was earned. The author’s major field of
study should be lower-cased.
The second paragraph uses the pronoun of the person (he or she) and
not the author’s last name. It lists military and work experience, including
summer and fellowship jobs. Job titles are capitalized. The current job must
have a location; previous positions may be listed without one. Information
concerning previous publications may be included. Try not to list more than
three books or published articles. The format for listing publishers of a book
within the biography is the title of the book (publisher name, year) similar to
a reference. Current and previous research interests end the paragraph.
The third paragraph begins with the author’s title and last name (e.g.,
Dr. Smith, Prof. Jones, Mr. Kajor, Ms. Hunter). List any memberships in
professional societies other than the IEEE. Finally, list any awards and work
for IEEE committees and publications. If a photograph is provided, it should
be of good quality and professional-looking. Following are two examples of
an author’s biography.
THIRD C. AUTHOR, JR. (M’87) received the B.S.
degree in mechanical engineering from National
Chung Cheng University, Chiayi, Taiwan, in 2004
and the M.S. degree in mechanical engineering
from National Tsing Hua University, Hsinchu, Tai-
wan, in 2006. He is currently pursuing the Ph.D.
degree in mechanical engineering at Texas A&M
University, College Station, TX, USA.
From 2008 to 2009, he was a Research Assistant
with the Institute of Physics, Academia Sinica,
Tapei, Taiwan. His research interest includes the development of surface
processing and biological/medical treatment techniques using nonthermal
atmospheric pressure plasmas, fundamental study of plasma sources, and
fabrication of micro- or nanostructured surfaces.
Mr. Author’s awards and honors include the Frew Fellowship (Australian
Academy of Science), the I. I. Rabi Prize (APS), the European Frequency and
Time Forum Award, the Carl Zeiss Research Award, the William F. Meggers
Award and the Adolph Lomb Medal (OSA).
11