You are on page 1of 13

Biologically Inspired Cognitive Architectures (2013) 6, 96– 108

Available at www.sciencedirect.com

ScienceDirect

journal homepage: www.elsevier.com/locate/bica

RESEARCH ARTICLE

Development of a self sustaining


cognitive architecture
Anjan Kumar Ray *, Gang Leng, T.M. McGinnity, Sonya Coleman,
Liam Maguire

Intelligent Systems Research Centre (ISRC), Faculty of Computing and Engineering, University of Ulster, Magee Campus,
Londonderry BT48 7JL, UK

Received 19 May 2013; received in revised form 13 July 2013; accepted 14 July 2013

KEYWORDS Abstract
Self-organising system; A context-aware cognitive system is a prime requirement for a sensor rich smart home environ-
Fuzzy logic;
ment. In this paper, we discuss the development and evaluation of a self-sustaining cognitive
Neural network;
architecture for the RUBICON (Robotic UBIquitous COgnitive Network) system which builds its
Reasoning model;
Cognitive architecture knowledge as per the environmental situations. The proposed cognitive architecture consists
of a reasoning module, a decision module, and a supporting memory module. An online slid-
ing-window based self-organising fuzzy neural network (SOFNN), which explores relationships
between the event inputs and desired reasoning outputs, is developed for the reasoning mod-
ule. We also propose a prediction model based on event information to support the reasoning
module for continuous training in the absence of external training data. The decision module
generates control goals for the robots according to the status outputs from the reasoning mod-
ule. We develop a MySQL based database for the memory module which supports the overall
system by storing processed information about the states of the environment and providing his-
torical information for enhanced understanding. The architecture is trained and tested with
environmentally realistic synthesized data to show its adaptation capabilities. The results dem-
onstrate that the proposed system can learn activities and track them within a smart home
environment. This initial implementation also highlights the potential of the architecture
and will serve as a very important test-bed for future work. We envisage that the proposed
combination of the prediction model and the reasoning module will eventually result in a gen-
eral purpose, self-sustaining, self-organising cognitive architecture for different applications
and thus the proposed architecture enters into the sphere of the biologically inspired cognitive
architecture (BICA) challenge.
ª 2013 Elsevier B.V. All rights reserved.

* Corresponding author. Tel.: +44 2871675499.


E-mail addresses: ak.ray@ulster.ac.uk (A.K. Ray), g.leng@ulster.ac.uk (G. Leng), tm.mcginnity@ulster.ac.uk (T.M. McGinnity),
sa.coleman@ulster.ac.uk (S. Coleman), lp.maguire@ulster.ac.uk (L. Maguire).

2212-683X/$ - see front matter ª 2013 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.bica.2013.07.006
Development of a self sustaining cognitive architecture 97

Introduction for real-time pattern recognition such as visual scene under-


standing and invariant object recognition. ACT-R (Adaptive
Components of Thought-Rational) is a hybrid cognitive
Cognitive architectures are frequently used to model human architecture. It aims to model human behaviour (ACT-R.,
behaviour and reproduce aspects of such behaviour, in arti- 2013; Anderson et al., 2004). ACT-R has three main compo-
ficial systems. There are a number of cognitive architec- nents: modules, buffers and pattern matcher. There are two
tures in the literature (Duch, Oentaryo, & Pasquier, 2008; types of modules named perceptual-motor modules and
Langley, Laird, & Rogers, 2009; Vernon, Metta, & Sandini, cognitive memory modules (Byrne, 2001). Perceptual-motor
2007). SOAR (State, Operator and Result) is a rule-based modules create interfaces with the real world. There are
cognitive architecture designed to model general knowl- two types of memory modules named declarative memory
edge (Laird, 2008, 2012). The learning mechanism in SOAR consisting of facts and procedural memory which represents
is an explanation-based learning (EBL) technique to formu- knowledge about different procedures. A production system
late rules. SOAR has been used, for example, to develop coordinates the action of the modules by matching the
computer games, human language processing and categori- information stored in several buffers, where each module
zation (Laird, 2008; Lewis, 1993; Magerko, Laird, Assanie, places a limited amount of information. The pattern match-
Kerfoot, & Stokes, 2004; Tambe et al., 1995). The cognitive er searches for a production that matches the current state
architecture ICARUS, designed for physical agents (Langley of the buffers. The ACT-R architecture is used in different
& Choi, 2006), relies on symbolic manipulation of knowl- application areas including different aspects of memory,
edge. The architecture includes modules for perception, perception, attention, reasoning and decision making, prob-
planning and execution where concepts are processed in a lem solving, and language processing (Peebles & Banks,
bottom-up way to match perception and goals that are pro- 2010; Taatgen, 2005; Trafton et al., 2005). Kismet, as part
cessed in a top-down way to match skills. In Prodigy (Carbo- of an articulated robotic head (Langley, Cummings, & Shap-
nell & Gil, 1990; Carbonell et al., 1991; Haigh & Veloso, iro, 2004), has five distinct modules in its cognitive architec-
1996), the learning and reasoning modules produce mutually ture; a perceptual system, an emotion system, a behaviour
interpretable knowledge structures. Its general-purpose system, a drive system, and a motor system. It tries to
problem solver searches through a set of control rules that understand the dynamics of social interaction by cues like
are based on ‘select’, ‘reject’, and ‘prefer’ to accomplish gaze direction, facial expression and vocal babbling. CLAR-
a set of goals. NOMAD, also known as Darwin automata ION is also a hybrid architecture that stores both action-cen-
(Edelman, 1993), uses a reward mechanism that drives tred and non-action centred knowledge in implicit form
learning in a large-scale neural network. It is mainly used using multi-layer neural networks and in explicit form using

Fig. 1 The Cognitive Layer within the high-level overall RUBICON architecture.
98 A.K. Ray et al.

Reasoning
target table

SOFNN based Status outputs


Event Window
g
reasoning for further
information control
module actions
Cognitive reasoning module

Fig. 2 Overview of the cognitive reasoning module.

symbolic production rules (Sun, 2007; Sun, 2009). It cap- handling symbolic entities from low level information and
tures the interactions between the explicit and implicit context-specific sub-symbolic structure to achieve higher
knowledge. There are a number of cognitive architectures, level cognition. Contrary to the ACT-R, the RUBICON system
which draw more specific motivations from the biological demands multiple rules to be activated at a time while pro-
brain. Franklin and Patterson (Franklin & Patterson, 2006) cessing a multitude of information. We had shown that a
proposed LIDA cognitive architecture to model human cog- self-organising fuzzy neural network (SOFNN) can be used
nition. In Novamente AI Engine (Looks, Goertzel, & Penna- for cognitive reasoning within a smart home environment
chin, 2004), self-organising and goal-oriented interactions (Ray, Leng, McGinnity, Coleman, & Maguire, 2012). In this
between patterns are responsible for mental states. The paper, the proposed cognitive architecture for the RUBICON
relational pattern organisation emerges from the dynamics system is designed to support online reasoning and decision
of network activations. The 4CAPS architecture (Just & Var- making with real-time multiple control goal generations and
ma, 2007) has a biologically plausible neural architecture self-organisation of its structure based on dataflow. A pre-
and assumes that cognition arises from the concurrent acti- diction feature with memory support is incorporated to
vation of multiple nodes in a collaborative manner. The the reasoning system to provide target data for its online
AMBR cognitive system (Kostadinov & Grinberg, 2008) at- adaptation so as to provide a self-sustaining capability.
tempts to implement a model of human reasoning based We develop, implement, and test the self-adaptive, on-line
on mind–body-environment metaphor. Tahboub and Al-Din cognitive architecture based on a SOFNN. These develop-
(2009) have presented a neuro-fuzzy reasoning system for ments make the system suitable for real-time application.
mobile robotics, exploiting the synergy between neural sys- This paper describes the first stage of the work.
tem learning capabilities in parallel with human-like fuzzy The remainder of the paper is organised as follows: the
reasoning. Gupta et al. considered computational percep- next section presents an overview of the RUBICON system.
tion and cognition under uncertainty (Gupta & Gupta, Then we describe the developed cognitive architecture for
2000) and used fuzzy systems to model this. Erlhagen (Erlha- the system. Implementation results are shown in the exper-
gen & Bicho, 2006) proposed a dynamic neural field ap- iments and results section. Finally we present the overall
proach to cognitive robotics where each field represents conclusion of this work.
the basic functionality of neuronal populations in different
brain areas. Although different architectures exhibit pro- An overview of the RUBICON system
gress within their designed domains, the proper solution
to the BICA challenge is still to come (Samsonovich, 2012).
The RUBICON system consists of four functional layers: a
The aim of the RUBICON system (EU FP7 RUBICON pro-
Learning Layer, a Control Layer, a Cognitive Layer and a
ject, 2011) is to build a self-sustaining, self-organising,
Communication Layer. Interconnection of these layers is
and goal-oriented robotic ecology. The architecture which
shown in Fig. 1. The Learning Layer processes raw sensory
is most appropriate for the RUBICON system is a hybrid cog-
information and provides meaningful events occurring with-
nitive architecture because it has the potential benefit of
in the environment. It relates the data to possible events
with classification confidences (e[0, 1]). A higher value of
confidence for an event confirms occurrence of that event
Status output 1 and vice versa. The Cognitive Layer builds up knowledge
SO
SOFNN 1 and understanding of the RUBICON system based on reason-
Input vector Status output 2 ing over these events learnt. It facilitates the Control Layer
SOFNN 2 with the decisions on ‘what to do’ under different environ-
mental scenarios. The Control Layer provides high level con-
trol using a group of available robotic devices by
Status
t output
t tm formulating and executing both action and configuration
SOFNN m
strategies to satisfy the objectives set for the RUBICON
system. These layers interact and cooperate through the
Fig. 3 The MIMO structure of the SOFNN based reasoning underlying communication channels provided by the
module. Communication Layer.
Development of a self sustaining cognitive architecture 99

Cognitive architecture for the RUBICON system soidal basis functions (EBF) representing the if-part (or pre-
mise) of fuzzy rules. The outputs from layer 2 are
normalised in layer 3. Layer 4 provides the weighted bias
In this paper, we focus on the architecture and development
for the network and layer 5 produces the overall output
within the Cognitive Layer. Reasoning (over current and
from the network. The network has the ability to self-orga-
prior events), decision making and memory are the three
nise its own structure during the learning process. We define
key properties of the proposed cognitive architecture. So,
Xt = [x1tx2t. . .xrt] as the input vector for the t-th observation
the proposed Cognitive Layer consists of three high level
(Xt, dt) where r is the number of inputs, dt is the desired
interconnected modules. They are a reasoning module, a
output (target) at time t and yt is the actual output of the
decision module and a memory module. A prediction model
current network. The actual output is obtained as
is in place to support self-sustaining continuous operation of  
the reasoning module. A tight coupling between events and Pu Pr ðxi cij Þ2
j¼1 w 2j exp  i¼1 2r 2
goals is a requirement of the RUBICON system. Furthermore ij
yðXÞ ¼ P h P i ð1Þ
it is anticipated that the portfolio of events and portfolio of u
exp  r ðxi cik Þ2
k¼1 i¼1 2r2
possible goals are dynamic, requiring a self-organising ik

system. Here, u is the number of neurons, cij and rij are the cen-
tre and width of the i-th membership function in the j-th
Cognitive reasoning module neuron. The row vector Aj ¼ ½ aj0 aj1 aj2    ajr 

Fig. 2 shows an overview of the cognitive reasoning module.


The cognitive reasoning module analyses the event related W= width of SW
data received within a specified time window, to create a
current view of the RUBICON system. The window control
block is responsible for the arrangement of appropriate
data. As a requirement of the RUBICON system, the cogni- Time
tive reasoning module processes information for a number
of reasoning targets. Descriptions of these targets along t-W+1 t
with ids are maintained within the reasoning target table Moving
i Direction
(part of the memory). We developed the cognitive reasoning
capability of the system based on a SOFNN. As the system Fig. 5 Sliding window of data for cognitive reasoning.
utilises multiple inputs for multiple reasoning targets, we
developed a multi-input–multi-output (MIMO) structured
SOFNN for the reasoning module as shown in Fig. 3 where Predicted output 1
each output refers to the status of related reasoning target. Currentt eventt
C SOFNN 1
These outputs are presented to the decision module as sta- inputs
Predicted output 2
tus outputs, reflecting the current understanding of the SOFNN 2
RUBICON system. Historical event
The self-organising fuzzy neural network (SOFNN), imple- inputs
menting Takagi–Sugeno (TS) fuzzy models (Takagi & Predicted output m
SOFNN m
Sugeno, 1985) and previously developed by the authors, is
a five-layer fuzzy neural network as shown in Fig. 4. Layer
1 is the input layer. The neurons in layer 2 implement ellip- Fig. 6 An overview of a MIMO SOFNN based prediction model.

R1 A1
Ω N Π

R2 A2
x1
Ω N Π

R3 Σ y
x2 A3
Ω N Π

Ru Au
xr
Ω N Π

I t layer
Input l l
EBF layer Normalised
N li d llayer Weighted layer Output layer

Fig. 4 Architecture of a 5-layer SOFNN.


100 A.K. Ray et al.

represents the set of parameters corresponding to the neu- where W2 is the parameter matrix, wjn is the output of the j-
ron j and w2j is the weighted bias which is defined for the TS th neuron in the normalised layer for the n-th training pat-
model as tern. The learning process of the SOFNN is divided into
T structure learning and parameter learning. The structure
w 2j ¼ Aj  ½ 1 x1 x2 ... xr  ¼ aj0 þ aj1 x1 þ . . . þ ajr xr 8j ¼ 1; 2; . . . ; u
learning adds new EBF neurons or deletes unimportant EBF
ð2Þ
neurons as per the data (Ray et al., 2012). The recursive
For the learning purposes, the output of the network for parameter matrix learning algorithm is as follows
n observations can be described in matrix form as 1
LðtÞ ¼ Q ðtÞpðtÞ ¼ Q ðt  1ÞpðtÞ½1 þ pT ðtÞQ ðt  1ÞpðtÞ ð7Þ
Y ¼ W 2W ð3Þ
Q ðtÞ ¼ ½I  aLðtÞpT ðtÞQ ðt  1Þ ð8Þ
Y ¼ ½ y1 y2  yn  ð4Þ
^ ^ ^
W 2 ¼ ½ a10 a11  a1r  au0 au1  aur  ð5Þ HðtÞ ¼ Hðt  1Þ þ aLðtÞ½d t  pT ðtÞ Hðt  1Þ ð9Þ

2 3 
w11  w1n 1; jeðtÞj P jeðtÞj
a¼ ð10Þ
6w x  w1n x 1n 7 0; jeðtÞj < jeðtÞj
6 11 11 7
6 7
6 .. .. .. 7 where Q(t) = [PT(t)P(t)]1 is an MxM Hermitian matrix (Q-
6 . . . 7
6 7 matrix), P(t) = WT = [pT(1)pT(2). . .pT(t)]T, M = u · (r + 1),
6w x w1n x rn 7
6 11 r1  7 HðtÞ ¼ WT2 ¼ ½ h1 h2    ^hM T ,
6 7
6 .. 7 eðtÞ ¼ d t  y t ¼ d t  p^T ðtÞ HðtÞ is the approximation error
W ¼ 6 ... ..
. . 7 ð6Þ
6 7 and eðtÞ ¼ d t  pT ðtÞ Hðt  1Þ is the estimation error. Fur-
6 7
6 wu1  wun 7 ther details can be found in (Leng, McGinnity, & Prasad,
6 7
6 w x11  wun x 1n 7 2005). The reasoning outputs depend on the current event
6 u1 7
6 . .. .. 7 information as well as historical information of events.
6 . 7
4 . . . 5 Therefore we employ a first-in-first-out (FIFO) sliding win-
wu1 x r1  wun x rn dow (SW) to handle this requirement. The sliding window

Status
Targets
Inputs SOFNN based t t
outputs - +
FIFO slidingg Prediction
reasoning
window model
module

Error signals

Fig. 7 Integration of the prediction model with the reasoning module for self-sustaining operation.

Fig. 8 Overall cognitive architecture for the RUBICON system.


Development of a self sustaining cognitive architecture 101

is shown in Fig. 5. Each sample data within the window rep- information for continuous training. This is described as
resents event information and desired reasoning target follow

Fig. 9 An overview of a sensor rich smart home environment.

Fig. 10 Visualisation of the cognitive process: a sample instance.


102 A.K. Ray et al.

Datat ¼ ½X t d t T ð11Þ ^ tþ1 ¼ yðX t ; X t1 Þ


d ð12Þ
The prediction model is trained offline and when the
Prediction model training is completed the model is fixed and deployed on-
line. Further changes to the model can be accommodated
The introduction of a self-sustaining capability demands the by further offline training and any updates to the model
provision of a continuous training signal to ensure the self-
learning and maintenance of adaptive capabilities so that
reasoning and cognition continue automatically. This capa- Table 1 RMSE and the network structure for the prediction
bility is realised through the introduction of a prediction model.
system to generate training data for the SOFNN and thus
self-sustain its operation. To enable this, the reasoning Prediction outputs RMSE Number of neurons
module requires real-time target data dt for each sampling User exercise 0.0752 8
period. To generate such target data, we propose a predic- User relaxing 0.0377 31
tion model based on current and historical event informa- User in kitchen 0.0717 3
tion. In this work, we exploit the SOFNN for its prediction Bring phone 0.0900 2
capability. As the RUBICON cognitive layer addresses multi- Open door situation 0.0748 2
ple reasoning outputs, we also consider a multiple inputs User cook activity 0.0881 2
multiple outputs (MIMO) structure for the prediction model Fire alert 0.0589 2
as shown in Fig. 6. This module provides predicted state for Burglary alert 0.0361 4
each reasoning sub-module for real-time online operation. Dripping alert 0.0377 27
For this current version of work, we use one-step-ahead pre- Cleaning situation 0.0489 21
diction as follows

Fig. 11 Visualisation of the cognitive process: another sample instance.


Development of a self sustaining cognitive architecture 103

can then replace the existing one as required. The predic- sion module publishes goals with expected finishing times
tion model differs from the reasoning module as it receives and refrains from publishing similar goals for the Control
both current and historical event inputs. The model is de- Layer over that period. In essence, the cognitive decision
signed to provide step-ahead predictions of the desired module assesses the proposed status of the ecology as pos-
states of the ecology. These predicted states are then used tulated by the cognitive reasoning module.
for real-time, online operation of the SOFNN by providing
the required targets so that it can adjust its internal struc- Memory module
ture as required when it reasons across the ecology. The
strategy for online operation is shown in Fig. 7. The provi- A cognitive system builds knowledge through reasoning, and
sion of these target data by the prediction model introduces planning for its actions, and memorising its decisions on var-
the self-sustaining capability as the target is now self-gener- ious events. So, memory is important to maintain and re-
ated by the current and historical event data. In this case, trieve information relevant to ongoing cognitive
the predicted outputs are used to form the sliding window processing. In the RUBICON cognitive layer, memory support
data for continuous learning. The SOFNN network is updated is essential for self-sustaining online operation and reason-
as per the impending error signals. ing over historical data. The memory holds knowledge about
the world as perceived by the Learning Layer, as explored by
Cognitive decision module the Control Layer and as informed by the cognitive reason-
ing and decision modules. We have developed a MySQL data-
The cognitive decision module is responsible for assigning
suitable control goals for the Control Layer. The goals de-
scribe the status of the world as desired by the RUBICON sys-
tem. The Control Layer explores services of available robots Table 2 Neuronal structure for each reasoning sub-
to carry out the actions expected from the cognitive deci- module.
sions (e.g., open door for a visitor). From a high level per- Reasoning outputs Number of Neurons
spective, the Cognitive Layer advises ‘what to do’ and the
Reasoning module Reasoning
Control Layer has the freedom to choose ‘how’ to achieve
without prediction Module with
it, within its known constraints. As the reasoning module re-
model Prediction Model
ceives continuous information on events from the Learning
Layer, it also generates continuous status outputs for the User exercise 1 3
decision module. Any action performed by a robot requires User relaxing 4 6
some time to complete. Accordingly, the decision module User in kitchen 1 2
posts a goal for a desired action and refrains from posting Bring phone 1 2
the same goal over a specified period of time to enable Open door situation 1 2
the Control Layer to attend the task. Moreover, multiple User cook activity 2 2
goals may be required to be posted at the same time as Fire alert 2 2
per the environmental situations. For example, a visitor Burglary alert 1 2
has arrived at the door and rings doorbell. At the same time, Dripping alert 1 1
a mobile phone also rings. The control goals are to attend Cleaning situation 3 4
the door and bring the phone to the user. The environment Total number of 17 26
situations (doorbell ringing and phone ringing) will continue neurons
for a while as seen by the reasoning module. But, the deci-

30

25
Number of neurons

20

15

10

Reasoning module without prediction model


Reasoning module with prediction model
5

0
500 1000 1500 2000 2500 3000 3500 4000 4500
Number of samples

Fig. 12 A comparison of the self-organisation nature of the reasoning module with/without the prediction model.
104 A.K. Ray et al.

base for this purpose. The memory module interacts with goal id 1. At this stage of the development, we do not deal
the other modules as shown in Fig. 8. It stores event infor- with control feedback. The Cognitive Layer is interfaced
mation from the Learning Layer, reasoning status outputs with the Learning and Control Layers through the Physically
from the reasoning module, predicted targets for the rea- Embedded Intelligent System (PEIS) tuple space (Broxvall,
soning module from the prediction model, published goals Gritti, Saffiotti, Seo, & Cho, 2006). PEIS is a middleware sys-
for the Control Layer from the decision module and control tem through which information is exchanged among hetero-
feedback from the Control Layer. It also maintains the high- geneous devices connected through a network.
level descriptions of the ecology through the event table,
reasoning target table and goal table. The ‘Event table’ Experiments and results
stores descriptions of all events passed to the Cognitive
Layer. The ‘Reasoning target table’ stores all desired rea- The RUBICON system for a smart home environment consists
soning outputs that the reasoning module produces. The of heterogeneous computational nodes such as environmen-
‘Goal table’ stores description of all goals that the decision tal sensors and mobile robotic devices. An overview of a
module uses for the RUBICON system. The memory element smart home with different sensors and robots is depicted
‘Data_for_event’ stores the confidences of the event data in Fig. 9. In order to evaluate the cognitive architecture,
that the Cognitive Layer receives from the Learning Layer. it is necessary to handle multiple events that may occur in
The memory elements ‘Data_for_reasoning_output’ and the ecology, and in particular extract higher-level intelli-
‘Data_for_prediction_output’ store processed information gence through this cognitive system. We have anticipated
from the reasoning module and the prediction module 19 events as inputs from the Learning Layer reflecting activ-
respectively. The posted goals for the Control Layer are ities of a user and the states of the environment and a set of
stored in ‘Cogl2cl_goal_issue’. The control feedback corre- 10 reasoning outputs to reflect the network’s capabilities of
sponding to posted goals will be received by the Cognitive reasoning across user activities and current states of the
Layer asynchronously. We have made a provision to store ecology. We also consider a set of seven goals for the Con-
the feedback information in the respective feedback mem- trol Layer. The anticipated events, reasoning outputs and
ory module; e.g., ‘Cl2cogl_feedback_goal1’ for feedback on control goals are shown in ‘Events Table’, ‘Reasoning output

User exercise
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
User relaxing
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
User in kitchen
1
Confidence

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
User cook activity

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Burglary alert
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Number of samples

Desired Reasoning module without prediction Reasoning module with prediction

Fig. 13 Example: cognitive reasoning outputs for data-points 3001–3900: set 1.


Development of a self sustaining cognitive architecture 105

table’ and ‘Control goal table’ respectively in Fig. 10. We be observed that only three goals are available in the tuple
have synthesized 4500 data samples (classification confi- space for that instance. The list is updated as new goals are
dence values e(0, 1)) including data for 19 events and 10 posted or a new instance of a previously completed goal is
reasoning outputs. The dataset is generated to ensure a decided. The situation is depicted in Fig. 11 where the ‘Con-
richness of variability with sufficient complexity to exercise trol Goals: to Database and PEIS’ table and the ‘Tupleview’
the reasoning capabilities of the cognitive system. show availability of seven goals.
A visualization of the cognitive process is presented in The SOFNN based prediction model is first trained offline
Figs. 10 and 11. The anticipated events, reasoning targets using the first 3900 samples of our dataset. Table 1 shows
and control goals are shown in these figures. At every the RMSE errors and the number of EBF neurons needed to
time-stamp, current event information is received from achieve a high level of prediction accuracy for the model.
the Learning Layer. Fig. 10 shows an instance of the cogni- After that, the trained prediction model is used in the online
tive activity illustrating input data for the prediction model operation of the reasoning module. At every time-stamp,
in the ‘Prediction Inputs’ table. The reasoning and predic- current event information is received from the tuple-space.
tion outputs are shown in the ‘Reasoning Outputs’ and ‘Pre- We use a sliding window of 300 data samples for this work.
diction Outputs’ tables respectively. The cognitive decision To validate the effectiveness of this approach we can com-
module assesses the status of the ecology as postulated by pare the performance of the reasoning system in the ideal
the cognitive reasoning module and assigns goals for the situation (where the ideal targets are available at each in-
control layer. The ‘Control Goals: to Database and PEIS’ ta- stant and thus without a prediction model) against the pro-
ble (Figs. 10 and 11) depicts the last posted goals for the posed self-sustaining architecture (where the targets are
Control Layer. It is shown that the decision module posts generated using the prediction model). It is to be noted that
a goal with an expected time (time format the ideal situation is impractical in a real deployment as the
‘yymmddhhmmss’) of completion and can post a single goal target signals are not known a priori in an online system.
(goal id 6) or multiple goals at the same time (for goal ids 1 Nevertheless, this comparison allows us to benchmark the
and 2) (Fig. 10). These goals are posted in the tuple space. performance of our self-sustaining approach in terms of
The ‘Tupleview’ shows the information related to the the difference in the network structures (neuron count)
events and control goals in the PEIS tuple space. It is to and the accuracy of the predictions. Fig. 12 illustrates the

Bring phone
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Open door situation
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Fire alert
Confidence

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Dripping alert
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Cleaning situation
1

0.5

0
3100 3200 3300 3400 3500 3600 3700 3800 3900
Number of samples

Desired Reasoning module without prediction Reasoning module with prediction

Fig. 14 Example: cognitive reasoning outputs for data-points 3001–3900: set 2.


106 A.K. Ray et al.

User exercise
1

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
User relaxing
1

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
User in kitchen
Confidence

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
User cook activity
1

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Burglary alert
1

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Number of samples
Desired Reasoning module without prediction Reasoning module with prediction

Fig. 15 Example: cognitive reasoning outputs for data-points 3901–4500: set 1.

Bring phone
1
0.5
0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Open door situation
1

0.5

0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Fire alert
Confidence

1
0.5
0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Dripping alert
1
0.5
0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Cleaning situation
1
0.5
0
3950 4000 4050 4100 4150 4200 4250 4300 4350 4400 4450 4500
Number of samples

Desired Reasoning module without prediction Reasoning module with prediction

Fig. 16 Example: cognitive reasoning outputs for data-points 3901–4500: set 2.


Development of a self sustaining cognitive architecture 107

evolution of the reasoning SOFNN when trained in an online in Table 3. These results are also separated into the two
mode using the 4500 synthesized data points for these two experiments using the data points 1–3900 and 3901–4500
cases. In both cases we can clearly see the adding and prun- respectively corresponding to the two sets of figures. In gen-
ing of EBF neurons as event data are presented to the rea- eral, the RMSEs are slightly higher for the network with the
soning network. The use of the prediction model prediction model but this is expected as we are using an
introduces more variation in the target data as compared estimate of the target outputs to achieve the self-sustaining
to the ideal case and as a result a slightly larger number feature.
of neurons are required to maintain a similar reasoning per- From these results, it is observed that the proposed
formance. Table 2 lists the final number of EBF neurons re- online cognitive system can identify the desired situa-
quired; i.e., 17 neurons in the ideal case and 26 neurons tions when the teaching data are available for the sys-
with the use of the prediction model. Thus, the introduction tem. It can also identify the desired situations with the
of this self-sustaining capability is associated with a slight help of the prediction model in the absence of the
increase in the computational overhead of the reasoning teaching data.
system via a larger SOFNN.
Furthermore, the real benefit of this self-sustaining ap-
proach is demonstrated through a comparison of the effec-
tiveness of the new architecture to reason across the Conclusion
ecology to determine the status outputs of the system in
this self-sustaining mode compared with an ideal case. In In this paper, we propose a cognitive architecture for the
these experiments the reasoning system is continually RUBICON system. An online self-organising fuzzy neural net-
adapted (trained) in an online mode using either the ideal work based on the sliding window is developed as a core
target data or the estimated target data from the prediction component of its reasoning capabilities. The cognitive rea-
model. Figs. 13 and 14 show the 10 reasoning outputs from soning module has been implemented and utilised to extract
the reasoning network (they are separated into two sets, knowledge from realistic events occurring within the envi-
each representing five of the outputs for ease of inclusion ronment. The proposed cognitive architecture supports
in this document). These two figures only present the results MIMO SOFNN structure to address multiple reasoning tasks
for the data samples between 3001 and 3900 and this sec- at a time. We also demonstrate the usage of a prediction
tion of the dataset overlaps with the training data used model to support continuous online operation. A supporting
for the prediction model. From these two figures we have decision module and a memory module complement the
demonstrated that the reasoning network trained using overall operation of the system. The proposed system has
the prediction model has comparable performance with been applied to a smart home situation. The results show
the ideal case, which is to be expected using this portion that the architecture has the ability to reflect the dynamics
of the dataset. To further validate our approach we extend of the environment and its cognitive capability. We show
our experiments to unseen data for the prediction model, that the system has the ability to configure its neuronal
corresponding to data samples 3901 to 4500 from the data- structure through adding and pruning of neurons while
set. Figs. 15 and 16 show the testing outputs for this portion exploring the relationships between the inputs and the de-
of the dataset. The network with prediction model identi- sired reasoning outputs. Validation of this self-sustaining
fied the reasoning situations as desired, with only a few approach confirms that although the prediction model ap-
glitches. In addition there is no drift in performance over proach increases the size of the reasoning SOFNN (as com-
time as the prediction model maintains accurate prediction pared to the ideal case), the system maintains an
of the target outputs during this testing phase. To further appropriate level of accuracy in the reasoning system. This
compare the performance, the RMSEs for the reasoning has been tested using both seen and unseen data. This is a
outputs with and without the prediction model are shown significant development of an online SOFNN by introducing

Table 3 RMSE errors for the reasoning module during different parts of the dataset.
Reasoning outputs Reasoning module Reasoning module Reasoning module without Reasoning module with
without prediction model with prediction model prediction model prediction model
(Data points 1–3900) (Data points 1–3900) (Data points 3901–4500) (Data points 3901–4500)
User exercise 0.0797 0.0880 0.1035 0.1060
User relaxing 0.0478 0.0454 0.0558 0.0419
User in Kitchen 0.0658 0.0757 0.0679 0.0832
Bring phone 0.0665 0.0948 0.0682 0.1089
Open door situation 0.0535 0.0790 0.0520 0.0692
User cook activity 0.0605 0.0847 0.0705 0.0984
Fire alert 0.0310 0.0640 0.0432 0.0717
Burglary alert 0.0826 0.0797 0.0653 0.0599
Dripping alert 0.0841 0.0843 0.0829 0.0833
Cleaning situation 0.0473 0.0605 0.0532 0.0717
108 A.K. Ray et al.

a self-sustaining capability which has widespread applica- Laird, J.E. (2008). Extending the SOAR cognitive architecture. In:
tion beyond the RUBICON ecology. Proceedings of the artificial general intelligence conference.
Memphis.
Laird, J. E. (2012). The SOAR cognitive architecture. Artificial
Intelligence and Simulation of Behaviour Quarterly (134).
Acknowledgements Langley, P., Choi, D. (2006). A unified cognitive architecture for
physical agents. In: Proceedings of the twenty-first AAAI
conference on artificial intelligence. Boston.
The authors would like to acknowledge support of the Euro-
Langley, P., Cummings, K., Shapiro, D. (2004). Hierarchical skills
pean Commission. This work is partially supported by the EU and cognitive architectures. In: Proceedings of the twenty-sixth
FP7 RUBICON project (Contract No. 269914) – annual conference of the cognitive science society, (pp. 779–
www.fp7rubicon.eu. 784). Chicago.
Langley, P., Laird, J. E., & Rogers, S. (2009). Cognitive architec-
References tures: Research issues and challenges. Cognitive Systems
Research, 10, 141–160.
ACT-R. (2013). Retrieved March 2013. http://act-r.psy.cmu.edu/ Leng, G., McGinnity, T. M., & Prasad, G. (2005). An approach for
about/. on-line extraction of fuzzy rules using a self-organising fuzzy
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., neural network. Fuzzy Sets and Systems, 150(2), 211–243.
& Qin, Y. (2004). An integrated theory of the mind. Psychological Lewis, R.L. (1993). An architecturally-based theory of sentence
Review, 111(4), 1036–1060. comprehension. In: Proceedings of the fifteenth annual confer-
Broxvall, M., Gritti, M., Saffiotti, A., Seo, B.S., Cho, B.J. (2006). ence of the cognitive science society, (pp. 108–113).
PEIS Ecology: integrating robots into smart environments. In: Looks, M., Goertzel, B., Pennachin, C. (2004). Novamente: An
Proceedings of the IEEE International Conference on Robotics integrative architecture for general intelligence. In: Proceedings
and Automation. Orlando. of AAAI 2004 symposium on achieving human-level AI via
Byrne, M. (2001). ACT-R/PM and menu selection: Applying a integrated systems and research. Washington DC.
cognitive architecture to HCI. International Journal of Human- Magerko, B., Laird, L. E., Assanie, M., Kerfoot, A., & Stokes, D.
Computer Studies, 55. (2004). AI characters and directors for interactive computer
Carbonell, J. G., Etzioni, O., Gil, Y., Joseph, R., Knoblock, C., games. In: Proceedings of the sixteenth innovative applications
Minton, S., & Veloso, M. (1991). PRODIGY: An integrated of artificial intelligence conference, (pp. 877–884).
architecture for planning and learning. ACM SIGART Bulletin, Peebles, D., & Banks, A. (2010). Modelling dynamic decision making
2(4). with the ACT-R cognitive architecture. Journal of Artificial
Carbonell, J., & Gil, Y. (1990). Learning by experimentation: the General Intelligence, 2(5), 52–68.
operator refinement method. In Y. Kodratoff (Ed.). Machine Ray, A.K., Leng, G., McGinnity, T. M., Coleman, S. A., & Maguire, L.
learning: an artificial intelligence approach (Vol. 3, P. (2012). Development of cognitive capabilities for smart home
pp. 191–213). CA: Morgan Kaufmann publishers Inc.. using a self-organizing fuzzy neural network. In: 10th IFAC
Duch, W., Oentaryo, R. J., & Pasquier, M. (2008). Cognitive symposium on robot control, (pp. 447–454). Dubrovnik, Croatia.
architectures: Where do we go from here? Frontoers in Artificial Samsonovich, A. V. (2012). On a roadmap for the BICA Challenge.
Intelligence and Applications, 171, 122–136. Biologically Inspired Cognitive Architectures, 1, 100–107.
Edelman, G. M. (1993). Neural Darwinism: selection and reentrant Sun, R. (2007). The importance of cognitive architectures: An
signaling in higher brain function. Neuron, 10(2), 115–125. analysis based on CLARION. Journal of Experimental and
Erlhagen, W., & Bicho, E. (2006). The dynamic neural field approach Theoretical Artificial Intelligence, 19, 159–193.
to cognitive robotics. Journal of Neural Engineering, 3, 36–54. Sun, R. (2009). Motivational representations within a computational
EU FP7 RUBICON project. (2011). Retrieved March 2013. http:// cognitive architecture. Cognitive Computation, 1(1), 91–103.
www.fp7rubicon.eu. Taatgen, N. A. (2005). Modeling parallelization and speed improve-
Franklin, S., & Patterson, F. G. (2006). The LIDA architecture: ment in skill acquisition: from dual tasks to complex dynamic
Adding new modes of learning to an intelligent, autonomous, skills. Cognitive Science, 29, 421–455.
software agent. Proceedings of the international conference on Tahboub, K. K., & Al-Din, M. S. (2009). A neuro-fuzzy reasoning
integrated design and process technology. Society for Design system for mobile robot navigation. Jordan Journal of Mechan-
and Process science. ical and Industrial Engineering, 3(1), 77–88.
Gupta, S., & Gupta, M. M. (2000). Perspectives on computational Takagi, T., & Sugeno, M. (1985). Fuzzy identification of systems and
perception and cognition under uncertainty. IEEE International its applications to modeling and control. IEEE Transactions on
Conference on Industrial Technology, 2, 221–224. Systems, Man, and Cybernetics, 15(1), 116–132.
Haigh, K., Veloso, M. (1996). Interleaving planning and robot Tambe, M., Johnson, W. L., Jones, R. M., Koss, F., Laird, J. E.,
execution for asynchronous user requests. In: Proceedings of the Rosenbloom, P. S., & Schwamb, K. (1995). Intelligent agents for
international conference on intelligent robots and systems, (pp. interactive simulation environments. AI Magazine, 16, 15–39.
148–155). Trafton, J. G., Cassimatis, N. L., Bugajska, M., Brock, D., Mintz, F.,
Just, M. A., & Varma, S. (2007). The organization of thinking: what & Schultz, A. (2005). Enabling effective human-robot interaction
functional brain imaging reveals about the neuroarchitecture of using perspective-taking in robots. IEEE Transactions on Sys-
complex cognition. Cognitive, Affective and Behavioral Neuro- tems, Man and Cybernetics, 25, 460–470.
science, 7(3), 153–191. Vernon, D., Metta, G., & Sandini, G. (2007). A survey of artificial
Kostadinov, S., & Grinberg, M. (2008). The embodiment of a DUAL/ cognitive systems: implications for the autonomous develop-
AMBR based cognitive model in the RASCALLI multi-agent ment of mental capabilities in computational agents. IEEE
platform. Intelligent Virtual Agents, Lecture Notes in Computer Transactions on Evolutionary Computation, 11(2), 151–180.
Science, 5208, 356–363.