Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
2Activity
0 of .
Results for:
No results containing your search query
P. 1
A Three-Layer Approach to Testing of Multi-Agent Systems (2009)

A Three-Layer Approach to Testing of Multi-Agent Systems (2009)

Ratings: (0)|Views: 77|Likes:
Published by Tomáš Šalamon
In this paper, a complex approach to testing of multi-agent systems is presented. Multi-agent testing should be divided into three layers. On the first layer, the individual agents should be tested. A stochastic approach to unit testing and the use of “stages” can be adopted. A stage is a perceptible segment of a virtual world where an agent is placed that is projected for agent unit testing. On the second layer, deadlocks and similar flaws of agent interaction should be fought. Our method of deadlock detection is based on monitoring recurring agent interactions. On the third layer, the behavior of the entire system is evaluated. Bottlenecks and “hot spots” of the system could cause serious performance problems. Stability of multi-agent systems in case of mass collapse of many agents should be tested using stress tests.
In this paper, a complex approach to testing of multi-agent systems is presented. Multi-agent testing should be divided into three layers. On the first layer, the individual agents should be tested. A stochastic approach to unit testing and the use of “stages” can be adopted. A stage is a perceptible segment of a virtual world where an agent is placed that is projected for agent unit testing. On the second layer, deadlocks and similar flaws of agent interaction should be fought. Our method of deadlock detection is based on monitoring recurring agent interactions. On the third layer, the behavior of the entire system is evaluated. Bottlenecks and “hot spots” of the system could cause serious performance problems. Stability of multi-agent systems in case of mass collapse of many agents should be tested using stress tests.

More info:

Published by: Tomáš Šalamon on Feb 21, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

06/10/2010

pdf

text

original

 
A Three-Layer Approach to Testing of Multi-Agent Systems
Tomas Salamon
1
 
1
Department of Information Technologies, University of Economics, Prague, CzechRepublic. salamont@vse.cz
Abstract.
In this paper, a complex approach to testing of multi-agent systems ispresented. Multi-agent testing should be divided into three layers. On the firstlayer, the individual agents should be tested. A stochastic approach to unit testingand the use of “stages” can be adopted. A stage is a perceptible segment of a vir-tual world where an agent is placed that is projected for agent unit testing. On thesecond layer, deadlocks and similar flaws of agent interaction should be fought.Our method of deadlock detection is based on monitoring recurring agent interac-tions. On the third layer, the behavior of the entire system is evaluated. Bottle-necks and “hot spots” of the system could cause serious performance problems.Stability of multi-agent systems in case of mass collapse of many agents should betested using stress tests.
Keywords:
Agent-based simulations, testing, methodology
1. Introduction
Multi-agent systems are distributed information systems composed of agents. Theagents are independent, autonomous units of code with their own goals that inter-act, negotiate, collaborate and communicate with each other in order to reach theirobjectives. Information is distributed among the particular agents and there is noglobal control of the whole system. Multi-agent systems are able to solve prob-lems and tasks that are not suitable or performable for “traditional” comput-ing [17]. They can be used in a wide variety of fields: e.g., geographic informationsystems, cybernetics, robotics, logistics and traffic optimization, agent-basedmodeling and many others.Agent-based modeling is a computational method using multi-agent systemsfor simulation in social science. Because the properties and behavior of multi-agent systems resemble the characteristics of real social systems, multi-agent sys-tems can be used as an experimental tool in this field [18].Multi-agent systems are a promising technology. Since the 1990s they haveshifted gradually from theoretical studies to real applications. Although theoreticalworks about multi-agent systems and their background are relatively abundant, the
 
2 Tomas Salamon
papers treating practical issues of their design, development, implementation, de-bugging, testing, deployment and maintenance are scarcer.Multi-agent methodologies published so far, such as Prometheus [14],Gaia [20], SODA [13], Tropos [3], etc. seldom deal closely with debugging andtesting of multi-agent software. Just a few works published in recent years areconcerned with debugging and testing of multi-agent systems, and they are mainlyfocused on a particular technique or method, not on the testing approach of multi-agent systems as a whole. Jonker and Treur [9] suggest an approach of verificationof multi-agent systems based on temporal epistemic logic. However, such a me-thod is suitable only in the phase of system design and does not solve the testingof an existing system. Others [5], [19] deal with agent-oriented unit testing as auseful technique for testing of multi-agent systems (further elaborated in this arti-cle) but cover just a part of the problem. Liedekerke and Avouris [10] and Ndumuet al. [12] describe debugging methods of multi-agent systems based on visualiz-ing of their operation; that approach could be helpful, but it isn’t able to cover theentire complexity of the problem. Poutakidis et al. [15] suggest multi-agent systemdebugging using design artifacts, an approach that is focused on testing agent in-teraction protocols. Gatti and Staa [8] conducted research focused on the presentstate of multi-agent debugging and testing, and concluded that there is no singlebest approach to multi-agent debugging and testing, and that the best solution is acombination of several existing techniques.Although there are a number of tools and approaches designed for testing vari-ous kinds of problems in multi-agent systems, we are still lacking a consistent me-thod for multi-agent testing.This paper deals with testing of multi-agents systems generally and with a spe-cial interest in testing agent-based models. It is based on our research and the ex-perience with debugging and testing of agent-based models in the JADE multi-agent framework [2] and in our own simple multi-agent framework called
 AgEnv
.This paper is organized as follows. In the remainder of this chapter, our ap-proach of three layers is introduced. In the following three chapters, testing on allthree layers is described. Finally, in the fifth chapter are presented our conclusionsand suggestions for future work.
1.1 Three-Layer Approach
Debugging and testing of traditional applications is founded on testing of the indi-vidual parts of the system. Generally, if we can check every single part of the soft-ware and their integration, we can theoretically consider the software free of er-rors. There is indeed a problem that we are not able to check every state and everyeventuality of today’s complex software, so a perfect testing is not practically fea-sible. Nonetheless, this problem is outside the scope of this article.
 
A Three-Layer Approach to Testing of Multi-Agent Systems 3
In the case of multi-agent systems, the situation is more complicated. Multi-agent systems are composed of the individual agents that are separately testable inthe traditional meaning. In addition, however, an integral part of multi-agent sys-tems are the mutual relationships of these agents, their relationships with the agentenvironment, and incidental emergent properties that appear in the system. Simplysaid, we cannot deem a multi-agent system just a sum of its parts. The emergentproperties of the entire system are even more important than in a “traditional” in-formation system, so a different approach to testing is needed. Possible principlesand problems of testing and debugging of multi-agent systems are discussed inthis paper.Our method is based on testing of a multi-agent system divided into three “lay-ers.” In the first layer we conduct unit testing of the individual agents. We testwhether functionality of the agents corresponds with their design objectives.Agent unit testing should be indeed an integral part of their development, espe-cially when test-driven development methodology [1] is utilized. In the secondlayer, the testing of agent interactions is performed. We search there for the errorscoming from the agent interactions, because even if the individual agent is work-ing properly, hidden errors could still show up during the interaction of two ormore agents. A propensity for deadlocks is a typical problem that is discovered inthis phase. The third layer constitutes testing of the entire multi-agent system. Al-though we may find the individual agents as well as their interactions errorless, thewhole system could still fail. On this level, performance problems and bottlenecksof system hubs, and vulnerability to mass collapses of agents could appear.
2. First Layer – Unit Testing
On the first layer we test the individual agents. Coelho et al. [5] offered a methodof using
 Mock Agent 
based on the concept of Mock Object by Mackinnon et al.[11] in object-oriented programming. Mock Agent is a dummy agent without innerfunctionality. It possesses a proper interface to communicate with the tested agent.Mock Agent can send and receive messages to and from the tested agent and aprogrammer can evaluate whether the reaction of the tested agent is correct. Dur-ing testing we can observe the internal states of the test agent using tools likeagent inspector, logging and others if they are present in the agent framework.The actual process of testing is as follows: the testing environment creates thetested agent, the mock agents (Coelho et al. [5] recommend to create one mock agent for every role of the tested agent) and all other components of the systemand starts a
Test Case
where tests to perform are defined.Then the mock agent sends messages to the tested agent according to the testplan and receives its responses. The tested agent is monitored by agent monitor (if such tool is present in the system) which logs its activity for the test case. TestCase compares records with the plan to find out whether the agent’s behavior iscorrect.

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->