You are on page 1of 12

Using Collective Decision System Support to Manage Error in Wireless Sensor Fusion

Arnold B. Urken Professor of Political Science Stevens Institute of Technology Hoboken, NJ 07030 aurken@stevens.edu

Presented at Fusion 05, the International Conference on Information Fusion, Philadelphia, PA., July, 2005.

1

Using Collective Decision System Support to Manage Error in Wireless Sensor Fusion*
Arnold B. Urken Professor of Political Science Stevens Institute of Technology Hoboken, NJ 07030 aurken@stevens.edu

Abstract – When sensor fusion uses voting methods to
produce collective decisions on the basis of incomplete and imperfect information that would be produced if voting information were perfect and complete, the collective outcomes will be error-resilient. These outcomes will not be changed by breakdowns in wireless network communications or decision making errors. Error-resilient collective outcome (ERCO) analysis makes it possible to predict how long to wait or how many votes to reach an optimal collective decision. ERCO analysis also provides a new framework for gaining strategic and tactical advantages from networkcentric information sharing. This framework raises new theoretical and empirical research opportunities for integrating voting theory and fusion research. Keywords: wireless sensor networks, distributed detection, error, decision fusion, voting systems, errorresilient. Patent pending. Portions of this work were supported by contract DAAE30-00-D-1011 to the Stevens Wireless Network Security Center, 2004. Approved for General Public Release.

network communications and faulty decision making. This paper outlines a new approach to wireless sensor fusion that uses voting systems to manage these errors. The paper is organized to explain how voting system can be designed to provide error-resilient sensor fusion. Section 2 provides a framework that explains the motivation for developing a new approach and summarizes the state of the art in building error management into voting processes. Section 3 presents the concept of an error-resilient collective outcome (ERCO) and explains how voting systems can be designed to measure ERCO efficiency for complex decision tasks in risky network environments. In these systems, voting methods are used to answer different and complementary questions about the same data. Section 4 applies this theoretical approach to a complex decision task in which voter ratings are processed through plurality, approval, and Copeland scoring methods in a Monte-Carlo simulation to compare their ERCO efficiency. And Section 5 discusses the simulation results and outlines key questions for future research.

2

Sensor Design and Deployment

1

Introduction

Current developments in the design and deployment of sensors are challenging existing methodologies for collecting data and producing useful information in wireless networks. In commercial and security applications of sensor technology, producing precise and accurate intelligence is being constrained by new standards for reliability, cost, processing speed and energy conservation. Although voting methods have been used to address problems of sensor communications in networks, sensor fusion techniques have not been developed to overcome errors caused by breakdowns in

New sensor designs and deployment plans are driving forces in the evolution of sensor fusion techniques. For example, sensors that use light-scattering technology to identify and detect more than one agent have led to proposals [1] for deploying large sensor arrays of multipurpose sensors in cities to provide protection against NBC (Nuclear, Biological, and Chemical) attacks. These deployments could provide early warnings that enable targeted populations to take evasive action and permit first responders to mitigate damage. Innovative use of materials is extending such capabilities by creating smaller, mobile, and inexpensive sensor systems that can increase the scope and accuracy

2

of multifaceted data that can be collected to produce knowledge [2]. By understanding the underlying complex patterns in such artificial environments, controls can be designed to generate precise and accurate sensor information. However innovations in the ability of sensors to produce complex knowledge have not taken full account of the problems of producing information in wireless networks. Sensor capabilities are limited by decisionmaking and transmission errors. Physical interactions with sensed environments can degrade sensor reliability and speed in detecting phenomena. Even if sensor performance is not degraded by environmental conditions, technology costs and energy constraints may limit the feasibility of deploying enough sensors to monitor a situation. Moreover, when sensors are not attacked by physical or cyber attacks, the wireless networks that are needed for transmitting data and producing knowledge pose risks. Data collection can be thwarted by malicious actions that divert messages to the wrong destination or overwhelm the processing speed and energy constraints provided by network architecture. Although malicious attackers can use commercial jamming devices to thwart wireless communications, the same effect can be caused inadvertently by environmental distortions from background radiation from buildings. For these reasons, decision fusion should not be considered an afterthought in the development and deployment of new techniques for sensor knowledge. Error should be integrated into the design of wireless sensor architecture. Wireless systems based on such designs will facilitate the development and deployment of innovative sensors in two ways. First, they can remove obstacles that limit deployment of emerging sensor techniques for producing more complex intelligence. And second, wireless sensor systems that are resilient to error will enable designers of sensors to increase the complexity of inputs that can enhance the scope and accuracy of knowledge.

2.1

Voting, Error, and Decision Fusion

Sensor fusion models have been developed to plan decision tasks and the collection of sensor data, address the ontological basis of fusion processes [10], use Bayesian techniques to manage the integration of sensor data [13], and design local sensor decision thresholds to maximize detection performance [12]. Fusion models that have used voting systems to achieve similar analytic

objectives [11] have drawn from voting theory as well as theoretical insights about processing data in computer networks [4, 5]. Each of these analytical perspectives addresses the problem of error in different ways, though neither perspective incorporates the concept of errorresilient fusion into the voting process itself. The computer science and computer engineering literatures have used voting methods in sensor fusion because processing of votes does not require much bandwidth or computational overhead. With notable exceptions [6], applications of voting systems are confined to the use of simple methods of weighting votes with majority rule to control sensor fusion processes. In these studies, there is usually a focus on how to weight the votes. Voting systems contain subsystems that enable individual voters to communicate information about preferences and judgments to form a collective outcome. Each voting system contains subsystems based on rules for the endowment of votes that can be used to express individual information, rules for the allocation of votes, and rules for aggregating votes to create a collective outcome. For instance, “plurality voting,” commonly used in elections in many Western democratic polities, includes an endowment of one vote, an allocation constraint that restricts assigning the vote to a single choice—normally without splitting or saving the vote— and a plurality aggregation rule that recognizes the choice with the most votes as the winner. Computer scientists counterbalance the application of voting methods with techniques for managing problems caused by breakdowns in network communication that prevent the production of collective outcomes. For example, statistical techniques are used to remove clutter, cleanse data, and weight votes. These techniques depend on the assumption that data collection satisfies quantitative and qualitative requirements necessary for the application of statistical methodology. These requirements entail the creation of networks that are computationally intensive, energy inefficient, and dependent on sequential sharing of information [7], In the theoretical voting literature, where elections are the focus of analysis, the most common assumption is that votes are collected successfully to produce a collective outcome that reveals the group preference. Problems in voting theory, social choice, and collective decision making analyses focus on how to process the voting data once it received so that collective outcomes with particular attributes can be created. And although

3

voting theorists often disagree about the properties of these attributes, the question of what happens if votes are missing does not normally arise in arguments for or against the use of a voting method [9]. In pre and post-election surveys sampling and data cleansing techniques are used to deal with missing data by making a priori assumptions that permit the creation of stratified samples. But voting analysts focus on problems of preference aggregation such as paradoxical and manipulated collective outcomes. Outside of the preference aggregation mainstream, voting theorists who address the problem of voter error by weighting individual votes based on the cognitive ability or competence of each decision maker. Neither of these analytical traditions of voting analysis considers what to do if and when error caused by network communications breakdown occurs. If all of the votes cannot be collected, group preferences cannot be inferred and individual voter preferences cannot be weighted to optimize group performance. Neither the voting literature nor the computer science literature builds error-management into the votecollection process itself [2]. Integrating error management into voting process design makes it possible leverage communication in the network applications layer to improve wireless sensor fusion. By interpreting network communications from the viewpoint of the recipient of votes, incomplete and imperfect voting data can be transformed into instantaneous and accurate information. Understanding the complex patterns underlying such voting transactions can be used manage sensor decisions in risky network environments.

3. Error-Resilient Voting Analysis
This section provides a formal definition of an errorresilient collective outcome (ERCO) and explains how to analyze voting systems to compute the probability of producing ERCOs.

3.1 What is an ERCO?
An ERCO, error-resilient collective outcome, is a voting outcome based on incomplete and imperfect information that would be produced if information about the voting situation were complete and perfect. Humans make intuitive use of ERCOs in simple situations. For example, suppose a central commander relies on ten sensors to report whether an object is A or B and bases an inference on the majority outcome. If the commander has already received 6 votes in favor of A, then the

outstanding 4 votes cannot change the collective outcome. The score in favor of A may increase or stay the same, but the outcome cannot be changed by lack of information caused by network communications and/or sensor errors. In our hypothetical convoy assessment example with only two choices, six votes satisfied the aggregation requirement to make choice A an ERCO. However ERCO-based distributed inference provides a generic form of automated decision support that enables commanders to manage risk and uncertainty when the collective outcome is not as clear-cut as it is in our hypothetical example. For instance, suppose that the commander has received 5 votes in favor of A and 2 votes in favor of B. In this case, the commander must either wait to receive more voting information or make an inference that might be very risky and counterproductive. In such complex situations, it is not clear if the outstanding information has been delayed by network traffic or if breakdowns in network communications or sensor failures have caused the problem. Under such conditions, without collective decision system support, commanders may unwittingly avoid risky choices that are reasonable and make risky choices that are unreasonable. For more complex, non-binary decision tasks, ERCOs can be defined for collective decisions with two or more choices for a fixed number of human or machine sensor voters and an aggregation rule that determines that a decisive collective outcome has been produced. At each stage of data collection, the number of outstanding votes in the network can be represented in terms of the percentage of the total number of voters or time required to collect a particular segment of the outstanding votes. If at any stage of the process of collecting votes, the number of collected votes satisfies the aggregation rule and the collective outcome cannot be changed by receipt of any combination of outstanding votes, then the collective outcome is an error-resilient collective outcome (ERCO). ERCOs can be produced for choices on single or multiple dimensions for collective decisions that take place in client-server or peer-to-peer computer networking environments [8]. However this paper describes ERCO production for a complex decision task along a single dimension: the number of vehicles in a convoy. Regardless of the cause(s), the costs of waiting can be significant. Lives and property will be lost and

4

opportunities for redeploying resources to counterattack or take evasive action will be missed. And emergency responders will not receive early warnings to prepare to care for victims.

3.2 Designing ERCO-Efficient Processes
The probability of producing an ERCO depends on the voting system used to process voting data and produce collective outcomes.

3.21 How Voting Systems Work
Ratings, inputs for a voting system, can be based on ordinal or cardinal scales. These inputs are processed according to rules for communicating the rating information, converting this information into votes and aggregating the results into a collective outcome. Rating communication depends on vote endowment and allocation rules that govern the expression of rating information. The vote endowment fixes the number of votes that can be used to express ratings while the vote allocation rule sets constraints on the allocation of the endowment. The aggregation rule determines how many votes are required to form a winning coalition.

5

The following chart illustrates the definition of these rules for three systems. Table 1—Subsystems of Three Voting Systems

When the original inputs are processed in AV and voters cast one approval vote for each choice that equals or exceeds their average rating, the vote allocations are

The OPOV system, a fully-specified description of “plurality” voting, reveals which choice is most frequently top-ranked in voter ratings. Approval voting (AV) shows which choices are approved (or disapproved) by a plurality or majority of voters. And Copeland voting reveals the relative collective intensity of preference that voters express in their ratings. Although voting theorists debate which voting system is best, each system answers different questions about the collective outcomes produced by the same voting or rating inputs. All voting systems may have paradoxical attributes and do not necessarily generate consistent collective outcomes. Consider the processing of the following voter cardinal ratings for three choices, A, B, and C. Table 2—Hypothetical Voter Rating Scenario

When these inputs are processed in a OPOV system with plurality rule, the allocations are Table 3—Conversion of Ratings into Single Votes

and the collective outcome is a three-way tie, a phenomenon associated with the “paradox” between individual and collective transitivity.

6

Table 4—Conversion of Ratings into Approval Votes

determine if an attack is reasonable. The sensors report the correct number of vehicles by rating an overlapping set of choices (from 0 to 4 vehicles) on a 0-10 scale, as shown below.

and B is the plurality winner (based on the definition of the aggregation rule determined by the number of voters who expressed approval (3 out of 3), which would not satisfy the requirement of a majority of the total number of allocated approval votes). Under Copeland voting with plurality rule, the ratings are first processed with Condorcet scoring, which computes the number of times that each choice is ranked higher than every other choice in voter preference ratings. These Condorcet scores are Table 5—Conversion of Ratings into Condorcet Scores

Copeland scores are then computed by subtracting the Condorcet scores to produce Table 6—Derivation of Copeland Scores

This illustration shows that voting systems can produce inconsistent collective outcomes, but also generate collective outcomes with different scores with consistent relationships. For example, B wins under AV and Copeland voting.

ERCO Production
The following example shows how voting systems produce ERCOs under OPOV system; the results for AV and Copeland voting follow the same logic and are presented below. In this example, ten sensors (including acoustic (AC) and infrared (IR) sensors) provide feedback to a commander to collectively identify the number of vehicles in a convoy so that the commander can

7

Table 7—Sensor Ratings for Convoy Assessment Task

When these ratings are converted into OPOV allocations, as shown below in Table 8: Table 8—OPOV Allocations based on Table 7

there is no majority winner (since “4 vehicles” receives only 4 out of 10 votes), leaving the commander, C2, without advice about whether to attack. And if IR1’s ratings are not received, the commander would be faced with a tied collective outcome. In this type of decision scenario, if all of the sensors were equally competent, ERCO analysis would focus on the probability of satisfying the aggregation rule at any point during the voting process. However sensors have diverse competencies in detecting objects depending on the manufacturer’s specifications and sensor limitations caused by different operating conditions. So we will assume that the ERCO objective is to produce a collective outcome that optimizes the group probability of making a correct collective choice with complex preferences and competencies. In this scenario, sensor votes can be weighted using the Shapley-Grofman theorem [10], which assigns weights to votes based on sensor competence or reliability using the following formula:

8

ln(p/1-p) where p = probability of a correct choice and (1-p) = the probability of an incorrect choice

(10),

Table 11—Example of an ERCO

(11)

(12)

If p=.2 for sensors AC1-AC3, .5 for sensors AC4AC6, and .8 for sensors IR1-IR4 have a .8 competence, the following chart shows the Shapley-Grofman (SG) [10] weights that would be used to adjust the value of the votes in Table 8: Table 9—Shapley-Grofman Weights

In this scenario, if the votes of AC2 and AC6 are not received, “4 vehicles” would be an ERCO.

4. Monte Carlo Analysis
To investigate the probability of producing ERCOs under OPOV, AV, and Copeland voting systems, Monte Carlo experiments were conducted in Mat Lab. Random variables include voter ratings (homogeneous or heterogeneous), decision competencies, and the time required for each set of voting information that successfully makes it from a sensor to the commander to form a collective outcome. Shapley-Grofman weights are used to adjust individual votes and time is represented by a Rayleigh distribution with a mean of 5 seconds. The following results are based on 20,000 runs for 100 sensors in a bimodal culture. In this culture, 75% of the sensors have homogeneous ratings and a high (.9) competence, and 25% of the sensors have heterogeneous ratings with a competence of .48. This scenario is typical of situations in which sensor disagreement can lead decision makers to take unreasonable risks. In such cases, human consumers of sensor fusion may be forced to rely on experience or intuition to resolve sensor disagreement. When only 1 or 2, sensors disagree, educated guesses may be reasonable, but, as in our scenario, when 25 out of 100 sensors disagree, additional decision support is needed to augment human capabilities. In each simulation run, a group of voters are randomly selected from a randomly chosen set of 100 voters that are drawn from a population with the bimodal cultural preference, competence, and vote transmission time

And the sensor votes in Table 8 would be transformed as shown below: Table 10—Sensor OPOV Allocations Based on Tables 8 & 9

with the “4 vehicles” choice identified as the winner under plurality or majority rule.

9

attributes associated with our scenario. Then voters are randomly selected to simulate the process of votes arriving at C2 (Command & Control) from distributed sensors. After a vote is received, the simulation finds the collective outcome under OPOV, AV, and Copeland voting methods and counts the number of times that a given collective outcome would be produced if all of the votes were collected. These counts are correlated with the proportion of outstanding votes and the cumulative vote transmission time associated with each ERCO. Then the counts of successes and failures in ERCO production are used to compute the probability of producing an ERCO. This probability can be used to compare the ERCO efficiency of voting systems under different scenarios. Ties can occur under all voting systems, though the probability of generating a tie is greater under some voting systems. For example, when preferences are heterogeneous, the probability of a tie is greater under AV than it is under OPOV or Copeland voting systems. The simulation results reported here are based on the assumption that ties are randomly broken. This assumption increases the performance of the AV system more than it improves the ERCO-efficiency of OPOV and Copeland systems. Ties could also be resolved optimally by, for example, by selecting the Copeland winner in a tied set if one existed. Other control variables such as false positive and false negatives are kept low and constant in the results presented in Figures 1-4 (below). All of these scenarios are based on the same initial conditions for a simulation run, but Figures 1 and 3 include homogeneous (similar) ratings or preferences, while Figures 2 and 4 show what happens when ratings or preferences are heterogeneous (diverse). Although all of the simulation results show that ERCO efficiency increases monotonically as a function of the proportion of outstanding voters or cumulative time, the relative efficiency of voting systems varies. For example, Figure 1 shows that the OPOV system is most ERCO-efficient when 75% of the voters have homogeneous preferences. As the proportion of outstanding voters declines, the probability of producing an ERCO increase rapidly so that ERCO efficiency exceeds .95 even when only half of the votes have been received. Under the same conditions, the AV and Copeland display an overlapping, less efficient ERCO production pattern. Both of these voting systems are

much less ERCO efficient than OPOV even when the proportion of outstanding voters approaches zero. Figure 1—Homogeneous Results based on Votes Collected

In this bimodal culture, Figure 2 shows that making the preferences of 75% of the sensors heterogeneous makes Copeland voting, not OPOV, most ERCO efficient. However Copeland voting’s ERCO efficiency increases more slowly and achieves a lower maximum than the OPOV system does (in Figure 1) when preferences are homogeneous. In Figure 2, with 50% of the voters outstanding, Copeland voting is 80% ERCO efficient and only reaches a maximum of .9 ERCO efficiency. Figure 2—Heterogeneous Results based on Votes Collected

Under these conditions, the OPOV and AV systems are much less ERCO efficient and display a relatively flat, overlapping pattern of ERCO production once 10% of the outstanding votes have been collected. Figures 3 and 4 present similar contrasts when preferences in this bimodal culture become more heterogeneous. Figure 3—Homogenous Results based on Time

10

In Figure 3, when preferences are homogeneous, OPOV’s ERCO efficiency closely approaches a maximum efficiency of more than .95 when only 250 out of 500 seconds have elapsed. In contrast, AV and Copeland voting display an overlapping pattern of ERCO efficiency that increases slowly and produces a maximum ERCO efficiency that is approximately 30% less ERCO efficient than the OPOV maximum. Figure 4—Heterogeneous Results based on Time

In Figure 4, preferences are heterogeneous and the Copeland method is most ERCO efficient. However the rate of change in ERCO efficiency is slower and the maximum ERCO efficiency is lower for Copeland voting than they are for OPOV when preferences are homogeneous. AV and OPOV display the same overlapping, less-efficient, and flat ERCO production pattern.

4. Discussion
ERCO efficiency can augment decision support for sensors by making it possible to predict how much (voting) information to collect or how long to wait to infer that the collective inference at any point in a decision making process is actionable. At the beginning of a sensor collective decision, C2 can determine how long to wait or how many votes to collect before taking action. ERCO efficiency analysis gives C2 an information advantage that can be used to take evasive action, launch a neutralizing counterattack, or dispatch first responders to mitigate the effects of an attack. During an attack, C2 can obtain on-demand intelligence about the implications of waiting longer to make a decision or collecting more information before making an inference. ERCO-based analysis may reveal if to wait for more votes to be received or time to pass or how much longer to wait or how many more votes must be collected to reach a distributed inference about the number of vehicles in the convoy.

In addition to revealing information about how long to wait and how much information to collect before reaching a distributed inference, ERCO analysis draws our attention to tradeoffs between voting systems and waiting and information collection. Since voting systems answer different questions about the same data set, choosing a voting system depends on what C2 wants to learn from the fusion process. In our scenario, for instance, if C2’s goal is only to learn which of the five choices in the convoy assessment situation the most frequently top-rated choice is in the shortest possible time, then, for example, the ERCO results in Figures 3 can be used with the OPOV results to reach an ERCO inference that meets the time constraint. For instance, C2 might set a time to decide of 200 seconds into the decision making process to achieve a high (.95) level of confidence. However, if C2 wants to know more about the collective assessment of the number of convoy vehicles, Figure 3 might be used to derive additional insights. For instance, if C2 wants to know how much more each of the five choices was preferred to every other choice, then Figure 3 would be used to take account of the Copeland results to verify the results derived from the OPOV pattern. In our scenario, the Copeland system is approximately 30% less ERCO-efficient than the OPOV system, does not become significantly more ERCOefficient as more time passes or more votes are collected, and displays volatility.. For these reasons, C2 might discount the value of gaining a second opinion based on Copeland voting and consider the AV results. So C2 might wait another 50 or 60 seconds to check the results for AV that produces higher ERCO efficiency than can be derived from Copeland voting.

5.2 Further Research
The scenario investigated in this paper illustrates the potential usefulness of ERCO analysis in providing collective decision support when the network environment includes imperfections in sensor decision making and network communication channels. The results of this scenario will be compared to those ERCO efficiencies produced when a) sensors are assumed to be perfect and communications channels are imperfect and b) sensors are imperfect and communications channels are perfect. ERCO production is also being studied for peer-to-peer scenarios and for situations in which decision tasks are multi-dimensional. In convoy assessment task, for example, additional dimensionality would be added by asking about other attributes of the convoy (e.g., vehicle shape, color, etc.) in addition to number.

.1 Tradeoffs among Voting Systems

11

As probability results are developed for a variety of ERCO scenarios, analytic models will be developed to describe the effects of increasing or decreasing the number of sensors and introducing periodic signal interruptions into the fusion process. In addition, situations in which the number of sensors is not known will be investigated. Concurrently, empirical tests will be conducted in wireless sensor test beds. * I would like to thank Russ Ovans and anonymous reviewers for their comments on an earlier draft of this paper.

International Conference on Telecommunications Systems, Monterey, October, 2003.

[9] Lloyd Shapley and Bernard. Grofman, Optimizing

Group Judgmental Accuracy in the Presence of Interdependencies, Public Choice, 1984.

[10] Erik P. Blasch and Susan Plano. Ontological Issues in Higher Levels of Information Fusion: User Refinement of the Fusion Process. Fusion 03, July, 2003. [11] David L. Hall and Sonya A.H. McMullen. Mathematical Techniques in Multisensor Data Fusion, Artech, 2004. [12] Ruixin Niu, Pramrod K. Varshney, Michael Moore, and Dale Klamer. Decision Fusion in a Wireless Sensor Network with a Large Number of Sensors, Fusion 04, July, 2004. [13] L. Ronnie M. Johansson and Robert Suzić. Bridging the Gap between Information Need and Information Acquisition, Fusion 04, July, 2004.

References
[1] Philip J. Wyatt, Early Warning and Remediation:
Minimizing the Threat of Bioterrorism, Journal of Homeland Security, April, 2002.

[2] Omid S. Jahromi, Bruce A. Francis, and Raymond
H. Kwong, Relative information of multi-rate sensors, Information Fusion, Vol. 5, pp. 119-129 (2004)

[3] Michele Zorzi and Ramesh R. Rao, Energy-

Constrained Error Control for Wireless Channels, IEEE Personal Communications, Vol. 46, No. 3, pp. 279-289, March, 1997 For Data Fusion and Dependable Computation, Reliability Engineering and System Safety, vol. 52, 139-151 (1996). common operational picture, Information Fusion, Vol 2, (2001) pp. 251-260.

[4] Behrooz Parhami, A Taxonomy of Voting Schemes

[5] Carl G. Looney, Exploring fusion architecture for a

[6] Ofer Melnik, Yehuda Vardi, and Cun-Hui Zhang,

Mixed Group Ranks: Preference and Confidence in Classifier Combination, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, No.8, pp. 973-981, Aug. 2004. Architecture for Distributed Inference in Sensor Networks, Proceedings of the Fourth International Conference on Information Processing in Sensor Networks, 2005. Decision Support Systems, Proceedings of the

[7] Mark A. Paskin and Carlos E. Guestrin, A Robust

[8] Arnold B. Urken, Time, Error, and Collective

12