Q-analysis and the Man-Machine Interface Frank O'Brien, Ph.D. Naval derwater Systems Center Newport Laboratory Newport, Rhode Island 02841 HCI INTERNATIONAL 1989, Boston, MA., U.S.A. Military and industrial information processing systems are large, complex, and dynamic. The complexity of informational requirements and system dynamics requires sophisticated mathematical capabilities to model the tnulti-dimensional relationships inherent in the man-machine system and subsystem components. Tt is axiomatic that hierarchical connections define the basic building elements constituting the man-machine interface. Many statistical and mathematical techniques exist to conduct research in man-machine interface issues. A not well-known mathematical methodology, Q-analysis, for assisting researchers in conducting a man-machine interface analysis is demonstrated. ‘Q-analysis is a mathematical technique that analyzes structural relationships of binary coded variables. The theoretical foundations of Q- analysis are derived from algebraic topology. Q-analysis provides a means to analyze complex multi-dimensional data sets. ‘Traditional probabilistic and/or Statistical modeling is often inappropriate in the contexts of exploratory and confirmatory data analysis. Q-analysis is especially appropriate for systems that require researchers to interact with data sets derived from highly skewed populations with multi-level data consisting of marked nonlinearities. ‘Q-analysis is applied by empirical researchers in the physical and social sciences as well as tournament games such as chess. Researchers will find Q- analysis to be usefull in understanding structural relationships. Quite often an understanding of data structure is the first step to empirical model construction. ‘A Q-analysis begins with an "incidence matrix’ containing binary coded information representing the occurrence of an event in two or more finite sets. Vector algebraic inner product operations produce matrices from which connectivity patterns are derived through equivalence class dimensioning, Graphical displays represent an indispensable tool for structural relationships depiction and offline interpretation. Analyses are conducted to observe pattern changes in the structure when information in the incidence matrix is augmented. Structural vectors that summarize the relationships among variables in the incidence matrix can then be compared to determine if "bottlenecks" or other anomalies exist. Other summary measures such as "eccentricity ratios" and “structural coefficients’ guide the interpretation of the online analysis. Researchers in possession of nonbinary data perform Q-analysis by selecting appropriate threshold values to establish the binary relations required for input to the incidence matrix. High flexibility s permitted in the selection of such “slicing parameters.” Existing relations may be in the form of frequency data, cell means, and other data levels. Flexible, interactive software to conduct structural analyses of many man- machine interface issues such as display utilization and decision-making networks, among others, has been developed. ‘The entire technique is demonstrated with a small data set. Literature references are provided for researchers to investigate the usefulness of Q-analysis in their research.Q-analysis and the Man-Machine Interface! Frank O'Brien, Ph.D. Naval Underwater Systems Center Newport Laboratory Newport, Rhode Island 02841 Overview: understanding structure in the "soft" sciences ‘Thematic in all fields of inquiry is the notion of structure. Although difficult to define precisely, researchers recognize the 6 importance of modeling behavior. Many scientists (and philosophers, artists and clerics) agree that understanding functionality is predicated upon an understanding of structure. To understand structure three languages of discourse are available: words, graphics and algebras (broadly conceived). Mathematical inquiry is conducted through functions, mappings and, at the broadest level, relations. Statistical analysis is largely confined to functional analysis. ‘The mathematical methodology of polyhedral dynamics analysis {or Q-analysis) was developed by the British mathematician R. H. Atkin in the 1970s. PDA was conceived as a descriptive and analytical language of structure. PDA can be defined as "the analysis of polyhedral interrelationships by purely algebraic means". It is written in the language of algebraic topology (the application of abstract [set theoretic] systems to questions about subsets of a topological space). Mathematical research has demonstrated the similarity of PDA to graph theory. An understanding of the mathematics or notation is not necessary to use the structural modeling methodology, for it can be used for practical and applied research tasks by empirical researchers in virtually any field of empirical inquiry after a relatively short course of instruction. What Q-analysis attempts to do in general is best described by Atkin: 1 without the generous assistance, encouragement and cooperation of many people; this research project could not have been completed. in particular, I would like to acknowledge the following persons: Mr. Luiz Cabral (Technical Reviewer) who encouraged and reviewed the research: Mr. Wayne King (Supervisor) who assisted significantly in the myriad managerial details of conducting the research; Dr. Susan Kirschenbaum (Engineering Psychologist) who provided the data and assisted in the interpretation, and Dr. John Dockery (The Joint Chiefs of Staff. Office of the Secretary of Defense, The Pentagon) who provided the STARPACK software /documentation and much technical assistance. Many other people of the Naval Underwater Systems Center provided the technical support necessary to conduct the research.The method of Q-analysis, which I have developed over recent years, concerns itself with this central problem of finding, analysing and interpreting the structures of relations in those areas of our experience we are describing as “soft”. The whole fleld can in a certain sense be treated as that of decision making.... [The] claim is that we are dealing with a structure which is, capable of being defined in precise mathematical terms [it is technically known as a simplicial complex), and in any particular study, there is a structure which acts the part of a (relatively) static backcloth .... to carry the “traffic’ of whatever kind-just as the euclidean space of the engineer must carry the traffic of “motion particles’. By ‘carry’ I mean only the connectivities of the geometries, at any dimensional level (the letter Q in the theory merely serves as a parameter for these dimensions), must be the determining features (the so-called topology) for allowed flow of traffic (or changes in that flow) PDA analyzes the structure of mathematical relations contained in well-defined sets which are constructed at one or more hierarchical levels via cover sets (everything at one level is completely subsumable by elements at a higher level such that, for example, the animal kingdom is a cover set for phylum which in turn is a cover set for class, then order, family, and genus-species). A major challenge is to actually construct this hierarchy through sets of relations. Often times we are forced to sample a universe adequately and postulate the hierarchy theoretically via iterative structural modeling such as q. Once the hierarchical levels are sorted out, the algorithm of Q- analysis can be applied iteratively until a satisfactory structure emerges. The q algorithm is essentially a single-link (nearest neighbor) hierarchical cluster analysis assuming a similarity coefficient calculated by counting common descriptors. The procedure provides interesting summary indices such as structure vectors, eccentricity ratios and first structure coefficients. For a fuller description of the theory and application, see the references. We at the Naval Underwater Systems Center have been experimenting with the q algorithm for about one year. The accompanying information is the result of one application to a data set of expert/novice differences in decision-making in the environment of submarine anti-submarine warfare. This analysis only skims the surface of PDA, More advanced procedures (e.g., graded polynomial patterns or time series applications to very complex structures) are being studied for application. As such, we are experimenting with the question: can PDA or q provide insights into decision-making structure that traditional functional analysis (e.g. factor analysis) does not provide? A brief review of the study Subjects: Three groups of naval personnel are examined: submarine commanding officers (COs), submarine school instructors and their students. Each group has 4 subjects.Method: Each group ts presented information on a relatively simple ASW (anti-submarine warfare) exercise previously conducted and recorded by the U.S. Navy. The actual maneuvers of ownship and the (enemy) "target" ship are played back to the subjects. The subjects are asked to select information they would desire in an actual submarine attack center to prosecute the target submarine. Thus, subjects in the experiment sample information on various computer displays to determine the current course, range, speed, etc. of the target ship, A computerized record is made of the information elements (display data) and latency of response for each subject. Eight (8) elements were available: four measured data categories and four computed data categories. Measured data correspond to "raw" unprocessed data while computed data have been computed by algorithms in the submarine computer system. For example, "range" of target has been computed from raw sensor data. The division of the 8 data elements into measured and computed available to subjects proved to be a significant distinction for analysis and interpretation of the study. This ASW exercise required three "legs" to prosecute the target. a leg being defined as the time in which ownship collects information before maneuvering to collect confirmatory information. At the end of each leg, subjects are informed of the current location, speed, ete. of the target. In the third leg, an anomaly was introduced which only experienced submariners would be able to detect. ‘Thus, the goal of the project is to determine how experts and novices differ in processing submarine information for strategic and tactical decision-making. Our goal was to determine what q could tell us about the response structure of each group: Summary of results ‘The accompanying figures and tables with explanatory notes present the major findings of the analyses. A separate sheet with introductory references is provided for researchers interested in more information about PDA and q.References Atkin, R. H. Multidimensional Man. Harmondsworth, Middlesex, England: Penguin Books, 1981. ‘A popular press account of Q-analysis. Motivates the argument that an interdisciplinary language of structure is provided through relational analysis. All of the difficult terns and concepis are covered. Highly enjoyable and stimulating as an intellectual work, this is probably the first place to start. . Mathematical Structure in Human Affairs. London: Heinemann, 1974. ‘Appendices contain very abbreviated theoretical developments of the set theoretic and topological foundations of Q-analysis. The main body of the book exemplifies q. Applications are made to questions about structure in chess, physics, art, reconstructing an ancient city. decision making in a university, etc. . Combinatorial Connectivities in Social Systems: An Application of Simplicial Complex Structures to the Study of Large Organizations. Basel, Switzerland: Birkhauser Verlag, 1977. A more integrated but still highly technical account of Q-analysis theory. This 1s probably the most readily available text on q by Atkin. An appendix gives a very detailed account of q applied to university structure. Gould, P. Q-analysis, or a language of structure: an introduction for social scientists, geographers and planners. International Journal of Man-Machine Studies, 13, 1980, 169-199. A stimulating highly readable introduction to Q-analysis. Gives examples of "success stories” of q applied to various fields of inquiry. Johnson, J. H. Some structures and notation of Q-analysis. Environment and Planning B., 8, 1981, 73-86. Avery technical summary of q) Johnson was Atkin’s doctoral student. His writings have expanded and modified q. He is perhaps the leading proponent of 4. having applied the theory to different areas. Q-analysis has been applied to many different fields of research as I discovered in my data base search. Articles have appeared in many different journals. The two journals (both British) that carry a significant amount of information on q are: Environment and Planning B. International Journal of Man-Machine StudiesTable 1, Figure 1 and Figure 2 The data of interest in this study are the frequencies of informational elements analyzed over time by each group in three "legs" of an anti-submarine (ASW) exercise. The weighted frequency data in each leg were divided up into equal thirds ("phases") to introduce within-leg time series observations. Table 1 shows a symbolic layout for a representative phase. The commanding officers are designated CO, instructors are designated IN and students ST. Each frequency (fj ) entry corresponds to the total number of times a person (Pi: = 1 to 4) analyzed a given informational datum (Dj; j = 1 to 8) during the phase. For Legs 1 and Il, Dj frequencies were summed for each subject to obtain the total looks (T,) for the phase. The slicing parameter (9, ) selected for the first two legs was the average of each subjects total looks, Note that the importance of informational element D8 necessitated a different slicing mapping for Leg IN. Essentially any occurrence of D8 in Leg Ill is allowed. Incidence matrix binary data were obtained by comparing each fij value with the established norm, as shown at the bottom of Table 1. ‘The 9 incidence matrices for each group can be defined as follows. Let 4 be the global relational operator specifying: Person { analyzes display datum j at the specified norm level in the specified time period. For each group let Kp(D:A) be the simpiictal complex with persons (P) as the simplices and the displays analyzed (D) as vertices, The conjugate simplicial complex, K,,(P:A"') , defines displays as the simplices and persons as the vertices. Kp(D:A) allows an investigation of multidimensional person connectivities while K,,(P:\7') allows an analysis of multidimensional display connectivities. Our principal interest lies in display utilization patterns (K,,) which might differentiate each group; however, Kp provided interesting insights into individual differences. The analysis focussed on "classical" Q-analysis as developed by Atkin (1974. 1977 and 1981). A software package STARPACK was used to carry out the primary analyses. First and second structure vector analysis of each simplicial complex provided a global view of structural differences of each group. Eccentricity ratio analysis provided interesting insights into expert/novice differences. Equivalence class membership for K,, was analyzed by a modified connectivity tree and “information demand” graph. To capture the multidimensional relationship between input/output mappings of persons and activities, the first structure coefficient (h) was employed. Finally, Q-analyses were carried out on individual commanding officers. Figure 1 summarizes graphically the binary data in each incidence matrix. ‘The squares correspond to people depicted as selecting one or more informational elements (computed or measured). For example, in the first Leg, first phase (A), CO1 analyzed D1, D2 and D3 while no CO analyzed D5 or D8. Figure 2 is derived from Figure 1 and shows a general view of measured vs. computed preferences for each group. Major group differences are evident.Table 1 Representative Weighted Relations Matrices Data Categories Toial Sing Faramtor Person pi__02 D3 D4 DS 06 D7 08 m i or] ty te a: te [T. $h ce | - : : oo | | 7 : : co taf sone 4 Int IN2 IN3 INS ny tie te ar faa ee fe | Te sm] tty fe soe fe |. oF T sT2 7 . : . S73 5 . . . st¢ | fay fae ao fee [Tb Te, Slicing mapoing, Lege | and I Athy 28 ia 28 ui 0, otherwit Leg it: Poof, Zor sie8 whereLEG3 coe smermucrons sree a — AT BNESS Mop | oa 2) OAS ON 8 1] Sa R AA P t) OFS Oya |! fs KS ey FW 6 Sox & | OF ESO/NO} OPED | Ce Fu Oo} | Figure 1. Many-to-many mappings for the legsCommanding cua Instructors >| © 4 IE J] ep ‘| Sagh & : ® 9 ® © 060 roquency é 2|@ g 2 tA 18 1 20 a 2 2A 28 50 1A 18 1 2k 8 A ee LegiPhase LegiPhase students aio é ©o®O Frequency 1A 18 1G 2h 2 2 3A 3B LegiPhase - Figure 2. Computed vs. measured selectionsFigure 3 and Figure 4 Figure 3 shows eccentricity ratios for each person by group. The eccentricity ratio is a measure of how similar or dissimilar an individual is with respect to his colleagues. A value of 0 indicates that the same number and type of displays were analyzed by two or more people. For example, CO1 in IA analyzed the same three displays as CO4. CO2 however analyzes two displays but none in common with his colleagues; hence, he is "infinitely" ecceniric. Note that the experts are the most eccentric, and students the least eccentric. These data support one of the major findings of the study: viz. experts show large variation in information processing styles. This implies that experts have open a variety of routes to solve a common but very difficult intellectual problem. Figure 4 shows first structure coefficlents (h). his a measure of multidimensional structural dependence. Two trends are evident in Figure 4. First, experts are uniformly higher on h indicating that they demonstrated more structural coherence over time in information processing. Secondly, all groups show a drop in Leg Ill, the difficult part of the experiment. Students show the sharpest drop.8 Figure 3. ‘Commanding Otters eo ° ° Students van com Person Eccentricities or 2 oe oc oo eon osm ests ost [61A 1B IC 2A BABB Leg/Phase Figure 4. First structure coefficientsFigure 5, Figure 6 and Figure 7 Figure 5 shows the results of the q analysis succinctly in the form of a connectivity tree. Much like a cluster analysis output, the connectivity tree shows the dimensionality (q) of display utilization and the number of equivalence classes (clusters of element connectivities considered pairwise) at each q level (q = -1 means the display is not used). The patterning of responses provides insights into expert/novice differences in submarine decision making. A brief summary of Figure 5: + The expert information processing structure is a compact dense network of rich interconnectivity. Experts show a pattern in each leg of first analyzing computer processed solution data (data labeled 1 to 4), then measured (unprocessed) information is analyzed. Experts know rules of thumb to convert raw data into solution information so they can form and entertain hypotheses quickly, thereby fostering faster more accurate decision making, However, in Leg Ill experts call more upon the computer for answers because the anomaly forced them into becoming very conservative at a time when mistakes can be very costly (e.g., they would get closer to the target in Leg Ill exposing them to possible counter-detection), Experts are seen to call upon a richer wider variety of system information than the novices. Experts know from experience that they must search broadly for information in the low signal-to-noise environment of ASW. + The instructor response structure more closely resembles COs than students. Instructors follow the textbook throughout the exercise. Notice, for example, that each inter-leg phase is almost identical (the higher dimensionality in Leg Il is caused by one subjects late participation in the experiment). + Students patterns are essentially random -- a "shotgun approach’. Figure 6 summarizes a p force analysis derived from the q values of Figure 5. This analysis examines changes in the importance each group attached to a unit of information. It is thus an additional way to summarize structural change over time. Successive phase q values for each datum (of highest dimensionality) are subtracted pairwise. Six categorical distinctions were derived. For example, in Leg I the commanding officers placed increasing importance on the datum D8. Each group shows a very different pattern in the differential importance placed on the information. COs show a steady increase in demand for the computed information (D1 to D4). Note inFigure 5, Figure 6 and Figure 7 (cont,) Leg Ull virtually all information has a high demand. Instructors show essentially no change in demand for information, and students show a steady decline in demand for measured and an increase in demand for computed information. Novices show more extreme demand preferences. Might this suggest for students a frustrating search for the clue (D8) in Leg III? Figure 7 summarizes geometrically by convex polyhedra the results of the q-analyses applied to each CO separately. A major finding of the study was the remarkable individual differences among the experts. Individual q- analyses allow more information to be derived about each CO. Bold letters (A, B, C) indicate simplices (phases) and numbers (vertices) represent individual data analyzed. The polyhedra show, in effect, each submarine commanding officer's cognitive model of prosecution strategy and tactics applied to the ASW experimental scenario. These polyhedra show a number of interesting relationships, including: + Differences in activity level. For example, CO1 was the busiest CO ‘and CO? the least busy. ‘+ Differences in information categories analyzed. CO2 is primarily interested in computed data (D1 -D4). Perhaps CO2 was trained for a different type of submarine warfare? CO1 is obviously the most experienced of the group; he shows a highly structured grouping of ‘computed and measured information. (His multidimensional coefficient {h) was the highest, .93). + For most of the COs, DS, and to a lesser extent D4 and D7 stand out as, common informational linksCommanding Ortcers he 2 | UE ‘Som EP ots onl boas - Figure 5. Connectivity tree for KPa )co st Low Demand High Demand increasing increasing LES GW & al ® ® 1 fiz 31¢ 410€ e i Ga 213 |©OO©OO® 4 uw iz] Be &® @ ; @l2 aT (0) @@ u @ G2) [T+] QD um Figure 6. Information Demandcor coz cos coe a 7 2 4, 2 2 wn ° A LZ. | 2, Z| bel ° os > Ze\\. : Leo Figure 7. Polyhedral representations of CO time-simplices