http://evi.sagepub.com Shifting Paradigms: Discourse Analysis as an Evaluation Approach for Technology Assessment
Orla Morrissey Evaluation 1995; 1; 217 DOI: 10.1177/135638909500100206 The online version of this article can be found at: http://evi.sagepub.com/cgi/content/abstract/1/2/217
On behalf of:
The Tavistock Institute
Additional services and information for Evaluation can be found at: Email Alerts: http://evi.sagepub.com/cgi/alerts Subscriptions: http://evi.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
Downloaded from http://evi.sagepub.com at INDIANA UNIV on July 9, 2008 © 1995 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.
Discourse Analysis as an Evaluation Technology Assessment
ORLA M O R R I S S E Y
University College at Cork
This article examines the application of discourse analysis in the empirical context of learning technology innovation. It outlines the various theoretical and methodological principles, as well as the technical procedures which have been followed in relation to the operationalization of an evaluation design using discourse analysis. The study itself is concerned with stakeholders’ subjective and experiential knowledge structures which underlie a culture of technology innovation. The article examines how technology innovation is discursively constituted through the creation of shared meanings and images among stakeholders.
This article seeks to outline an approach to discourse analysis which supports a learning evaluation paradigm in technology assessment. Within such an approach the evaluation agenda is explicitly concerned with the communication strategies, selfformation and self-reflection of social actors. A learning model implies a concern with the transformation of individual learning and actions into collective learning and actions; this requires the effective communication of different interests from the starting point of how social actors understand and evaluate their own experiences, values, beliefs and actions. This model is associated with the social constructivist tradition of Guba and Lincoln’sFourth Generation Evaluation (1989) and represents a paradigmatic shift from assessment-type models of evaluation. Such a model also challenges the practice of program-oriented evaluation where the normative interest is in social engineering. The discourse analysis design has been conceptualized to incorporate the stakeholder perspective and it begins with the premise that stakeholders have different constructions of reality. Like Guba and Lincoln’s hermeneutic cycle, the discourse analysis is designed to help make explicit the process of reflexivity for stakeholders. Guba and Lincoln (1989: 52) define stakeholders as ’groups at risk’; they share a direct stake in the evaluation, whether this is status, power or financial resources. Evaluation 217
Downloaded from http://evi.sagepub.com at INDIANA UNIV on July 9, 2008 © 1995 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.
The learning value of such an approach is defined by the operational context. A hermeneutic or interpretative approach is adopted in the analysis of stakeholders’ discourse. pilot testing and experimentation and implementation. and to promote an embedded evaluation culture within the program’s area of activity. Epistemologically. how social actors make sense of their own realities. making explicit the role of individual stakeholders in the evaluation process and the value of their own knowledge and experiences. i.2 The program is divided into three strands: R&D. is to work directly with stakeholders’ accounts of their actions and beliefs. The analysis of discourse opens up the possibility of understanding the symbolic dimension of ’change’ in specific social contexts of innovation. Its defined is to objective support learning technology solutions across the European Union the through development. Not for commercial use or unauthorized distribution. Such an advocacy agenda suggests an actionresearch approach to evaluation. validation and diffusion of learning technologies.
Context of the Research
This article arises from an empirical study carried out as part of ARTICULATE.e. developed originally as part of a wider evaluation study. to identify the combined effects of the ’voices’ and the global relevance of the discourse outside its purely semantic context. The focus of the evaluation exercise thus moves away from the traditional evaluation criteria of causality and predictability to a search for an understanding of the
meanings subjects bring to a particular context. The emphasis of the discourse analysis design.sagepub. i. A critical theory of discourse analysis is used to inform the interpretative framework.
. Given the uncertain environment of technology innovation.’ The program is part of an ongoing EU initiative in R&D on Telematic Systems for Flexible and Distance Learning. and (3) the communicative strategies which are encoded in patterns of meanings and action orientations.
Downloaded from http://evi. The central focus or objective of the study is to facilitate a research agenda which is concerned with subjective and experiential knowledge structures.methodologies designed from the stakeholder stance are concerned with the subjective interests invested in the evaluand-the area of activity to be evaluated. empirically defined as the local context of interpretation and
action. All rights reserved. The notion of embedding an evaluation culture simply means bringing evaluation closer to the human subject and internal decision-making processes. a trans-European evaluation project on learning technology innovations funded under an action program of the European Commission. Guba and Lincoln point out that most stakeholders are unaware either that they are being evaluated or of the value of information produced in evaluations. (2) how technology innovation is discursively constituted. there is a need for evaluation methodologies which capture baseline experiences. This framework addresses three levels of discourse production : (1) how stakeholders come to share a common appraisal of their actions. The role of ARTICULATE and its associated evaluation activities’ within the program is to contribute to the understanding of the actual and potential contribution of learning technology innovation.com at INDIANA UNIV on July 9. and how these define the social context of technology innovation. 2008 © 1995 SAGE Publications. the researcher is concerned with the question of how everyday practice is understood.e. it consists of 22 projects involving trans-European partnerships.
an approach which has the potential of dialogical communication (openness to negotiation) and learning (understanding the norms that influence individual and collective action) becomes central to the action-research model implied. what is noteworthy about how actors come to construct and represent their realities. Hence. reproduce or transform.
. The concepts of stakeholders and cultural logic thus serve as key referents to a discourse analysis approach. these include the self-formation of actors in organizations. Cultural logic refers to cultural meanings which become manifest through cultural discourses and which have a structuring effect on identity formations.e.e. There are a number of features of the monitoring system which influence the discourse analysis design.Part of the task of the overall evaluation project is to develop a multimethodological evaluation system which captures change at the micro-social and organization level of the innovation activities. The objects of analysis of the evaluation are the program’s pilots and projectsviewed as collective stakeholders-on technologies and systems development.4 i. and the methodological distinctions categorized herein as an interest in ’understanding’ and an interest in ’explication’ may be interpreted as a synthesis between a ’bottom-up’ approach to evaluation and a ’topdown’ approach.sagepub. The emphasis on change defines a traditional evaluation function. cultural logic implies a cultural reference for understanding the collective claims made by different stakeholders which allows us to identify a particular discourse ecology. i. the focus of the development of a monitoring system is to map the propositional field of education technology to the cultural logic of the pilot and project activities. All rights reserved. the monitoring system is envisaged as a reflexive tool and the notion of self-assessment is pertinent (in line with the learning/communication model). the subjective systems of self-understanding and meanings and objective ascribed criteria. rationale and objectives of the program taken as a whole. The notion of the stakeholder in evaluation research refers to those interested parties who constitute the constituency of the evaluand (in a direct sense). 2008 © 1995 SAGE Publications. The notion of a reflexive tool implies a learning evaluation approach. Specific phenomena are singled out for inquiry. the cultural patterns of relevance which pilot and project actors internalize. Alternatively.com at INDIANA UNIV on July 9. The background of the participating actors is indicated by the technological profile of the projects and the technical imperative which defines the goals. the methodological objective is to combine a model of ’understanding’ with a model of ’explication’. Second. when we consider the objective of the evaluation to promote reflexivity within learning technology pilots and projects. Not for commercial use or unauthorized distribution.
Downloaded from http://evi. In other words. the social construction of images of innovation and the role of formal evaluation actions. First. meaning the interaction and synthesis of different constructs produced and interpreted through discourse. actions and cognitions. Part of the their task is to explore the social conditions of advanced learning technology (ALT) which implies (1) an understanding of the social effect of ALT and (2) that a collective claim can be made which is communicable and potentially realizable. but when designed from the perspective of a learning model it is intended to capture the dynamic and evolving relationship between the above elements.
ethnography and conversational analysis. conversational analysis and variation analysis as distinct approaches in the linguistic analysis of discourse (Schiffrin. transformation and reproduction of objects and subjects of social life. speech act theory and theories of hegemony and ideology.
. is not clearly understood as a definable concept. in the study of political and media discourse the main emphasis is on the systematic categorization of events and participants. agency and social transformation. Schiffrin identifies speech act theory. 1993: 41). Not for commercial use or unauthorized distribution. For example. as a locus of ideology. and models which integrate along this continuum.e. frame analysis. methodological and theoretical traditions has meant that discourse.. or what in speech act theory is referred to as the illocutionary act. The positions (i. but constructs meaning for reality (Fairclough. for example: critical hermeneutics. cultural reproduction and transformation. the principal tenets of a critical discourse analysis theory need to be addressed. 1994).Principles of Discourse Analysis
outlining the design of a discourse analysis and its application in a specific empirical context. It is not simply a descriptive tool for representing reality. Cicourel. i. interactional sociolinguistics. The problem primarily stems from the adoption of techniques without explicit reference to methodological concerns. in that it contributes to the production. Top-down approaches are concerned with higher
Downloaded from http://evi..e. Within sociology alone one can identify different methodological paradigms in the application of a discourse analysis theory. textual analysis. constituting and constructing the world in meaning’. ethnography of communication.com at INDIANA UNIV on July 9. for example. interpretation and reception of meanings. distinguishes between top-down and bottom-up approaches as a general categorization of models of discourse analysis. but of signifying the world. for example.e. Van Dijk (1985: 5) defines discourse as ’a central and manifest cultural and social product in and through which meanings and ideologies are expressed or (re-)produced’. pragmatics. The following definitions assume the constituting and structuring effect of discourse: according to Donati (1992: 138) discourse is a ’language event the act through which ideative and symbolic constructs are actualized and made real in the human world’. All rights reserved. and discourse analysis in particular. paradigms) entailed in these definitions focus on different constitutive elements of discourse and the semantic space in which social meanings are produced and contested. Discourse is understood as ’language in action’. The different theoretical presuppositions entailed in these definitions conceive of language as a social process. A theory of discourse analysis has been developed within a range of disciplinary fields and theoretical traditions. They also imply a communication paradigm. i. Fairclough (1993: 64) understands discourse as ’a practice not just of representing the world. Methodological distinctions are drawn
. the communication that is made possible in ’saying’. power. 2008 © 1995 SAGE Publications. The sociological interest in discourse and its analysis is in the production.
between micro and macro or structural and contextual sensitive approaches to discourse. as well as production and interpretative processes.sagepub. Textual and frame analysis based on social semiotics and critical hermeneutics are combined in this study as techniques of analysis. Similar paradigmatic distinctions are made in linguistics. The mix of disciplinary.
whereas the use of frame analysis facilitates the macro level of abstraction. 1986: 464). In summarizing Donati’s discussion of framing the following definition is offered: a frame is a standardized. The identification of frames of reference allows us to move from discourse to the narrative structures which underpin meaning formations.
. identify and label their experiences of the world’ (Snow et al. Bottom-up approaches emphasize structural features of a discourse. the dynamics of communication are determined by how frames are symbolically constituted. The approach adopted for this study is an integrated model in that it seeks to combine a textual and frame analysis into its analytical framework. 1992: 141). All rights reserved. i.com at INDIANA UNIV on July 9.e. The analytical tools of textual analysis allow us to identify and understand the structural cohesion of a text beyond the mere descriptive reference to its propositional content.sagepub. phrases and sentences. For example. Kress’ distinction between discourse and text provides a useful reference for an integrated model.. Frames are understood as the macro-cognitive structures which social actors mobilize in the creation of shared meanings and understandings and which give meaning and reference beyond vocabulary style. A frame analysis enables the analyst to move beyond propositional claims and how they are articulated in terms of the mode of discourse. which is primarily concerned with discursive procedures. including ideational and interpersonal meanings. causality and responsibility in relation to propositional claims and the . perceive. Textual signifiers also enable the analyst to identify a set of meaning relations. function and social significance of language (Kress. This is indicative of the permeability of institutional discourses and shifting discursive procedures. whereas the study of discourse is concerned with the content. Gamson (1988: 220-1) develops the notion of frames as cultural themes which are mobilized in the process of
discourses. These are inferred from the ecology of utterances which make up a discourse. what they appeal to in terms of collective identity construction. The concept of a frame originated in the cognitive sciences to denote a higher-order pattern or schema-that known structure which gives meaning to perceptions.structure of argumentation and interactive communication.or deep structure which indicate the context of goals and beliefs. frames may be aligned or played off against each other. 2008 © 1995 SAGE Publications. to incorporate knowledge claims as cultural constructs. 1981: 24-8). form and structure of language and belongs to the linguistic domain. 1985: 27). discourse is more than a mere collection of words. but identifying it (Donati. syntactic structure and propositional content (see Corsaro. It is
Downloaded from http://evi. Textual analysis concerns the materiality.5 This is the micro level of analysis. Not for commercial use or unauthorized distribution. Goffman uses the concept of frame to indicate interpretative schemata which enable social actors to ’locate. for example. Social movement theorists point out that frames are strategically used in the creation of collective identities and in competing for symbolic opportunities. predefined structure of perception which enables the social actor to make sense of his or her reality. the use of textual signifiers enables the analyst to establish a relationship between agency. A frame analysis is concerned with the deep structure of language use and the communicative dimension of discourse. not just in the sense of describing reality. Although we may identify a dominant frame of reference in any given text. or which structure competing
levels of abstraction
The discourse analysis allows us to identify the constraints of the communication process on reflexivity and self-formation. The critical element of this approach is that it avoids the abyss of evaluative relativism by reconstructing the discourse(s) to a higher-level synthesis. There is an explicit link between the application context of the discourse analysis and the promotion of reflexivity within the reference group of the study. All rights reserved. as well as the subject. This is analogous to the ‘learning-by-doing’ model valued by innovation actors in dealing with technical problems. cognitive and other processes which play a key role in decision-making structures and in defining norms of action for innovation activities. or frame alignment. which appeal to constructs such as ’technical proof’ and ’what is socially desirable and acceptable’ from the broader discourse domain of new communication and information technologies. The discourse analysis design is comparable to Guba and Lincoln’s hermeneutic dialectic cycle in that it is designed to make explicit the creation of shared understandings about what it means to be involved in innovation activities. the task of identifying a frame is an interpretative process. 2008 © 1995 SAGE Publications. encodes and how these
symbolic messages are carried.e.
. For example. Not for commercial use or unauthorized distribution. which act as discursive signifiers in indicating a frame of reference. The discourse analysis combines the pragmatic goal of facilitating an evaluation culture with a clear methodological focus.sagepub. This opens up the opportunity for stakeholders to critique. particularly in terms of values and belief structures (i. The discourse analysis also explains the symbolic mechanisms. The analysis of discourse allows us to reconstruct from ’sense making’ the context in which actors find themselves. reflect on and/or reformulate their constructions as part of their own self-evaluation work (Guba and Lincoln. Part of this task is to identify internalized representations or schemes of the external environment and represent them according to the discursive procedures which structure their meaning. The coherence of cultural meaning systems (cultural logic) is constituted by discourse. how do narratives function in creating shared meanings? How do discursive procedures maintain ideological closure or open up the possibility of a reflexive discourse? This is particularly pertinent to evaluation approaches which focus on the collective actor as the object of study. Reflexivity essentially concerns the idea that everyday practical accounts are not only selfreferring. predefined expectations). Since the discourse is the product of
Downloaded from http://evi. Collective learning and meaning formations are made possible through communication. Adopting a frame or macro analysis enables the analysis to move beyond mere referential procedures (lists of linguistic and textual signifiers) to asserting an interest in language in action where agency is the central sociological tenet. but that these accounts reproduce and transform the social situations to which they refer.com at INDIANA UNIV on July 9.
Supporting a Communication Paradigm in Action Research
Promoting reflexivity defines the action dimension of the research in that it assumes that stakeholders have something meaningful to say about the world they occupy. 1989: 149). Other than in the use of textual features such as metaphors and analogies. This is achieved by making explicit symbolic.only by identifying
framing of a theme that we can begin to look at the symbolic particular frame.
otherwise known as a liaison officer.interactive and communicative event involving the social exchange of meanings. it seeks to answer the question. management project meetings which involve all the participating partners. however. The idea of setting up round-table discussions as defined contexts for the participation of stakeholders limits what can be meant in a discursive situation. The
Downloaded from http://evi. ’are shared meanings being constructed?’ The methodological approach.the internal reporting structures in order to elicit feedback from participants. at the level of descriptive findings. 2008 © 1995 SAGE Publications.e. where attention may be given to paralinguistic features. hence. although not in as much detail as one would require for a conversation analysis design. at an action level. This design follows an action-research framework. is unstructured and open to the participants’ shaping the event for their own needs. used in . The point of intersection between these two agendas is the pursuit of a constructivist or hermeneutic approach to evaluation.com at INDIANA UNIV on July 9. who has experience in the field and acts as a mediator between the analyst and the focus groups involved in the study. All rights reserved. Each round-table is audiotaped and transcribed in detail. the discourse analysis attempts to answer the question ’how are shared understandings emerging among project participants?’.sagepub.) The design of the discourse analysis is operationalized through self-evaluation fora. i.
The substantive area of inquiry involves two trans-European project teams: Project A concerns the development of a new dimension in computer-aided teaching and education and Project B concerns the development of an integrated system for simulation-based training in small and medium enterprise environments. Two hours are set aside for a round-table discussion which is facilitated in both cases by a fieldworker. Approximately 4 hours of discussion involving 23 participants across both round-table fora amount to 64 pages of text and over 3000 coded lines to be analysed. The round-table discussions were set up as part of each project’s concertation activities. attempts to incorporate two epistemological positions-combining a service role with a research agenda. The facilitator of the round-table discussions is given a set of questions which serve as loose indicators of the type of data the research delineates as possible topic frameworks. The format of the round-table.’ Otherwise the analysis would become too arduous to be meaningful. Identity tags were. and thus contains the focus of the analysis on the evaluation question of the construction of shared meanings in highly uncertain environments. its reveals something about the dynamics of what is unfolding in relation to the cultural logic of the focus groups and how this translates into normative lines of action. Not for commercial use or unauthorized distribution. otherwise known as round-table discussions. hence the focus of the study on naturalistic settings (natural language events) in a time/context frame which captures the process of innovation.
. (The names of the projects and identities of the partners and their organizations remain anonymous in all public presentations of this study. however. The intention behind these constructed events is to provide a forum which would have the potential to facilitate a process of self-evaluation for the participating actors.
com at INDIANA UNIV on July 9. The analytical level involves the deconstruction and coding of the text. Not for commercial use or unauthorized distribution. In order to reconstruct stakeholders’ constructions. All rights reserved. ’analytical’ and ’interpretative’. Codes are constructed for textual features which give coherence to the discourse. since she or he probe suggesting propositions and introducing topic areas for discussion and metaphors or specific meaning contexts. i. For the purposes of analysis the propositional content is organized sequentially. This is particularly useful where there are time constraints placed on the evaluation research. This is particularly pertinent in a context where a culture of communication and shared meanings is assumed. expanding circles of interviews are conducted with different stakeholders in a given social network. From the point of view of structural cohesion. including those stakeholders such as ’users’ who are indirectly constituted in the discourse? This dimension of the analysis is further expanded by identifying other textual functions which indicate ideational and interpersonal meanings. and these are staggered to allow for the analysis of each individual interview as a reference for conducting subsequent interviews with other stakeholders. intertextuality allows the analyst to look at relational processes. Each level is referred to as a separate process although they are analytically related.sagepub. The framework begins with features of intertextuality. how social practices are reproduced.e. and refer respectively to how discourse is used to reflect experientially and how the roles of participants are encoded in the discourse. 2008 © 1995 SAGE Publications.’
Downloaded from http://evi. 1993: 84-5). in other words. Guba and
Lincoln’s hermeneutic cycle requires an extended timeframe in order to create such intertextual contexts.are not excluded from the analysis. For example. and how texts are reproduced and transformed. in circumventing what can be said in relation to a topic area.
. This enables the analyst to identify the symbolic meaning of how individual or collective actors represent their actions to themselves from the standpoint of other stakeholders. it is also the starting point for identifying themes and topics and the intertextual context of the discourse. 1985: 44). in that participants recreate their personal narratives as part of the historical and collective narrative of the project. The round-table fora are an appropriate setting for the type of interaction which in its very essence is intertextual. for example. how assumptions are carried over from one speaker to another. maintained or transformed. The descriptive level concerns the prepositional content of the text and gives the first snapshot of the overall content of the discourse. An easy way of approaching this in dealing with discourse is to ask of the data: how do subjects position themselves vis-a-vis other stakeholders. In its broadest definition intertextuality refers to how utterances are shaped by prior utterances to which they are responding and the subsequent utterances that they anticipate (see Fairclough.
facilitator’s discursive contributions
Process of Analysis
investigative process moves iteratively between three levels of analysis which are as ’descriptive’. which are frequently found in the rhetorical structures of learning technology discourse. These correspond to the cognitive and action functions of discourse (Halliday and Hasan. Intertextuality serves as an analytical tool in understanding how subjects are constituted through texts. for such nebulous concepts as ’learners’ or ’users’.
Figure 1 illustrates the relationship between the different levels of analysis.sagepub. but textual signifiers such as metaphors and analogies may suggest a frame. Salient themes. The circular representation (1) refers to the centrality of reflexivity to the discourse analysis as a whole and (2) indicates the many iterations and reformulations required in a hermeneutic approach to discourse analysis. qualitative. once identified. The third level refers to the reconstruction of the texts in relation to the state of knowledge. in other words. Not for commercial use or unauthorized distribution. i.
The As the
Levels of Analysis
investigation process is constructed as iterative.
. the communication dynamic is structured by the symbolic packaging of frames. depends on a critical interpretative framework which offers a deep-structure analysis of the text.com at INDIANA UNIV on July 9. analyst moves from one level to another she or he is forced to reflect back and forth. The global relevance of the text refers to how the discourse of learning technology innovation is constituted and the social significance attached to technology innovation. Frames are not literally outlined in a text. heuristic and open. The mode of discourse (scientific. beliefs. natural. etc.
Figure 1. serve indices of meaningful speech acts. but coding is only a general categorical device and in forming the
Downloaded from http://evi.) is also important in signifying the context of meaning formations and the role of the active subject. however. its symbolic meaning structure. This is the descriptive level and it serves as a key reference for the analyst as she or he within and between the other two levels. what they appeal to and how the act of appealing is constructed. All rights reserved. These are subject to a deeper structural analysis by unpacking the framing mechanisms adopted. 2008 © 1995 SAGE Publications. Level 2 is done step by step as the analyst codes the text according to each analytical category. Such a framework often requires many iterations and reconceptualizations of the analytical categories adopted. Although we may identify a dominant frame of reference in any given text. wants and attitudes that are articulated.In the first step of the analysis salient themes are identified and mapped out as macro-propositional indicators of the discursive exchange among the focus group. At this level we are ready to address the role of discursive practices in creating the institutional context for leaming technology innovation and their structuring effect on shared meanings. The full sense of frame analysis.e.
The coding and search arrangements enable the analyst to carry out a ’content analysis’ which helps to clarify her or his interpretation of the dominant mode of discourse adopted by the participants in the round-table. extrapolate the meaning of the context in which the data are gathered for this How the subjects in the study interpret this ’event’ gives us an indication of the study. The facilitator/ fieldworker also has a role in terms of providing general observations and contextual comments to guide the analysis.
Since the textual
analysis allows for structural elements in the discourse to be captured. All rights reserved. the analyst has to move cyclically between the categories. are not designed purely for a traditional content analysis.sagepub. The final interpretation. for example. The process itself allows for this type of openness and flexibility. the methodological decisions made at each stage have to be addressed in terms of their implications for the final interpretation. Hence. It enables the analyst to embed contextual information into the data sets. the thematic structure. according to theme. This type of flexibility makes the ethnograph suitable to the type of open and iterative model of analysis suggested here. of the pragmatic goals of discourse analysis when designed from the potential of perspective an evaluation strategy which stresses the stakeholder stance. and their feedback on the findings will be a valid input into the final interpretation and critical to the development of discourse analysis as an evaluation methodology appropriate to a learning model. The degree of openness of the participants to this event is indicated by how they position
Downloaded from http://evi. propositional claims and frames with the expression of affinity and relational processes (social identities and events/ actions). 2008 © 1995 SAGE Publications. The coding is designed instead to facilitate the descriptive and explanatory dimensions of the analysis. The coding format is flexible and allows for the possibility of overlapping between boundary segments and nesting codes into larger units of meaning. The usefulness of this organization feature becomes more apparent when the analyst is dealing with multiple data sets. The discourse analysis has to have some practical input into the projects that have participated in the round-table discussions. Once the text is transcribed it is loaded on to a software program known as the Ethnograph. the purpose of which is to yield frequency of occurrence of certain categories. however.links between. particularly given the quantity of text involved in the study. The Ethnograph software program is designed around the twin concepts of decontextualizing and recontextualizing data. the appropriateness of different analytical tools and the use of conceptual models. to enable the analyst to work her or his way through different levels of the analysis process without turning the analysis into an arduous task. While the different levels are conceived of as analytically distinct. The movement between and within levels involves a critical review of assumptions. The program allows the analyst to search for key sections. for example. frame. The categories. Not for commercial use or unauthorized distribution. the numbered text is then hand coded according to the analytical framework described earlier and computed.8 This numbers the lines of the text. modality. does not belong to the researcher alone. however. they are also interrelated.
.com at INDIANA UNIV on July 9.
but that it is also central to the learning process: it entails the possibility of transforming practical actions. there is no ’added value’ unless learning can be realized in the first place and then transferred to other social contexts. you often forget to do that self-reflection.
The discourse analysis shows that reflexivity is not only a significant factor for the communication culture of a transnational project (made up of different stakeholders with different interests)..
. The discourse analysis shows that this is the case when partners operate from the standpoint of all the stakeholders involved.
.sagepub. I think we are forced also to do some self-reflection. not necessarily in the shape it is done now. Indeed. When the discussion shifts in one of the round-table discussions to the value of participating in this type of event. But at the same time. one participant points to reflexivity as a discursive procedure which has benefit for the social relations of a project and the realization of a
if you’re asking what did we discover in the last one-and-half hours. In those sections of the discussion where technical topics are foregrounded the objective mode of a scientific discourse is adopted and a didactic mode emerges at times as a more forceful type of text as participants seek to establish their own identity in rating a certain kind of knowledge (expertise) or perspective over another.e. we have quite a lot of ideas coming and this self-reflection.. The analysis also looks at how the participants position themselves subjectively or objectively in terms of their affinity to propositional claims by using textual signifiers such as ’transformation’ and ’modality’..themselves vis-a-vis each other in the context of the round-table fora and relationally in terms of the dominant assumptions encoded in the projects’ official text. Not for commercial use or unauthorized distribution. Let us turn briefly to the discourse produced on the evaluation function. evaluation is not perceived as having a formative influence on development projects. This is what usually makes up the narrative structure which participants develop through a dynamic discourse. and because of the always going frame of time . even when this is experienced as constraining on individual actions or interests. and I really believe that self-reflection is quite often very useful. because it looks like if you are always in an argument situation that you are always arguing with one another..9 i. you certainly realize that you are learning. when we look at how the field of innovation is discursively constituted.. the rhetoric of goals and objectives. it is an opportunity to show that you respect the other person. since this 227
Downloaded from http://evi. And it is clearly because also. Reflexivity is the self-organizing principle of R&D projects which function as temporal organization units. One of the key observations of this study (based on subjective accounts of the innovation process) is that when project actors deal with specific and transient problem contexts over the lifecycle of a project.com at INDIANA UNIV on July 9. All rights reserved. for example. the time question .. The round-table fora are presented and consequently perceived as self-evaluation or self-reflexive exercises. The self-reflective mode noted in the discourse analysis is the basis of how stakeholders account for their own positioning in events. 2008 © 1995 SAGE Publications. The discourse analysis also suggests that reflexivity lies outside formal evaluation criteria. can be captured by intertextual relations between the immediate talk in action and other discursive events constituted as texts.’o In both cases the overall mode of discourse is informal and textual meanings are encoded in explanatory accounts with participants positioning themselves subjectively and normatively vis-a-vis each other in seeking shared meanings. This.
rather than a reflexive and leaming tool to enable stakeholders to reconstruct their thinking. sectoral and disciplinary boundaries lag behind sophisticated technical and market frameworks. a prototype. should be considered an important ingredient in organic project cultures where partnerships are based on the autonomy of the ’expert’.
. The alignment of a technological and moral frame of reference appeals to the scientific episteme (structures of knowledge which enable us to view the world in a certain way). The process by which these data are gathered also indicates that without appropriate discursive procedures. This point is bome out in the following discussion. it becomes clear from the narrative structure of the discourse that as projects move through their lifecycles partners are led to recontextualize the ongoing process of innovation. All rights reserved. change becomes increasingly difficult to identify and channel in terms of stated objectives and goals according to the official texts which define the projects. This creates specific problems for projects in developing concrete projections for the future.com at INDIANA UNIV on July 9. is perceived more as a measurement or assessment tool for ’hard’ project outcomes. function as supranarratives circumscribing the factual or scientific world of innovation activities and what stakeholders may say in relation to its episteme. This is not only found in propositional claims and argumentation structures. A distinction is made between the ’profit-only’ agenda of stakeholders (interpreted as an indirect reference to big business interests) who require this type of evaluation (costfor the sake of the future of benefit. it is apparent from the discourse constructed that the term ’evaluation’ is internalized more as a measurement/assessment tool than a processual tool (identifying the process of decision-making over the lifecycle of a project. which are part of the intertextuality of the round-table discussions. then. Communication aimed at dialogical consensus. for example. The discourse on this theme follows official evaluation discursive procedures which distinguish between formative and summative evaluation.sagepub. but within the context of a set of a priori assumptions which communicate the notion that technological innovation is always socially desirable. 2008 © 1995 SAGE Publications.
Downloaded from http://evi. Not for commercial use or unauthorized distribution. Evaluation. in terms of the everyday realities. Yet. as if these were the only forms of evaluation. Summative evaluation is perceived to require stabilized features to evaluate and this is translated to mean the evaluation of outcomes. it creates problems for partners who have to transcend cultural and spatial distances in order to communicate.emerges as a salient concern in both round-table discussions. but in the manner in which these are framed. in particular the process of transforming individual action into collective action). for example. Outside what is internalized as the official requirements of evaluationrepresented by the audit metaphor-reflexivity is identified as a key process in understanding the change dynamic involved in processes of innovation. Because projects fail to transfer learning and knowledge from one context to another. therefore. would not fit this category. cognitive and cultural frameworks for new institutional alliances which seek to traverse national. These official texts. etc. A learning model which would enable a formal appraisal of actions such as decision-making processes is not part of the projects’ repertoire.) and stakeholders who participate in R&D technological innovation and knowledge (assuming more altruistic motives). that it is concerned more with project outcomes (outcome driven) such as products. While evaluation is not perceived to have a formative influence on development projects.
The communication model is implied in the salience of ’negotiated consensus’ as a theme at both round-table discussions. there is a cognitive level underpinning normative expectations which enable shared meanings to be created and communicated. You actually need real communication with your partners. These exemplars are taken from that part of a discussion thematically identified as
Downloaded from http://evi. All rights reserved. which is guided by a market frame of reference. the trainee and trainers or among the different trainees. 2008 © 1995 SAGE Publications. This appeal is most salient in the context of grievances which are articulated in relation to the governance structure of a project. so it’s the same in development like in the leaming environment. say.
Projects are integrated in terms of (1) the notion of collective risk taking. there must be a collective cognitive understanding about what is different about the innovation. While propositional claims found in the discourse indicate that stakeholders have different images of learning technology innovation. it’s the same here in the project. education. problems can be solved out. But they can actually communicate on line with each other. for example. and (2) the metaphorical constitution of learning technologies as ’The Future’ (see later). According to Tepper. In Project A. for example. and it’s not enough just to send an e-mail and then wait
answer. and participation in European transnational projects is constructed as a shared opportunity structure for future markets and European competitiveness.
. audio and video communications between. training and knowledge production are metaphorically constituted as ’markets’ through the use of commodity metaphors. I have chosen a few exemplars to illustrate this normative relationship between technology innovation and economic possibility. which appeals to the social desirability of technology innovation and development. The alignment of a cultural and moral frame of reference appeals to the notion of a project culture which presupposes a communicative rationale based on shared norms and consensus building. 1993: 337) or collective vision. the difficulty in ’marrying’ the technical and management coordination of the projects. say. because in the
system we say what we actually would like to have is. This is the level of ’leitbilder’ (Tepper. The ’information society’ and ’knowledge society’ metaphors create the context for a shared vision of ’the future’ and this is dominated by a market frame of reference. For example. the level of reflexivity which takes place towards the end of the discussion is indicated by the comparison drawn between issues pertinent to the leaming environment which technology development seeks to enhance and the need to develop appropriate cultural and social frameworks for technology development:
multiple training system. and over the course of the projects’ lifecycles these images may change. As soon as you spoke to the other table or on the telephone.com at INDIANA UNIV on July 9. Not for commercial use or unauthorized distribution. Both constructs are carriers of cultural messages about European identity and are dominated by a moral frame of reference.The project actors make a distinction between communication as a conduit for sending and receiving information (the transmission metaphor) and communication as a process of organizing.
the function of ’leitbilder’ is to create the notion of a shared vision and collective projection for the future application and diffusion of new technologies.sagepub. ’Know-how’ is a marketable commodity. Because change is the most fundamental concept of innovation. but if you just
have off line communication with fax and e-mail the communication sometimes gets stuck.
’The Future’ is a cognitive construction which defines the context of meaning... Now we have the experience how you use it and it took us quite a while to be able to handle it. if you buy this component..
come to us
and ask us. they are selling know-how. For example. propositional claims indicate that there is no clear objective for future developments. And that’s what they’re paid
for. it gives us the opportunity to work and handle the new and now we have the time for more than 2 years to handle the networking stuff and how to handle how to get across platforms. the construction of the collective agent is mediated by the grand metaphor: ’The Future’. But now we can tell people. this architecture. within this consortium with all the problems we ran into.’opportunity structures’ and where participants discuss the opportunities which have been opened up through their involvement in trans-European actions for innovation and competitiveness in the field of learning technology.. All rights reserved. or this component and if you program it like that. Drawing on both anecdotal and more systematic evidence generated as part of the
Downloaded from http://evi.
to the imperative of the economy and the post-industrial for new markets or market niches is used as both a cognitive and normative standard. the discourse indicates the need for reflexive learning to assess the difference between cognitive and normative frameworks for change. Hence.
our department. Similarly. Although future-oriented goals underpin stakeholders’ involvement in trans-European projects and serve as cohesive mechanisms at both project and program level. how
can we use
technology. and this is our new target market within the project’s community. especially for
. light sizing..com at INDIANA UNIV on July 9. The difficulties commonly experienced by project participants in such loosely organized research networks are qualified by a set of shared assumptions about what it means to be involved in European innovation.
this is going in the direction of out-sourcing. At the level of symbolic integration collective identities are bound up with the role of the ’expert’. okay. Not for commercial use or unauthorized distribution.sagepub.
. This creates a symbolic context for transnational cooperation which is rationalized in terms of building a European research profile to determine future lines of action for new technology and new institutional arrangements in the areas of education and training. that gives us special advantage to be competitive. but also using the PCs in order to address the small and medium size enterprises..
. character architecture and not only using the workstation. you have a good chance to be successful and that is 1 year or 2 years that they were doing their own work. The notion of cross-fertilization between expert domains carries a cultural value which resonates with the official cultural agenda of EU initiatives in information and communication technology. This kind of learning lies at the intersection between a futuristic vision and what stakeholders could hope to achieve within the context of their immediate actions. things like that. So it gives them a good chance to go to industry and tell people.. 2008 © 1995 SAGE Publications.
selling products. portability. look that’s the way we can do it and we have a chance to be successful and so. opportunity structures metaphorically constitute collective risk taking and this is dominated by a market frame of reference indicated by the discursive procedure of reification.
this issue of the user profile-the user being these sorts of designers. Their symbolic significance.broader evaluation study of which the discourse analysis is a part. All rights reserved. We are conscious that because the thing is constantly developing. many of the claims of R&D practitioners in the field of learning technology innovation about ’user involvement’-an issue close to the heart of social scientists and education evaluators-are dismissed as rhetoric. of their user group. particularly to end-users. we are conscious in a sense the people who are developing. networks create future opportunity structures. i.
The particular example occurs in the context of a discussion about the value of the ’user’ as a yardstick for measuring the success of one particular product over another. the user is both a discerning customer and a leamer. you know to confirm and to. or a modelling environment such as Lap View which is one we’ve mentioned. which is part of the technical lexicon of technology texts.. It appeals to the importance of commercial actors as both users and knowledge producers. at a conceptual level. There is little explicit in the discourse produced as part of this study which indicates that technology innovation projects are driven by user needs. The network concept is a metaphorical reference to loose organizations (less formal. so in other words those packages have been upgraded by the commercially.. say authorware or any similar package. Constructed to mean material resources. the network concept is part of the self-
Downloaded from http://evi. For example. As technical constructs. indicates that. however. The manner in which metaphors are used to convey the image of a specific innovation or the generic concept. At the level of the symbolic integration of the innovation projects. Even in the history of this project of two-and-half years. the symbolic meaning structure of ’networks’. it also appeals to the significance of the transfer of expert knowledge about markets to an R&D context. appeals to a vision of the future.e. commercial judgements of the people who make them to reflect the needs of that. But we supplemented that by doing interviews with three major companies and the sort of work they were doing to get a better. ’The Future’ is constructed as a family of images which function as interpersonal stabilizers at the level of project participation. Communicating a vision of the future for ALT innovations is part of the everyday discourse of innovators. metaphors such as ’virtual classroom’. ’workbench’. 2008 © 1995 SAGE Publications. is found in a cultural text which appeals to the notion of developing appropriate cognitive and social frameworks for new and future modes of knowledge production and delivery. technology innovation per se. it structures a collective narrative to create a focus for shared meanings to emerge.
. there have been two or perhaps three versions of those. user orientation is assimilated as a central feature and determinant of the future of learning technologies. quasi-institutional arrangements) such as trans-European projects. networks are communication media.
. However. but um and we had our own sort of experience and knowledge. Learners are discursively constituted as active subjects in technology innovation
through consumer metaphors:
And I mean we are conscious that things change quite rapidly in this area. ’network’ and ’gruyere cheese’ are used by the focus groups to configure and communicate a vision of the future of leaming technologies in terms of techniques and social structures which are familiar and accessible. Not for commercial use or unauthorized distribution. The centrality of the user is also indicated by the alignment of a market and pedagogical frame of reference.sagepub.com at INDIANA UNIV on July 9.
in one project the cooperative goal is used in appealing to the notion of ’partnership’ as something more than a means-end relationship. this company may get a better job
done than by spreading the money all over Europe. But these European projects are not for getting a job done. The discursive procedure of referentiality is used to indicate ’what is new’ by appealing to what is already known within the realm of human experience. This in turn appeals to the action-shaping influence of innovation actors and the relative autonomy of innovation activities.. 232
Downloaded from http://evi. processes of negotiation and integration are foregrounded as the key constituting organizational structures.. pressures might organizations. This creates the moral narrative. The cultural subtext of European cohesion is also appealed to in the use of the network concept. practical rationality is given precedence over technical rationality:
Yes. All rights reserved. . In other words. In one of the roundtable discussions a marriage analogy is used to exemplify an alternative organizational culture..
. intellectual stimulation that doesn’t have perhaps some of the overlay of company politics. we talked about it in the morning. well if we give all of the money just to one company. so I don’t have to cope with the politics of my boss and my
. but also for forming the links between the different partners.
In the absence of a hierarchical organizational symbol system where consensus is coerced to ’fit’ a managerial framework. i. 2008 © 1995 SAGE Publications.. This is achieved through the use of metaphors and analogies. For example.com at INDIANA UNIV on July 9.
professor. The grievances articulated in relation to the managerial and coordination problems which arise in such loosely bound organizational forms are redefined and transmuted by an appeal to the active involvement of partners in self-formation free from organizational constraints and managerial frames of reference.sagepub... At the level of symbolic meaning structures. or pedagogical claims about replacing ’chalk-and-blackboard’ techniques. the initial uncertainty which surrounds its invention and the realization of its potential social impact. example I come from a university background.. at one of the roundtables the social and technical background of the telephone is used to construct a narrative about the meaning of innovation.
project propositional claims about the difficulties involved in building a with partners who come from such disparate institutional and cultural backgrounds are redefined by one partner in terms of the unique intellectual experience which a partnership culture endows:
I would actually suggest that there’s an interesting sort of collaboration with colleagues around the table here. This is also found in propositional claims about the significance of being able to transfer knowledge across expert domains and using communication and information technology in place of ’paper-and-pencil techniques’. about the efforts we have spent in the project and we said. Or you know internal of that for exist in I mean. Not for commercial use or unauthorized distribution. For example.e. At the level of textual meaning the properties of a new system are explained in terms of a structure which is familiar and hence can be communicated. has a lot of stimulation . a collective European identity which is facilitated through a culture of technology. which is rather ..image of the projects.
I remember reading that when it was first invented. This is achieved through working directly with the research participants’ discourse and their understandings of the social context of learning technology innovation. The application of a discourse analysis design in an action-research context makes possible the opening up of discursive procedures and. opportunity structures. etc. oh! was it the telephone or some other device. facilitates the meeting of the theoretician and the practitioner in praxis. which as you know has been proved totally unreal. Not for commercial use or unauthorized distribution. 2008 © 1995 SAGE Publications. the communicative dimension acquires specific importance in cross-cultural temporally bound settings. in so doing.a moral message is encoded to suggest (1) the determinate nature of technology:
of the future and
But if you remember in the case of the telephone. idea formation and evaluation of experiences in terms of future actions.sagepub. This is not to underplay the ideological constructs found in the discursive constitution of technology innovation.
This interpretation does not claim to be a definitive reading of the texts it addresses. These form part of the narrative structure which. along with key frames of reference. because without shared knowledge and belief structures communication becomes impossible. when it was. The methodological approach attempts to develop a picture of technology innovation from the point of view of the stakeholder. In conclusion. Narrative structures may serve to valorize dominant images in order to negate or gloss over varying and possibly antithetical interests and. The identification of key frames of reference enables the analysis to reconstruct the supranarrative form as a discursive procedure which functions to encode ideological and cultural preferences. macro propositions emerge which constitute thematic links for discussing technology innovation in the institutional spheres of education and training in Europe. for example. the relationship between a learning model of evaluation and discourse analysis needs to be asserted. constitute a collective social identity (at least temporarily). hence.
Downloaded from http://evi. hence. Stakeholders operate from the point of view of the other stakeholders even in loosely bound and temporal organizational structures. The article is essentially concerned with the question of how shared understandings emerge in a context where institutional boundaries have yet to develop clear meaning formations. The analysis itself follows a hermeneutic model in emphasizing how stakeholders come to share a common appraisal of their actions and.
. it was agreed that this would be of no use except in a very limited area. All rights reserved. While claims and counterarguments are developed over the course of the round-table discussions. the author has set out to develop an argument for the need for processes of analysis which could explicate and invoke communication and reflexivity as a key referents of a learning evaluation approach in technology assessments.com at INDIANA UNIV on July 9. but rather to point out the need for the explication of discursive procedures which delimit processes of understanding and action. function as strategies of closure. For the purposes of this article. generate intersubjectively meaningful contexts of action.
com at INDIANA UNIV on July 9. All rights reserved. W. Textual signifiers such as modality and transformation have an ideological significance in the production and delivery of discourse. The seminal work of Halliday on social semiotics and textual analysis. (1992) ’Political Discourse Analysis’. Lincoln (1989) Fourth Generation Evaluation. 7.which legitimates the need for technology assessment.Notes
1. Klandermans et al. 1993) have provided useful analytical tools for developing a critical interpretative framework in this study. (eds) From Structure to Action: Comparing Social Movement Research across Cultures. (1993) Discourse and Social Change. Fairclough. Not for commercial use or unauthorized distribution. Official texts refer to documentation outlining projects’ contractual obligations under the DELTA Program. London: Sage.
. The Ethnograph software is described in Tesch (1990). (1981) ’Communication Processes in Studies of Social Organization: Socio-
logical Approaches to Discourse Analysis’. scientific discourse removes or mystifies the role of agents and processes. Guba. 2.sagepub. while project activities are involved in technologies and systems
developments. Donati. CT: JAI Press. Pilot activities are mainly concerned with experimentation and/or validation of existing technologies. London: Sage. and Yvonna S. London. Gamson. N. G. The Study reported in this article was undertaken as part of the overall ARTICULATE evaluation project sponsored by the Delta Office of the European Commission. Diani and R. and its use in an integrated model (see Fairclough. including universities. The official aim of trans-European cross-culturally to support European R&D to is transfer between institutions across the different Member States partnerships expertise as a means of increasing European market competitiveness. The task of developing a coherent evaluation framework and strategy for the program in question is carried out by a project consortia entitled ARTICULATE. passivization and objective modality (see Fairclough. 7) for a detailed description of the methodology of hermeneutic cycles as a constructivist inquiry. Pilot and project activities are distinguished according to two distinct purposes of the program. official events such as ’technical audits’ at the European Commission where projects are annually assessed in terms of meeting their contractual obligations. 4. 8. A. in B. 3. Greenwich. 6. 9. 10. and coordinated by the EDRU unit at the Tavistock Institute.
Corsaro. 2008 © 1995 SAGE Publications. Eyerman (eds) Studying Collective Action. The environment of the evaluand is theoretically described as ’uncertain’ because of the quasi-institutional context of innovation activities and the loosely bound organizational settings in which this activity is taking place. Trans-European partnerships are constituted by research institutions. W. 30. and public reports which are contractually commissioned for reporting progress and outcomes at intermittent periods over the lifecycle of a project. E. (1988) ’Political Discourse and Collective Action’.loosely associated with the concept of risk&mdash. This is marked by features such as nominalizations. in M. For example. P. small and medium size enterprises and large businesses from EU Member States working efforts.
5. Text: An Interdisciplinary Journal for the Study of Discourse 1(1): 5-62. See Guba and Lincoln (1989: Ch. A. In fact it is this uncertainty&mdash. 1993: 236). Cambridge: Polity. Sage Modem Politics Series Vol. which exists as a horizontal activity within the overall program. R.
Downloaded from http://evi.
Downloaded from http://evi. R. E. K. New York: Walter de Gruyter. A. Tesch. in T. A.
. Van Dijk. 2008 © 1995 SAGE Publications. Micromobilization and Movement Participation’. D. D.A. Burke. American Sociological Review 51: 464-81. Tepper. A. van Dijk (ed. and Text: Aspects of Language in a Social-semiotic Perspective. Oxford: Blackwell. A. Hasan (1985) Language. T. Benford (1986) ’Frame Alignment Processes. Not for commercial use or unauthorized distribution. Vol. Schiffrin.. Her research experience ranges from projects in telematics through to the field of biotechnology. Her interest in qualitative methodologies and action research has led her to focus on discourse analysis as a reflexive and communicative approach to social
enquiries. Oxford: Oxford University Press.
Halliday. Snow.) Discourse and Communication: New Approaches to the Analysis of Mass Media Discourse and Communication. (1993) ’Future Assessment by Metaphors’. van Dijk (ed. New York: The Falmer Press. (1990) Qualitative Research: Analysis Types and Software Tools.sagepub. University College Cork. bioethics and the construction of public discourses. All rights reserved.com at INDIANA UNIV on July 9. in T.) Discourse Analysis in Society: Handbook of Discourse Analysis. R. (email: STSO8006@IRUCCVAX. K. Gunther (1985) ’Ideological Structures in Discourse’. (1985) ’Introduction: Discourse Analysis in (Mass) Communication Research’. Behaviour and Information Technology 12(6): 336-45. S. Context. and R. Kress. 4. A. London: Academic Press. critical sociology and feminism. D. Ireland. M.
O R L A M O R R I S S E Y is a member of the research team at the Centre for European Social Research. (1994) Approaches to Discourse. Her academic interests are in qualitative methodologies. Worden and R.