Session T2G

AN EXTENDED METHODOLOGY FOR EDUCATIONAL SOFTWARE DESIGN: SOME CRITICAL POINTS

Fernando J. Lage1 , Yuriy Zubenko 2 and Zulma Cataldi3
Abstract This paper is part of a series of research lines related to the design, development and evaluation of educational software. These research lines intend not only to account for the problems faced by professors when attempting to incorporate computational applications to their educational practices, but also to provide them with a computational solution for their developments. In this particular case, the methodological aspect of the topic is considered to be relevant, since this is a crucial point to build a new product. This is why the activities matrix of the processes is presented, integrating pedagogical aspects to the conventional activities matrix, this integration being the central core of our proposal Index Terms Educational Computer Science, Educational Software, Software Design, Software Methodologies. possible in order to satisfy the curiosity of the client or user. Prototypes with increased functionalities will be gradually handed in, in order for the clients/users to try them during a period of time previously agreed on, and thus be able to incorporate their suggestions and changes during the early stages of the life cycle. On the other hand, it is important to know as soon as possible if the interpretation of the product by the developers is in agreement with the clients/users needs and considerations. In many cases, users cannot give a detailed idea of what they want. Therefore, developers do not know exactly what they are supposed to create, which makes each prototype a revision and refinement of requirements as a way of approaching the end product. The following stages are defined in the incremental prototype life cycle [12]: 1. Feasibility (FAC) 2. System requirements definition (RES) 3. Specification of prototype requirements (REP) 4. Prototype design (DPR) 5. Detailed design of the prototype (DDP) 6. Prototype development (codification) (DEP) 7. Prototype implementation and testing (IPP) Iterative refinement of prototype specifications (enlarging its aim and/or scope). If the desired aim and scope were achieved, it is possible to go back to stage 2. (RIT) 8. Design of the final system (DSF) 9. Implementation of the final system (ISF) 10. Operation and maintenance (OPM) 11. Withdrawal (if pertinent) (RET) Following there is a description of each of the stages of the selected life cycle that will be part of the activity matrix (1): Feasibility (FAC): This stage defines the software product and determines its feasibility in the life cycle from the viewpoint of the relationship cost-benefit, as well as the advantages and disadvantages as regards other pro ducts. System requirements (RES): The functionalities, interfaces and type of design required for the development of the system (or program) are defined at this stage. Specification of prototype requirements (REP): This stage consists in the specification of the functions, interfaces and output required for the prototype. Increments in

INTRODUCTION
A life cycle is proposed for the design of the educational software, and its selection justified. The stages of the cycle are considered and the activities matrix is defined. Based on this matrix, the tools and techniques to be used at each of the processes considered are described. The starting point for the proposal is a life cycle of evolutive prototypes with successive refinements. For the methodological stage of the design, the classic representation instruments provided by the cognitivistconstructivist approach are incorporated. The methodological proposal considers the construction of educational programs from an integral viewpoint, taking into account the pedagogical aspects of the life cycle. Special interest is paid to the configuration of the profiles of the different professionals of the development team, to the design of the program, and to the confection of the process documentation.

LIFE CYCLE SELECTION

The selected life cycle model is that of evolutive prototypes with successive refinements, due to several reasons: When the software is to be developed by request, it is desirable to have an outline of the program as soon as
1 2 3

Fernando J. Lage, Facultad de Ingeniera, Universidad de Buenos Aires. Paseo Coln 850, 4 Piso. Ala Sur, 1063 - ARGENTINA Informat@mara.fi.uba.ar Yuriy Zubenko, Universidad John Kennedy ARGENTINA Informat@mara.fi.uba.ar Zulma Cataldi, Facultad de Ingeniera, Universidad de Buenos Aires. Paseo Coln 850, 4 Piso. Ala Sur, 1063 - ARGENTINA Informat@mara.fi.uba.ar

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-13

Session T2G
percentages of the system overall functionality are considered here. Prototype design (DPR): The plan of the prototype plan is put into action; since once the restrictions are agreed upon with the users, they will expect to see at least some of the functions working. An analysis is required here in order to determine how the work is going to be done, what modules are going to be done, what logic is going to be applied, and which functions are going to be used. Detailed design of the prototype (DDP): This stage is a verified specification of the control structure, data structure, interface relations, size, basic algorithms and assumptions of each component of the program. In this stage, not only are the algorithms that will carry out the functions to be performed by each module defined, but they are also documented. Software design is a process centered in four different attributes of the program: data structure, software architecture, procedural detail and interface characterization. During this process, the requirements are to be translated to a software representation that can be established so as to obtain the required quality before coding begins. Prototype development (codification) (DEP): The codification or detailed design is carried out in a language the machine understands. Prototype implementation and testing (IPP): It consists in achieving a proper functioning of the software product in the computer system, operationally working, including objectives such as data and program conversion (if there were any), installation and training. The test must ensure that all of the sentences have been tested, and that all tests granting that the defined input actually produces the expected results have been carried out at the external functions. Iterative refinement of prototype specifications (RIT): this stage increases the functionality of the system by going back to the REP stage in order to increase prototype functionality or by moving on if the d esired objective and scope were achieved. Design of the final system (DSF): This stage consists in an adjustment of the final restrictions or conditions and an integration of the last modules. Implementation of the final system (ISF): It is the computer system operationally working, including objectives such as data and program conversion (if there were any), installation and staff training. Operation and maintenance (OPM): The computer system starts working; this objective is repeated for each update. Withdrawal (RET): It is a suitable transition between the functions carried out for the product and their successors. Next, the basic processes for this life cycle are defined, as well as the activities for each of them. Processes include those concerning software development and those, which are specifically related to education issues, although in the proposal there is no specific educational theory defined. However, the activities added to the activities matrix indicate a cognitivist-constructivist approach. Here the use of a theory with an active subject who builds its learnings from his own experience, the way Bruner did [3], must be considered; or that it can sustain the meaning and quality of what it had learnt since it incorporated the new knowledge to a solid structure of concepts already acquired [1], pointing out the difference between significative learning and mechanical learning. It can also be considered a sociological view [17] in which the building of meanings as a process of internalization implies mediation by the association of ideas that meanings are taken from the exterior, and the piagetian according to which the subject builds meanings in an autonomous way [13]. Table 1 present 23 processes considered to make the activities extended matrix for educational software. In Times Italic those processes are highlighted that have relative activities to preventive pedagogic aspects. In the extended activities matrix [5] they are added new processes with activities concerning the definition and consideration of educational, didactic and pedagogical aspects to develop the educational software and in some of the existing processes for the development of standard software, activities concerning educational aspects are added, and in some of the existent processes for the development of standard software concerning activities are added to consider educational aspects. Once each of the activities are defined, the tools and techniques to be used for each of them are still to be considered, taking into account the model of the life cycle selected (evolutive prototyping) and the educational theory behind the development. The following table shows the methods, techniques and/or tools to use for each of the different processes. In Table 1 are indicated the methods, the techniques and tools used more frequently in each of the processes of the activities matrix. They can be divided in technical processes: with the use of cost/benefit analysis. Pert, CPM, Gant, mo deling, DFD, DD, E/R diagrams, programming languages, modularity, prototyped, software testing, between other; and processes with pedagogical components: educational theories, metacognitive strategies, conceptual maps, use of the Ravens test [14], Wilcoxons test for small samples and others.

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-14

Session T2G
Processes Output document Methods/Techniques/ Tools to use
Interview, survey

Process of Adopted an Educational Theory Identification of the educational needs Adopted life cycle Process of life cycle model selection Process of initiation, planning and estima- Project negotiation plan tion of the project Risk analysis and contingencies plan. Process of project follow-up and control Historic register of the project (program), Process of software quality negoti ation Process of concepts exploration Quality guarantee plan. Quality enhancement recommendations. Needs report. Possible feasible. Solutions

Gantt or Pert diagram (CPM)(2) Estimation empirical models Modeling. Prototyping. Revisions. Audits. CPM analysis. Planning techniques. Software quality metrics Cost-Benefit analysis. DFD(3). Prototyping DFD Modules

Specification of hardware and software functional requirements. Process of program assignment (system). Specification of the interfaces of the system or program. Functional description. Architecture. Specification of the objectives and concepts structure. Process of educational requirements analysis Selection of contents and relevancy.

Cognitivist approaches. Constructivist approaches. Cognitive strategies. Specification of software requirements, user interfaces and Structured analysis. Process of software requirements analysis other software. Hardware interface and with the physical DFD. DD(4). E/R (5) diagrams. system. Prototyping techniques. Identification of mental processes to stim ulate. Use of cognitive strat egies. Definition of the activities to be carried out by st udents. Ausubel and Novak theory.[1] of the contents Concepts hierarchization. Use of conceptual maps. [11] Process of design Description of software design and architecture. Structured programming. of the software Description of information flow, bases, interfaces and y Object-oriented programming. algorithms. Prototyping techniques. Process of implementation and modules inte- Data for tests. Documentation of the system or program Programming gration and the user. Integr ation plan. Languages Installation plan. Programming Process of installation Installation report. Languages Support requests history Statistical analysis. Process of operation and support Process of maintenance Process of withdrawal Process of verification and validation Process of software prototypes evaluation Process of software internal and external evaluation Process of evaluation in context Process of configuration Process of technical documentation Process of didactic documentation Process of staff training Maintenance recommendations Withdrawal plan Verification and validation plan Test plan. test specification and summary. Tested software. Design of the evaluation instrument. Test summary. Sample selection. Design of the evaluation instrument. Test summary. Sample selection. Design of the experience. Experimental and control group s definition. Configuration negotiation plan Technical documentation plan. Didactical documentation confection plan. Training plan Reapply the life cycle Black box tests and white box tests Structured questionnaire, open and semi-open. Structured questionnaire, open and semi-open. Pre-post analysis techniques. Raven test. [14] Wilcoxon no parametric test. Data PERT and Gantt base diagrams

Table 1: Definition of the methods, techniques and/or tools to use at each process.
Professionals from the area to which the software development will be applied Professionals in software development Project coordinator Professors and specialists in pedagogy to determine the contents to be included and experts from the develo pment area Analysts and programmers. For project analysis and codification. As in every project supported by base engineering, this will be the task of the software engineer. Support technical staff (graphic design and sound) Depending on the dimensions of the development, there will be operators, graphic designers, sound and video specialists, etc.

Table 2: Professionals required for the development

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-15

Session T2G
THE WORK TEAM
The creation of educational programs is a task concerning many different areas. The formation and conformation of development teams is fundamental for this type of educational projects. To do so, it is necessary to consult specialists in software developments, efficient planners and professors familiar with the interest area for which the program will be developed, since a pedagogical model will have to be generated for each case according to its particular needs. Even though the conformation of the development teams for commercial software requires the presence of a good project leader and good relations between the members of the project, in this particular case the situation is critical due to the interdisciplinary nature of the team, centered on the task of integrating and coordinating professionals from different disciplines. Basically, the profiles required are shown in table 2. Some authors like Marqus [10] consider the software development project to be on an axis centered on the pedagogical team. It should be taken into account that the program to develop presents two aspects: the achievement of technical quality, which is the responsibility of the technical team and should be the main axis, and software design and development, which is the responsibility of specialists in the area. The determination of the tasks to assign to each of the development team members at each of the stages plays a crucial role when optimizing development times. It should be mentioned that in some cases it may be necessary to consult specialists as regards very specific computer and network technologies applied to the educational field, as well as psycho-pedagogues to advise the team when a product with a given characteristic is required. One of the aspects to take into account is a perfect definition of the activities to be carried out by each member. Knowing who does what, makes the identification of the responsible of each of the development stages easier, and allows an evolutive following of each of the software components during their development. On the other hand, depending on the proposed development model, there will be evaluations of the product where all the team members will take part. The second critical point appears when trying to define how would be the interaction with the software, because the question What does interaction means? arises. Interaction is produced by the mutual presence where events take place y by the complementary circularity where perceptions and cognition are modified being part of the presence and conduct of the other [8]. In this case between the program (mediator) and the user. In order to define the pedagogical interactivity that teaching materials allow, interactivity must be defined, which comes from inter that means between us and pedagogical activity which is to interfere or to interpose teaching actions for working out concepts or the development of competition, that allow to understand and to transfer to the action the essence of the implicated objects in order to act in an appropriate way [8] The pedagogical interactivity implies promoting the communication, which means to let the other to contribute and to play a leading role through the devising of didactical situations and hypermedial material for meaning exchange. This led us to design a context of resignificance-recreationanswer with computers and didactical software as mediators. These connotations must be interpreted by the programmers, with the aim of achieving the wanted interactivity in each environment. The third critical point is r lated with the concept of e quality and its polysemy. Educational programs have some very particular characteristics according to the curricular goals and the specific needs of the target group. That is why to specify the quality of educational programs is a process which consists of its grade of adaptation to a specific context where a series of variables converge, such as: curricular characteristics, target type, ages, teaching style, etc, and so require a proper analysis. To make a proper analysis the quality of an educational software must be evaluated according to three basic aspects according to this order: first pedagogical-educational aspect, communication aspect, and finally technical-economic.

FIRST DESIGN OF PROGRAM

The starting point is to detect the educational need for the development of the program; this can be achieved through an interview and by an opinion poll between teachers that use educational software. Starting from now an underlying educational theory must be chosen for the application: behaviorist, constructivist, cognitive, or others. Although many times it is the institution the one who states the educational theory to follow, many others it is determined by the curriculum or it is established by the teacher who later will use the program. In this design the adopted perspective is the cognitive constructivist, that is to say citing authors such as Ausubel and Novak [1], Vigostkii [17], Bruner [3], there is a review

SOME CRITICAL POINTS

The first critical points in the development of educational software is related with the need of a common language between the members of the team, the solution we have given to this problem is to have in the development team computer professionals with a teaching career and practice, using both characteristics to continue gathering the rest of the team.

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-16

Session T2G
of these authors in Cataldi &Lage [4] in reference to the production of learning materials. Afterwards, a good representation of knowledge must be obtained, where it is possible to use Novaks [11] conceptual maps as a graphics representation that will allow the incorporation of new information to the one already existent in a significative way. They can also be used nets of meanings that are a kind of conceptual maps with some characteristics of semantic nets. Specialists in contents design these nets or maps. In every subject or unit some things need to be very clear: the goals to achieve, the identification of concepts, their connections and the previous knowledges required. After that, the activities that lead to the attainment of the proposed learnings should be looked for, as a way of selecting the ones that are the most advisable for implementation with problematic situations looking for the strengthening of the superior thinking functions, as Vigotzkii [1] points out. Computer design allows to convert nets or maps of meanings into algorithmic structures, with actions and messages according to different events. To design the communications environment, and when the team has little knowledge about computer systems, the technique known as storyboard is commonly used. These series of sketches should be considered as a way of grouping the needs, and are carried out based on a previous agreement between team members. Expert software developers and human interfaces design specialists elaborate some prototypes of the screens for this purpose. Therefore, storyboards are not used here, which is not the case from the viewpoint of professors and programmers, who use them to interpret the requirements. by the students can be defined with increasing difficulty and complexity levels. However, this could be a questionable point, since if the software is to be used as teaching support, tasks and activities should be proposed by the professor by means of clear and precise instructions, and not by the program. Some authors consider the possibility of carrying out a subjective analysis, i.e., a general panorama of the task from the point of view of the task itself and of the subject solving it, as well as from the standpoints of difficulty level, structure, and organization and articulation with the objectives and possible solution paths. [15]. Valencia considers the analysis from an objective description of the instructions, as well as the analysis taking into account factors that present problems and/or make knowledge significant, from a cognitive point of view.

DOCUMENTATION
One key aspect to consider is the development of both internal and/or external technical documentation throughout the life cycle of the product. The former includes all those program comments that will be useful when carrying on modifications at later stages, be it by the same development team or not. On-line help for users needs also to be planned, and for this purpose - if it is to be implemented - a special module shall be developed. External documentation is all documentation referring to the material created since the initial stage of analysis, with the relations and entities diagrams, data structures, processes flow diagrams, descending mo dular design, etc., and all other information considered to be relevant for the interpretation of the development of the program. Finally, it should be mentioned that there is a need for a clear and didactic user's manual, to be used by the professor as as sistance. This manual should include all the technical aspects required for the program to work, and it can also include a solutions guide for frequent problems. A didactic manual or guide can also be made, in order to provide the professor with all the information needed to use the program. This didactic applications guide or manual should include objectives, contents, addressees, proposed activities, and it would be interesting to include which educational theory or theories were considered to develop the program, and what kind of treatment the program allows for the mistakes made by students during the learning process. The results of the evaluations carried out should also be considered, taking technical aspects into account - be these positive or negative -, and functionalities, with a detailed presentation of statistical results and taking into account the instrument created to collect those results and sample selection and type of those evaluating the product.

DESIGN
After the analysis of educational requirements follows the analysis of software r equirements. Then, the stage of the design of the educational software is one of the most important stages of the life cycle of this product, and specialists both in pedagogy and in contents interact with software design specialists from a comp utational point of view. The design stage can be divided into two large substages: the process of contents design and the process of software design. Later on, the contents design will be integrated to that of software, this being a key point in the process because cognitive strategies are tested here so that students have the possibility of acquiring knowledge by means of the different tasks proposed. A proper structure of the concepts is extremely important; topic, objectives proposed and activities should be correctly defined for students to be able to acquire knowledge. It could be said that both tasks and activities - these being the different problem situations related to the topic under study and which are presented to students to solve should be defined. In some cases, the tasks to be carried out

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-17

Session T2G
EVALUATION REFERENCES
The evaluation of a program designed according to the proposed methodology of extending the activities matrix, has been carried out. During the development students with similar characteristics to potential users, evaluated two prototypes as it is described in Cataldi & Lage [6] m aking the appropriates alterations and changes. The development team evaluated the program internally [2], whereas teachers and students evaluated the final product externally [2,6], being carried out the pertinent modifications then. Finally, in order to make a contextualized evaluation (in a context similar to the one for which was created the program) in a course of students counterpart pairs were made with the Raven [14] test of progressive matrixes. The aim of this was to obtain two groups: a control and an experimental one, working with two programs. The control group with a software that did not consider pedagogical aspects, and the experimental groups with the one developed with the e xtended methodology. It was then when we confirmed the thesis: that a software which considers the pedagogical aspects in his life cycle produces a better learning of the concepts, that one developed with a methodology that does not contemplate them. [7]. The results of this test through the use of the not parametric test by Wilcoxon [9] applied to the grades obtained by students, makes evident a significant difference in favour of the program developed with this methodology. Saying this it is not trying to validate the methodology but to repeat similar experiences [7]. [1]. [2]. [3]. [4].
Ausubel D., Novak J., y Hanessian H. (1997): Psicologa Educativa. Un punto de vista Cognitivo. Trillas. 3ra Edicin. Bork A. (1986): El ordenador en la enseanza. Anlisis y perspectivas de futuro, Barcelona, Gustavo Gili. Bruner J. (1988): Desarrollo cognitivo y educacin. Morata. Madrid. Cataldi, Z., Lage, F., Pessacq, R. y Garca Martnez, R. (1999): Revisin de Marcos Tericos Educativos para el Diseo y Uso de Programas Didcticos. Proceedings del V Congreso Internacional de Ingeniera Informtica. Pginas 172-184. Editado por Departamento de Publicaciones de la Facultad de Ingeniera. Cataldi Z., Lage F., Pessacq R. y Garca Martnez R., (2000a). Metodologa de Diseo y Desarrollo de Software Educativo. VI Congreso Internacional de Ingeniera Informtica ICIE 2000. 26-28 de abril de 2000. ISBN 987-98197-0-5. Pgs. 330-347. Cataldi Z., Lage F., Zubenko Y., Pessacq R. y GarcaMartinez R. (2000b): Evaluacin contextualizada de software educativo. CACIC 2000. Congreso Argentino de Ciencias de la Computacin. 2-7 de octubre, Ushuaia. Cataldi Z., Lage F., Pessacq R. y GarcaMartinez R. (2000c): Evaluation of educational software from an integral perspective. CACIC 2000, 2-7 de octubre, Ushuaia. Fainholc B. (1998): La interactividad en la Educacin a distancia. Paidos, Buenos Aires. Ledesma D. A. (1980): Estadstica Mdica. Eudeba Marqus P. (1995): Metodologa para la elaboracin de software educativo en Software Educativo. Gua de uso y metodologa de diseo. Barcelona Estel. www.xtec.es/~pmarques, www.doe.d5.ub.es Novak J. y Gowin D. (1988): Aprendiendo a aprender. Martnez Roca. Barcelona. Piattini M. (1996): Anlisis y Diseo Detallado de Aplicaciones Informticas de Gestin. Rama. Madrid. Pozo Municio I. (1994): Aprendices y maestros. Alianza. Raven J. C. (1979): Test de Matrices Progresivas. Escala General. Vol. 3b. Paids. Buenos Aires. Valencia M. E., Toro I. y Donneys C. (1998): Desarrollo de aplicaciones hipermedia: propuesta para el diseo educativo. TISE98. Co nsultado el 28/9/99 10 hs. en www.sofia,univalle.edu.co/gidse Vigotzkii L. (1978): Mind in Society. The development of higher psychological process. Cambridge. M. A. Harvard University Press.

[5].

[6].

[7]. [8]. [9]. [10].

[11]. [12]. [13]. [14]. [15].

CONCLUSIONS
It has been described the extended methodology for preventive the pedagogical aspects in the life cycle, for which specific learning activities were added. With the intention of contrasting a software developed with methodology a product was developed, it was evaluated and tested in a context ualized way. The results indicate that the students' learnings are significantly improved. This kind of results should be gathered in order to validate or refute the methodology through time. To produce educational programs it is required a good coordination between technical and pedagogical aspects, this leads to bear the critical aspects pf the project due to the lack of common reference frames between software engineering and the educational science. This leads to negotiation of meanings and the production of a common language to explain concepts such as interactivity and quality for educational software. Lastly, it is thought to develop as future investigation lines the extensions of other life cycles in order to adequate them to the educational needs, contrasting the efficiency of the products.

[16].

NOTES (1) The activities matrix for software design was adapted from Juristo Juzgado N. (1996): Proceso de Construccin del Software y Ciclos de Vida en m dulos de Proyectos de Software. Universidad Politcnica de Madrid. (2) CPM: Critical Path Method. (3) DFD: Data Flow Diagram (4) DD: Data Dictionary (5) E/R: Entity/Relation Diagram

0-7803-6669-7/01/$10.00 2001 IEEE October 10 - 13, 2001 Reno, NV 31 th ASEE/IEEE Frontiers in Education Conference T2G-18