MODULE 7

EVALUATION MODELS IN EDUCATION

INTRODUCTION
Curricular evaluation is a system process of inquiry. The aim of investigation is to determine whether the curriculum as designed and implemented has produced or is producing the intended and desired results. Thus, evaluation models in education as tools for decision making go beyond measurement for it requires judgment on the part of the educator. Hence, the crucial role of evaluation in the success of any curricular reform effort is axiomatic and indisputable and no reasonable evaluation can be made without appropriate measurements done (Posner, 1995). Evaluation can be conducted as formative, summative, or both. Formative evaluation is a way to detect problems and weaknesses in components in order to revise them. In projects with sufficient time and funding, formative assessment is conducted prior to the implementation of the final program. In practice, many projects begin with the best effort and conduct a formative evaluation with implementation, correcting weaknesses and errors as the project unfolds. Summative evaluation is a process that concerns final evaluation to ask if the project or program met its goals. In both types, the media or instructional program can be evaluated, but typically the summative evaluation concentrates on learner outcomes rather than only the program of instruction. Traditional tests and other evaluation methods commonly employed in classrooms are used in both instances, but specific kinds of evaluation can be used in formative evaluation. Records, observations, interviews, and other data will permit the use of qualitative analysis of information for formative and summative evaluation. General Objective To know the different models of curriculum evaluation.

Specific Objectives. After reading this module, you should be able to: 1. Define the term, evaluation.
2. Know and understand the meaning of program or curricular evaluation.

3. Identify the different major evaluation models in education. 4. Know the characteristics of an evaluation instrument 5. Answer the questions found at the last page of this module.

Models of Education Evaluation

1. Scrivens Model 2. Stakes Congruency-Contingency Model 3. Stufflebeams CIPP Evaluation Model 4. Tyler Evaluation Model 5. Provus Discrepancy Evaluation Model Scrivens Model The Scrivens Model (1967) relates to issues such as entity, goals or weightings. He defines curriculum evaluation as gathering and combining performance data with a weighted set of goals scales to yield the comparative or numerical ratings and in the justification of: a. data gathering instruments b. the weightings c. the selection goals

Stakes Congruency-Contingency Model

Robert Stake (1988) proposed the congruence-contingency model. The congruence or matching occurs between the intended and the observed data in terms of 3 major areas: 1. Antecedents 2. Transactions 3. Outcomes The analysis is based on the matching of what has been planned and what has actually occurred. Stake defined an antecedent as any condition (student profiles, teacher profiles performance record etc.) that exists prior to teaching and learning that may affect outcomes. Transactions refer to the interactions refer to the interactions between and among students and teachers and the different dimensions of the learning environment.

Outcomes are the product of instructions or the consequences of education. Outcomes are the product of instruction or the consequences of education. The desirable outcomes are the development of knowledge, skills, attitudes, values and habits among the learners. In addition, other consequences such as the impact of new programs on the perceptions and attitudes of teachers and administrators are also analyzed. In the system analysis vocabulary, the term ANTECEDENTS, TRANSACTIONS and OUTCOMES used by Stake in his model correspond to the words INPUTS, PROCESSES or THROUGHPUTS and OUTPUTS, respectively. The model is shown below:

Congruence-Contingency Matrix of the Stake Evaluation Model

Type of contingency Intended Data Observed Data Type of Contingency

Antecedent

Congruence

Antecedent

L O G I C A L

Contingency
Transaction

Contingency
Transactional

Congruence Contingency
Outcomes

E M P I R I C A L

Contingency Congruence
Outcomes

Stufflebeams CIPP Evaluation Model According to Stufflebeam (1982), evaluation is undertaken for the purpose of acquiring fundamental knowledge about the program, making decisions or judgments, getting data or information as the basis of the program planning intervention. The Phi Delta Kappa National Study Committee on Evaluation, chaired by Daniel L. Stufflebeam, produced and disseminated a widely cited model of evaluation known as CIPP (Context, Input, Process, Product) model. Comprehensive in nature, the model reveals types of evaluation, of decision settings of decisions, and of change. In shaping their model, Stufflebeam and his associates defined evaluation in the following way: Evaluation is the process of delineating, obtaining, and providing useful information for judging decision alternative. Stufflebeam clarified what was meant by each of the parts of the definition as follows: 1. Process. A particular, continuing and cyclical activity subsuming many methods and in using a number of steps or operations. 2. Delineating. Focusing information requirement to be served by evaluation through such steps as specifying, and explicating. 3. Obtaining. Making available through such processes as collecting, organizing, and analyzing, and through such formal means as statistics and measurement. 4. Providing. Fitting together into systems or subsystems that best serve the needs or purposes of the evaluation. 5. Useful. Appropriate to predetermined criteria evolved through the interaction of the evaluator and client. 6. Information. Descriptive or interpretive data about entities (tangible or intangible) and their relationships. 7. Judging. Assigning weights in accordance with a specified value framework, criteria derived there from, and information which relates criteria to each entity being judged. 8. Decision Alternatives. A set of optional responses to a specified decision question. The evaluation process, said Stufflebeam, includes the three main steps of delineating, obtaining, and providing. These steps provide the basis for a methodology of evaluation. In the flow chart form the model which consists of rectangles (with small loops attached), hexagons, ovals, a circle, a fancy E, solid and broken lines with arrows and three types of shading. Crosshatched, the hexagons show types of decisions; hatched, the ovals, the circle, and the big E depict activities performance; and mottled, the rectangle stands four types of evaluation (Figure 5).

The four types of evaluation . The Phi Delta Kappa National Committee pointed to four types of evaluation: Context, Input, Process, and Product, hence the name of the model, CIPP.
Context evaluation is the most basic kind of evaluation. Its purpose is to provide a rationale for determination of objectives. At this point in the model, curriculum plannerevaluators define the environment of the curriculum, and determine unmet needs and reasons why needs are not being met. Goals and objectives are specified on the basis of context evaluation. Input evaluation is that evaluation of the purpose of which is to provide information for determining how to utilize resources to achieve project objectives. The resources of the school and various designs for carrying out the curriculum are considered. At this stage, the planner-evaluators decide on procedures to be used. Process evaluation is the provision of periodic feedback while the curriculum is being implemented. It has three main objectives: the first is to detect or predict defects in the procedural design or its implementation during the implementation stages. The second is to provide information for programmed decisions, and the third is to maintain a record of the procedure as it occurs. Product evaluation, the final type, has its purpose to measure and interpret attainments not only at the end of a project cycle, but often as necessary during the project term. The general method of product evaluation includes devising operational definitions of objectives, measuring criteria associated with the objective of the activity, comparing these measurements with predetermined absolute or relative standards, and making rational interpretations of the outcomes using the recorded context, input, and process information. Four types of decision. The hexagons represent four types of decision: Planning, Structuring, Implementing, and Recycling. Note that planning decisions follow context evaluation; structuring decisions follow input evaluation; implementing decisions follow process evaluation; and recycling decisions follow product evaluation. Three types of changes. In these settings three types of changes may results: neomobilistic, incremental, and homeostatic. Neomobilistic change occurs in a setting in which a large change is sought on the basis of low information. These changes are innovative solutions based on little evidence. Incremental changes are a series of small changes based on low information is so rare that it is not shown on the CIPP model . Homeostatic change goes back to structuring decisions.

The model plots the sequence of evaluation and making from context evaluation to recycling decisions. The committee has touched up the model with small loops that lock like bulbs on the evaluation blocks to indicate that the general process of delineating, obtaining, and providing information is cyclical and applies to each type of evaluation. The ovals, the circle, and the E in the model represent types of activities, types of change, and adjustment as a result of the evaluations made and decision taken. The CIPP model presents a comprehensive view of the evaluation process. Said the Phi Delta Kappa Committee, To maximize the effectiveness and efficiency of evaluation, evaluation itself should be evaluated the criteria for this include internal validity, expervasiveness, timeliness, and efficiency.

The Tyler Evaluation Model No description of evaluation models would be complete without including the contributions of Ralph W. Tyler who developed the objectives-based evaluation model. The essence of the Tylerian model is that evaluation consists of the measurement of whether the objectives of an educational program, project, or product are accomplished, while the idea may seem simplistic today. Tyler changed the focus of evaluation fifty years ago from the measurement of the inputs into an educational program (e.g., counting how many books are in the school library and listing the degree held by the teachers) to a comparison between a programs objectives as stated and what was actually achieved (e.g., what were the goals of the new reading curriculum and how well do students actually read after completing this curriculum).
Tyler proposed the following sequential steps in conducting evaluation: 1. Establish broad goals or objectives 2. Classify objectives 3. Define objectives in behavioral terms 4. Find situation in which achievement of objectives can be shown 5. Develop or select measurement technique 6. Collect student performance data; and 7. Compare data with behaviorally stated objectives Clearly, the focal point of the Tyler model is on the objectives. Thus, the effectiveness of the approach depends on the establishment, classification, and definition of objectives, which become the basis of data collection and analysis. The model suggests a dynamic, cyclic process where the data and information provide the necessary feed back to

the evaluator on the need to refine or formulate objectives. Modifications and adjustments of the objectives enable the system to function optimally. Below is the illustration of the Tylers model.

Tyler Seven Steps Evaluation Model

1
Establish Objectives

2
Classifying Objectives

3
Define Objective

4
Select indications

7
Analyze Data

6
Collect Performance Data

5
Develop Measurement Techniques

Report

The Provus Discrepancy Evaluation Model

The Discrepancy Evaluation Model (DEM), designed by Malcolm Provus and reflected in the CIPP model, is an effective way to evaluate academic programs. Provus defined evaluation as the process of agreeing upon program standards, and using discrepancy between outcomes and standards to evaluate the program. As such, the DEM can be used as a formative assessment to determine whether to revise or end a program.

The model is ideal for finding problems by means of discrepancy. It can also be used to design a program from inception to conclusion, leading to summative evaluations. In recent years there have been many approaches to program evaluation based on discrepancy analysis of Malcolm Provus in the 1960s, but with his untimely death and his books out of print, the testament to this heritage lives on primarily through the work of Daniel Stufflebeam and his associates at The Evaluation Center. While the DEM is called an evaluation method, Scriven considers the term evaluation to be inappropriate and seems to prefer monitoring as a more appropriate description. Alter described discrepancy evaluation as follows: The Provus Discrepancy Model designed by Malcolm Provus in 1969, is a well tested and commonly accepted utilitarian model to use in evaluating academic programs. He defined evaluation as the process of agreeing upon program standards governing that aspect of the program, and using discrepancy information to identify weaknesses of the program. His stated purpose of evaluation is to determine whether to improve, maintain or terminate a programHis model is primarily a problem-solving set of procedures that seeks to identify weaknesses (according to selected standards) and to take corrective actions with termination as the option of last resort. The DEM uses stages and content categories to permit comparisons. The stages are: Stage 1. Program Definition. To asses program design by first defining the necessary inputs, Stage 2. Stage 3. processes, and outputs, and then, by evaluating the comprehensiveness and internal consistency of the design. Program Installation. To assess the degree of program installation against initial standards. Program Process. To assess the relationship between the variables to be changed and the process used to effect the change. Stage 4. Program Product. To assess the design of the program to se if it reached its goals (objectives, outcomes). At each of the four stages, the defined standard is compared to actual program performance to determine if any discrepancy exists. The use of discrepancy information always leads to one of four choices. Proceed to the next stage of evaluation if no discrepancy exists.

If a discrepancy exists, recycle through the existing stage after there has been a change in either the programs standards or operations. If number 2 cannot be accomplished, then recycle back to stage 1 program definition to redefine the program, then begin the discrepancy evaluation again at 1. If number 3 cannot be accomplished, terminate the program. The DEM is most effective under the following circumstances : a. When the type of evaluation desired is formal, and the program is in the formative, rather than summative stages. b. When evaluation is defined as continuous information management addressing program improvement and assessment, and where evaluation is a component of program development. c. Where the purpose of evaluation is to improve, maintain or terminate a program. d. Where the key emphasis of evaluation is program definition and program installation. e. Where the roles of the evaluator are those of facilitator, examiner of standards, observer of actual behaviors, and design expert. f. When at each stage of evaluation, program performance is compared with program objectives (standards) to determine discrepancies. g. Where the program evaluation procedure is designed to identify weaknesses and to make determinations about correction or termination. h. Where the theoretical construct is that all stages f programs continuously provide feedback to each other. i. Where the criteria for judging programs includes carefully evaluating whether: (a) the program meets established program criteria, (b) the actual course of action taken can be identified, and (c) a course of action can be taken to resolve all discrepancies. stage

The Model suggests that in each of the five stages (as shown below), it is necessary to compare the performance with the identified standards. As is true with the other models

under the scientific category, the application of the Provus evaluation paradigm relies to a great extent on the identification of appropriate performance standards. A clear set of standards enables the evaluator to make judgment on what is valuable and what is not. However, determining the criteria on which to build the standards for evaluation is often problematic. Often times, viewpoints differ on what and whose values or standards are applicable and suitable in a given situation. The lack of agreement on the indicators of success sometimes discourages beginning and inexperienced evaluators to undertake program analysis.

Comparison of Performance and Standard Based on the Provus Dicrepancy Evaluation Model STAGE
I

PERFORMANCE
Design (space, personnel, resources, materials)

STANDARD
Design criteria

II

Installation (actual operation)

Installation Fidelity

III

Processes instruction, leadership

Process Adjustment

IV

Products (based on stated objectives)

Products assessment

Cost (economic and sociopolitical implications)

Cost benefit

CONCLUSION
Many more models of evaluation can be included in the list provided. In fact, every situation where a decision is to be made or where a policy needs to be formulated may require an evaluation process. For this reason, it is important that the student understands

the essential characteristics of a good evaluation system. In general, the steps involved in formulating a good evaluation model include: 1. Defining and identifying the objectives of the program or project to be evaluated. This serves as the starting point for the entire evaluation process. 2. Deciding on the parameters to be measured given the objectives. This is the second step which requires that the evaluator be conscious of the critical indicators for every objective stated for the program or project to be evaluated 3. Deciding on the measurement procedure to be undertaken. In this stage, the evaluators decide on the appropriate measurement device that needs to be developed in order to measure the required parameters of the evaluation model 4. Measuring the parameters of the evaluation model. 5. Validating the findings against the perceptions of experts, the people involved in the educational program, the public in general, and, more importantly, against the objectives set forth by program or project.

Questions to Answer
1. What is evaluation?, program evaluation? 2. Differentiate the two types of evaluation. 3. Name some other models of evaluation in education that you know. 4. Propose a title of study related to educational program assessment and make a conceptual framework for this using one of the discussed evaluation models.