• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • CommentGo Back
Download
 
8/28/09 10:51 AMMeasurability & Educational ConcernsPage 1 of 12http://www.newfoundations.com/EGR/Measurability.html
 
An earlier version of this article appeared in
 Educational Theory 24
(Winter 1974) pp. 52-60
Measurability and Educational Concerns
 by Edward G. Rozycki
RETURN
edited 1/24/09
Contents
Introduction: squaring the circleCriteria of MeasurabilityBehaviorOne Cannot Write Behavioral ObjectivesNon-PartitionablesVoluntary Behavior is not MeasurableTo ConcludeFootnotesReview & Discussion Questions
Introduction: Squaring the Circle
District's failure to write a measurable written language goal was not afatal procedural error and did not constitute a denial of FAPE
-- I.P. and Centennial, DP 00-115
[
1a
 
]The measurability of goals and outcomes has continued over the last half century to be a vexing problem foreducators at all levels of the enterprise. But it is not necessarily lack of technical skill that impedes theirefforts. Measuring some kinds of educational outcomes is very much like the effort to square the circle witha straightedge and compass: its failure derives from conceptual rather than technical issues.Thus, this essay is about what
can
and
cannot 
be
measured 
. It is intended to bear on problems of evaluation.By a
measurement procedure
I will mean a procedure the outcomes of which -- called
data
-- can bequantified.[
1b]
What will be said about measurability will bear on those evaluation procedures in whichmeasurement plays an important role. Our conclusions will be relevant to problems of curriculum and goalevaluation; but they will also be shown to be important in answering the more general questions as to whatshall be taught and how. This is because certaincurricular optionsderive from mistaken notions about thenature of human behaviorand of educational concern. To indicate the directions we will take, let us consider a statement from Robert E. Stakes' "The Countenanceof Educational Evaluation":. . . the responsibility for describing curricular objectives is (that)... of the (curriculum)evaluator. . . it is his responsibility to transform the behavior of a teacher and the responses of astudent into data. . . (Also) it is his responsibility to transform the intentions and expectations of an educator into "data."
[2]
 Note that Stake distinguishes the "data" of intentions and expectations from the data -- one can almost hear"hard data"-- of teacher-pupil behavior. This is a futile distinction. If by "data" is meant "the outcomes of ameasurement procedure, then educationally relevant teacher-pupil behavior does not provide any firmer datathan intentions and expectations.According to Paul Whitmore,
 
8/28/09 10:51 AMMeasurability & Educational ConcernsPage 2 of 12http://www.newfoundations.com/EGR/Measurability.html
The statement of objectives of a training program must denote measurable attributes observablein the graduate of the program, or otherwise it is impossible to determine whether or not theprogram is meeting the objectives.
[3]
Unhappily, this statement is not merely false; it is preposterous. These introductory remarks are based on theassumption that the quoted authors, when speaking of data and measurement, mean what we have set themout to be above. Only this "hard Science" usage can invest their claims with an interest commensurate withthe self-assuredness with which they have been advanced. It will be our endeavor to demonstrate that suchclaims are founded on grand-scale confusion.
Criteria of Measurability
Our criteria will be simple, in accord with our common sense, but -- where possible -- technically accurate.Roughly stated, our first criterion is this one must be able to unambiguously classify and tally the simpleoutcomes of an alleged measurement procedure. We can state criterion #1 more formally as
#1. The Criterion of Partitionability
: Simple outcomes must he classifiable into categories --sometimes called "elementary events" -- which are both mutually exclusive and exhaustive of all possible kinds of outcome.
[4]
 For example, if we know that an outcome is of type A, we must know that consequently it is of no othertype that contrasts with type A. (It could belong to a sub-type X which is completely contained in A.) Also,all outcomes must be classifiable.Let us consider two examples of alleged measurement procedure, one of which meets criterion #1. For thefirst, someone is using a yardstick to assign lengths to different tables. The outcomes of his procedure areclassified as, say, x-inches, y-inches, etc. Every outcome is some number of inches. No outcome is both xand y inches. Criterion #1 has been met.
[5]
The second alleged measurement instrument is the procedure for using the Amidon-Flanders Categories of Interaction Analysis.[
6]
All teacher verbal behavior in the classroom is categorized into one of eight types:1, accepts feeling; 2, praises or encourages; 3, accepts or uses ideas of student; 4, asksquestions; 5, lectures; 6, gives directions; 7, criticizes or justifies authority; 10, silence orconfusion.But if a teacher says to a student, "Are you always so jumpy?" this, treated as an outcome, may beclassified either as 7, or 4, or even 1. Thus, the interaction analysis categories cannot found a measure. (Thereader who is familiar with the Amidon-Flanders system may be able to raise some objections here, makingreference to certain procedures that are used to resolve this problem. We will re-examine this system laterand show that in fact the problem is not resolved).Criterion #2 -- again, roughly put -- will be that the identity of the object measured must not be destroyedby performance of the alleged measurement procedure, e.g. no identity criterion for the object measured canbe incompatible with criterion #1.
#2. The Criterion of Construct Validity
: Given an alleged measurement procedure, the identityof the (possibly hypothetical) object to be measured must remain invariant through (repeated)
 
8/28/09 10:51 AMMeasurability & Educational ConcernsPage 3 of 12http://www.newfoundations.com/EGR/Measurability.html
performance of the procedure.
[7]
 Obviously, measuring the length of a table with a yardstick does not change the table. The way we identifythe table in the first place --although informal (Could it be otherwise?) -- is independent of the procedurefor measuring its length. The way we identify teacher-behavior in the Amidon-Flanders system may obscurethe identity of what is being measured, for depending on whether "Are you always so jumpy?" is classifiedsyntactically, in terms of teacher intent or in terms of student-uptake, we may get different classifications.What loses its identity here is what the teacher did; that he "category-7-ed" may not tell us. (This is not thediscussion promised above; we are not yet finished with Amidon and Flanders)Criterion #2 must not be taken to mean that the object to be measured must remain intact physicallythroughout the measurement procedure; only that it identity -- which is not a "physical" thing -- remain so.The identity is preserved if the proper historical relationship between the outcomes and the object ispreserved. The pieces of ash being weighed on the scale must be known to have come from pieces of thespecimen to he so measured.
[8]
Suppose we wanted to use some form of the Iowa Tests of Basic Skills to measure scholastic achievementfor the population of a certain school. However, we devise the following procedure: We give out answerblanks to every student entering the lunchroom telling him that if he hands in a filled-in form at thecafeteria counter, he will receive a free lunch. What is to be measured is the individual pupil's achievementin certain basic skills. Suppose further that we receive back a filled-in answer sheet from each studentwhich in fact correlates well with his grade-point average; repeating the procedure we establish thereliability of the measure. We are nonetheless still disinclined to say we are measuring individual scholasticachievement -- especially if not every student need fill in his own answer sheet. We might with justificationsay we were measuring something, but it would certainly not be individual scholastic achievement.Problems of construct validity only arise where we have a tradition of identification that is independent of aparticular measurement procedure. We needn't worry about it for, say, the Stanford-Binet if all that suchtesting were being used for were its predictive validity, e.g. to indicate the probability of future academicsuccess. What Stanford-Binet measures does not seem to be identifiable independent of the test. And forpurposes of prediction it does not matter. But, as concerns types of human behavior, we do have ways of identifying them independently of procedures that purport to measure them. And our educational and moralconcerns derive entirely from these traditional ways of classifying human behavior.
[9]
As educators, we are interested in voluntary rather than reflex behavior. We are concerned withresponsibility, self-control and intention. We can identify behavior which exhibits these qualities with fairlyhigh reliability. If certain measurement procedures cannot, this still gives us no cause to disavow ourinterests.
[10]
To sum up: if for some object, criterion and criterion #2 cannot both be met, then it is
in principle
not measurable. Identity conditions must be compatible with partitionability requirements.
Behavior
There are few concepts more confused -- and confusing -- than behavior. Robert F. Mager. for example,defines it as "overt action" but gives no criterion for overtness.
[11]
B. F. Skinner insists that behavior "mustbe described in physical terms,"
[12]
but problems arise exactly at that point where one tries to determinewhat the word "physical" restricts one to. D. O. Hebb offers perhaps the clearest definition in this tradition:"Behavior is the publicly observable activity of muscles or glands of secretion as manifested in movementsof parts of the body or in the appearance of tears, sweat, saliva and so forth."
[13]
Behavior by this
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...