release 1Paul Bacsich, BELA 2 21 December 2006
of these subcategories.
(An interesting question is whether “senior management” should be included
as a separate subcategory.)As a rule of thumb in MIT90s, there should be about two dozen criterion topics. (Theoretical justification of this comes from the Balanced Scorecard approach and from Activity Based Costing; amore operational justification comes from observation of scoring meetings in the BenchmarkingPilot.)Since the MIT90s categories are rather broad and not closely correlated to learning, it is often helpfulto ensure a balanced coverage also against other classifications, especially the categories of ELTI.
(There has been some so far inconclusive discussion about the lack of a “pedagogy” category in
Step 3: Generalisation of criterion topics
Since benchmarking has an irreducible element of comparison about it, it is vital to ensure thatcriterion topics are phrased in a way that could apply to other institutions. However, it may be thatyou wish to keep certain institutional-specific topics (these are then regarded as local criteria and noattempt is made to generalise these or correlate them to other institutions) and/or sub-institutionaltopics (these are also regarded as local criteria, but are specifically
but such local criteria should be in the minority. (In theBenchmarking Pilot for Pick&Mix, institutions decided not to use any local criteria.)
Step 4: Abstraction, normalisation and range restriction of associated metrics
It is very often the case that criterion topics derived from strategy documents have strong aspects of measurement to them, especially if they have been copied directly from KPIs (Key PerformanceIndicators). For example, an institution may have a KPI as to how many modules are on Blackboardin each of the next three years. This KPI has to be adjusted in two ways:
Replacement of the trade name by the general VLE
Normalisation of the metric to a percentage.
Thus the KPI becomes “percentage of modules mounted on the VLE”.
Then one should consider whether a higher (or in some cases lower) value of the metric is “better”.
Sometimes there is no sense in which it is; often there is a range
over which themetric is relevant but breaks down outside that (think of an amplifier turned up too high). As an
example consider the metric “cost of 1 credit of e
l”. Is a module with lower cost
better? Or just worse-looking?
Step 5: Creation of best practice statements
While there are circumstances where a metric is useful
for example when discussing reliability of acomputer system
it is much better in most cases to consider the criterion in a more holistic way
called “organic” or “process” view
, and regard the metric as contributing information not asfundamental.
This also puts the metrics in their place
which is why we use the word “associated”.
A best practice statement is designed to symbolise the best result that a typical organisation can plan
for. Note that one should probably use the phrase “better practice” but nobody does. Very often there
may be several variants of best practice and these should be noted. As an example, for a criterion topic
learning strategy”, the best practice statement might be:
We recently received information about a workshop by Luckin