Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Kill The REF in Complex Circumstances

Kill The REF in Complex Circumstances



|Views: 4,185|Likes:
Published by Bill Cooke
This is a critique of the operation of the HEFCE Research Excellence Framework 2014 by Bill Cooke
This is a critique of the operation of the HEFCE Research Excellence Framework 2014 by Bill Cooke

More info:

Published by: Bill Cooke on Jun 20, 2012
Copyright:Attribution Non-commercial Share Alike


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF or read online from Scribd
See more
See less



This is personal opinion. If citing please attribute. drbill.cooke@gmail.com
Bill Cooke: in a personal capacity
There are rumours going round British universities that some are making explicit promotion,
teaching workload, hiring, and firing decisions based on the “Research ExcellenceFramework”, the REF. This is odd, because the REF hasn’t actually happened, and th
ere is ahuge and expensive clustering of committees, rules, and guidelines, never mind a process of detailed assessments designed by HEFCE that have yet to take place. One might think that if decisions on the outcome of the REF of such import to individual careers can competentlybe taken before it
actually happens, then the REF per se isn’t needed. Or, conversely, thatactually, these decisions can’t be competently taken. What I go on to show here is that onHEFCE’s own terms, they can’t be taken ethicall
y, and perhaps legally either; but also thatthe REF itself is similarly broken.The stated intention of the REF is to distribute money from HEFCE to universities forresearch, on the basis of their research performance hitherto. Past performance is taken asan indicator of the future. As institutions get money on this basis, it should be a self fulfillingprophecy. Unlike OFSTED inspections though, there is no assessment of what an institutionhas made of the resources it has had
(no “value add” measure).
To make this resourceallocating judgement, university research is compared by various disciplinary
“Units of Assessment”
(UofA). What gets compared are, largely,
research outputs
mainly, butnot exclusively peer reviewed journal articles and monographs, with some variationsbetween disciplines. For HEFCE, institutions comprise agglomerations of their individualresearch staff, deemed
. Assessment is per capita; each research activeperson is counted,
and each person’s
best” publications
counted up and each of the four is evaluated by being given a score from 1 (bad) to 4 (best). Only category 3 and 4work attracts institutional funding. You might think that having your work categorised as
being of “quality that is recognised internationally in terms of originality, significance andrigour” puts you on a solid basis.
if that is all your work is, it is awarded a “2”. To get a“3” work needs to be of quality “that is internationally exc
ellent in terms of originality,
significance and rigour but which falls short of the highest standards of excellence.”
A “4”,
the holy grail, is
quality that is world-leading in terms of originality, significance and
This is personal opinion. If citing please attribute. drbill.cooke@gmail.com
I know that these categories are not mutually exclusive, or logically related, but Ihave to go on to my bigger point.
There is window dressing about the “research environment”
, but this is essentially how itworks. As those of us who teach organization studies try to explain, there are ways of assessing organizational performance other than agglomerating the performance of individual organizational members, and there are choices about which is best depending oncontext. The example I use is professional football, which is apparently highly individualized,and highly competitive. It is tautological, but despite this competitive individualism, thesuccess of a team can only be assessed by its success as a team, that is, by its collectiveachievement. Were it assessed on the organizations-as-individuals logic employed by theREF, each team member
s performance would have to be assessed on necessarilyindividualizable criteria
for passes completed, or time in possession of the ball. Thenwhich team is
would be judged by summing up each player
s counts for the chosencriteria. Indeed, a couple of days ago, in the UEFA Euros tournament (I write this the dayafter England beat Ukraine), the Guardian did just this, headlining data which showed inEngland v France, England players completed a smaller percentage of their passes than theFrench players. The England the team was therefore facing serious problems, and theFrench were the better side. But the actual result of that game was that England and Francedrew: neither could claim to be the better team on the only criterion that mattered, thenumber of goals scored. And, at the time of writing, England have topped their UEFA groupmini-league, with more points than France. When it came to doing what they weresupposed to do, England were the better team.HEFCE and the REF
’s primary mode of analysis is
simplistic for its context, and causesunanticipated problems. How could such clever people at HEFCE make such an elementarymistake ? The general answer is the tendency of technocratic managerial experts to hubris,as if there is nothing that cannot be tamed by them.
The specific answer is that HEFCE’s, and
university ad
ministrators’ eyes got too big for their managerial bellies. It became apparent
that the REF could be used to shift the balance of power between administrators
who theTimes Higher
now styles “university leaders and managers” –
and those who actually do thework of research and teaching. Academics were famously difficult to manage. Much of theirwork was private, in the classroom, the study, the laboratory or the archive; so much of 
This is personal opinion. If citing please attribute. drbill.cooke@gmail.com
what they did was heterogeneous, intangible, and politically inconvenient; and so much of its significance takes years to be realized. In addition,
academics also didn’t want to be
managed, and more than an insignificant few
didn’t do the work for which they collected
salaries way above any national average.Along with customer satisfaction surveys for lectures, an individualized, a per capita REFenables (it seems)
the atomization of universities’ intellectual endeavour, and, the
performance of each faculty member to be exposed, measured and judged, on its ownterms, and against that of others. This is helped by the techniques of human resourcemanagement (HRM)
probationary periods, target setting, annual performance appraisals,and performance related pay. HRM is also associated with disciplining, by both itsproponents, and its Foucauldian critics. But against those who see mutual cause and effectin the simultaneous rise of HRM and Thatcherism, I counterpose my anecdotal experience of its practices being developed and realized in public sector by the political left. When Iworked for Leeds City Council in the 1980s, advocates of what now are seen as everydayHRM tools
job descriptions, person specifications, annual appraisal
presented them asobjective methods to counter racism and sexism. It was the stroppy, brave people fromcouncil
s’ and health authorities’
equal opportunities units who drove the HRM movement,as much as it was privatising, union-busting right-to-managers. When I left Leeds for myfirst proper HRM job at British Telecom in London, I found most of my colleagues developingits new entrepreneurial culture had a similar public sector background.This ambiguity, of processes which are managerialist, anti-collective, even oppressive, andat the same time potentially progressive, pervades both the REF, and academics
 engagements with it. The whole evaluation of submissions is by panels of disciplinary peers,resulting from the collective, if tacit, understanding that it is better done like this than anyother way. Of course, this peer review is not anonymous
the reviewers know who haswritten what. How panel members solved the puzzle of doing the actual quantity of readingwould be of interest to Booker judges. In the last RAE, in my UofA, 18 people apparentlyread and ranked 12,590 research outputs. Most journals have a word limit of 8,000 words(often exceeded), and each piece really should be difficult to grasp on one reading. That

Activity (17)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
Bill Cooke added this note
this is now available, along with an earlier piece about business schools on www.criticalfaculties.org
Bill Cooke added this note
Someone - somehow - has had this classified as containing "mature content". Sabotage, or what ?
Bill Cooke added this note
I am really intrigued to see the number of reads this has had. I am thinking of a follow up in the next few weeks, entitled "10 Things We Can Do About the REF". Am not sure what they are, so if anyone has ideas, post them here, or email me - drbill.cooke(at)gmail.com Thanks for tweeting, facebooking, and circulating the .pdf to your listservs - keep going pals !!!
Katie Best added this note
This touches on so many important issues that we all feel sore about but does so in a way which doesn't just sound like wingeing. Which is refreshing! No mention of the challenges of being an Early Career Researcher, though: a category which lots of universities seem unable to fathom, deal with or in some cases even to recognise
Bill Cooke added this note
Technical issue: you shouldn't have to login to read this. Not sure about downloads, though. Happy to send .pdf of latest version (I am correcting typos as I spot them) on request. Bill
Jonathan Stephen Davies added this note
Excellent piece Bill. I wonder if the union could be persuaded to investigate the legal questions?
Bill Cooke added this note
Sorry about the word splitting in places - production issue I can't resolve. Welcome comments - here, facebook, twitter, listservs. As is written in personal capacity please use my personal email to correspond if you want to - drbill.cooke(at)gmail.com Peace and Love. Bill
Bill Cooke liked this

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->